datomic

Ask questions on the official Q&A site at https://ask.datomic.com!
jaret 2021-04-14T02:04:00.285300Z

I did not know that! Cool.

jaret 2021-04-14T02:06:21.285500Z

Yep you cannot do that on the same process. You can however make the choice independently per process. https://docs.datomic.com/on-prem/operation/valcache.html#vs-memcached

tatut 2021-04-14T12:29:09.287Z

has there been any news on cloud disaster recovery, the discussions donโ€™t seem active https://forum.datomic.com/t/cloud-backups-recovery/370/10 we currently have a hand rolled tx log backup/restore solution but itโ€™s a bit painful to maintain

kenny 2021-04-14T15:24:23.287200Z

Maybe you're aware already but there's a feature request on ask.datomic: https://ask.datomic.com/index.php/431/cloud-backup-and-recovery

kenny 2021-04-14T15:28:21.287500Z

Oh, cool. So 4 the first hour and 1/hour until the day ends. It's an interesting possibility. Not exactly trivial since each query group points to the same ddb table. You'd need to understand when the load would occur for each compute group in the system.

ghadi 2021-04-14T15:30:20.287700Z

ddb default scaling policy is reactive -- if you know when the load is going to arrive ahead of time, you could make a cron-based policy

ghadi 2021-04-14T15:31:32.288300Z

or you can take fate into your own hands and have a lambda fire periodically that controls scaling

ghadi 2021-04-14T15:31:51.288500Z

but -- should only have one controller in charge. Policies don't compose

kenny 2021-04-14T22:54:19.298300Z

The recommendation for enums in Datomic is to https://docs.datomic.com/cloud/schema/schema-modeling.html#enums. Using :db/ident as an enum value complicates the use with d/pull. Everywhere you pull an entity, you'll need to write code to deal with the nested enum entity. For example, say I have a :user/type enum that is defined as suggested by that doc (card one, ref attr). Anytime I pull my :user/type attribute, I need to write code to unnest the :user/type value.

(d/pull (d/db conn) '[:user/type] [:user/id "a"])
=> #:user{:type #:db{:id 87960930222153, :ident :user.type/a}}
How are folks dealing with this? Just always wrapping the d/pull return with something like (update :user/type :db/ident)? Perhaps always remembering to specify the pull pattern for all enum attributes as (:user/type :xform my-ns/get-db-ident), where my-ns/get-db-ident is just (def get-db-ident :db/ident)?

tatut 2021-04-15T08:18:18.304100Z

is walk so slow? Iโ€™ve used it heavily and never had it be a problem in my work loads

2021-04-15T09:56:53.304300Z

We had a specific usecase where we had very big resultsets (~150,000 entities + nesting). That's when we replaced the postwalk. Generally it's fine for sure.

2๐Ÿ˜…
tatut 2021-04-15T12:36:13.304700Z

that is a big result, out of curiosity, how much was the difference between walk and xform with that result?

2021-04-15T16:10:28.307100Z

Unfortuinely I don't have exact numbers. We also optimised the query at that time.

kenny 2021-04-15T16:40:11.308Z

We also have very big query results.

2021-04-15T21:43:58.308200Z

To give a very rough idea about our postwalk approach vs a direct update fn;

(time
  (do (map (fn [x] (update x :field :db/ident))
        corpus)
    nil))
"Elapsed time: 0.093924 msecs"
=> nil
(time
  (do (walk/postwalk (fn [x]
                       (if (and (map? x)
                             (:db/ident x))
                         (:db/ident x)
                         x))
        corpus)
      nil))
"Elapsed time: 552.667018 msecs"
=> nil
(corpus is a list of 150,000 maps btw)

1๐Ÿ‘
tatut 2021-04-16T05:58:23.313700Z

interesting, thanks

tatut 2021-04-16T05:58:43.313900Z

but isnโ€™t the first one not a good benchmark, as map returns lazy seq

1๐Ÿ˜…
2021-04-16T12:41:15.315Z

You're absolutely right. I knew something was up.

2021-04-17T13:41:48.323500Z

(time
  (do (mapv (fn [x] (update x :field :db/ident))
        corpus)
      nil))
"Elapsed time: 142.214586 msecs"
=> nil
For completionist sake:

kenny 2021-04-25T21:33:10.399600Z

Thought I'd persist this question on ask.datomic so others can reference it and add their opinion: https://ask.datomic.com/index.php/606/recommended-way-to-handle-db-ident-enums-when-used-with-pull

1
Tyler Nisonoff 2021-04-14T23:03:31.298400Z

one way i was doing this was to take my nested entity thats returned from the pull and then run it through a post-walk like so:

(clojure.walk/postwalk
    #(match %
       {:db/ident ident} ident
       :else %)
    nested-entity)
if you know that all maps with db/ident are referencing enums you want to unwrap

1๐Ÿ‘
kenny 2021-04-14T23:14:27.298600Z

Oh, nice! That's a pretty heavy penalty to pay for using enums as idents though.