datomic

Ask questions on the official Q&A site at https://ask.datomic.com!
robert-stuttaford 2020-08-04T07:47:59.076100Z

@jaret @marshall what does it mean if i can see a datom in a d/db but not in a d/history of that same db?

jaret 2020-08-04T12:13:42.076800Z

@robert-stuttaford any chance the attribute has :db/noHistory set to true?

jaret 2020-08-04T12:48:56.078400Z

@robert-stuttaford second thought, you’re getting the history db from the db you see the datom in? If so, that sounds like something we would want to investigate. Would you be able to give us a small repro or better yet, a backup that shows this behavior?

robert-stuttaford 2020-08-04T13:26:40.078900Z

that's right - db and (d/history db)

robert-stuttaford 2020-08-04T13:27:36.079800Z

@jaret it's in our prod db, which has all our PII in it, started circa 2012 🙂

robert-stuttaford 2020-08-04T13:27:55.080200Z

perhaps we could arrange a zoom and i could show you via screen share, and then we can see about next steps from there?

2020-08-04T14:51:05.080500Z

Hey there, we are using datomic and are currently diagnosing a performance issue related to a recursive rule. We have a tree structure in datomic, that for each node, links a parent (or not), so a schema like this:

(def schema
  [;; Additional tree attributes omitted
   {:db/ident       :node/parent
    :db/valueType   :db.type/ref
    :db/cardinality :db.cardinality/one}])
Now, we sometimes look through the tree to find, say a descendant of a node. To do so, we have a recursive descendants-of? rule that binds this collection of descendants to ?node:
(def descendants-of?
  '[[(descendants-of? ?root ?node)
     [?node :node/parent ?root]]
    [(descendants-of? ?root ?node)
     [?node :node/parent ?intermediate]
     (descendants-of? ?root ?intermediate)]])
So far so good, we can do queries that reason about descendants, for example finding all descendants for a root:
(d/q '[:find ?node
             :in $ ?e %
             :where
             (descendants-of? ?e ?node)]
           (d/db (conn))
           root-eid
           descendants-of?])
Now, sometimes we have a candidate set of nodes, and of those candidates, we need to find the descendants, say like this:
(d/q '[:find ?node
             :in $ ?name %
             :where
             ;; assuming that `?name` only exists in one tree
             [?e :node/name ?name]
             (descendants-of? ?e ?node)]
           (d/db (conn))
           “name”
           descendants-of?])
In the most pathological case, where all nodes are named ?name, we will be binding all nodes in a tree to ?e, and then find the descendants of those ?e s and bind those to ?node. The result will be the same as the query above: all nodes but the root. However, this query appears to be much slower. I think that makes sense intuitively if we assume that descendants-of? is kinda expanded per ?e. For each member of ?e, we can potentially redo descendant seeking, if those descendants are also in ?e. Is there a way to optimise here, if there are potential queries that bind descendants and ancestors to ?e ?

➕ 1
kenny 2020-08-04T18:03:15.082100Z

Using :as with :db/id does not appear to have any effect. Is this expected?

(d/q
    '[:find (pull ?e [(:db/id :as "foo")])
      :where
      [?e :db/ident :db/cardinality]]
    (d/db conn))
=> [[#:db{:id 41, :ident :db/cardinality}]]

kenny 2020-08-04T18:04:36.082400Z

Other db/* attrs work:

(d/pull (d/db conn)
          '[(:db/id :as "foo")
            (:db/ident :as "ident")]
          :db/cardinality)
=> {:db/id 41, :db/ident :db/cardinality, "ident" :db/cardinality}

kenny 2020-08-04T18:05:23.082500Z

This one is strange since :db/ident is included twice.

kenny 2020-08-04T18:05:43.082700Z

:db/doc is not included twice.

(d/pull (d/db conn)
          '[(:db/id :as "foo")
            (:db/ident :as "ident")
            (:db/doc :as "doc")]
          :db/cardinality)
=>
{:db/id 41,
 :db/ident :db/cardinality,
 "ident" :db/cardinality,
 "doc" "Property of an attribute. Two possible values: :db.cardinality/one for single-valued attributes, and :db.cardinality/many for many-valued attributes. Defaults to :db.cardinality/one."}

souenzzo 2020-08-04T18:14:31.083900Z

@kenny :db/id isn't an attribute. where is no Datom[e :db/id v]. It's a "special thing" that d/pull assoc into it's response.

kenny 2020-08-04T18:15:52.084700Z

So? I don't think pull requires the selection to be attributes. From the doc "Pull is a declarative way to make hierarchical (and possibly nested) selections of information about entities."

kenny 2020-08-04T18:17:06.085800Z

Even if that was a requirement, I don't think it makes sense for it to behave differently than everything else.

souenzzo 2020-08-04T18:17:52.086400Z

I agree that it's a bug

favila 2020-08-04T18:18:26.086500Z

Why not:

'[[(descendants-of? ?root ?node)
   [?node :node/parent ?root]]
  [(descendants-of? ?root ?node)
   [?intermediate :node/parent ?root]
   (descendants-of? ?intermediate ?node)]]
?

favila 2020-08-04T18:19:13.086700Z

The second implementation of decendents-of? necessarily scans all :node/parent if ?node is unbound

favila 2020-08-04T18:19:39.086900Z

if you know ?root is bound, you can start from that

favila 2020-08-04T18:19:54.087100Z

in fact, datomic has a syntax for ensuring a rule input var is bound:

👀 1
favila 2020-08-04T18:20:07.087300Z

'[[(descendants-of? [?root] ?node)
   [?node :node/parent ?root]]
  [(descendants-of? [?root] ?node)
   [?intermediate :node/parent ?root]
   (descendants-of? ?intermediate ?node)]]

2020-08-04T18:46:20.087600Z

Ah let me take a look, I may have simplified the example a bit too much when lifting it from our codebase

kenny 2020-08-04T19:05:05.087800Z

Opened a support ticket for this https://support.cognitect.com/hc/en-us/requests/2797

2020-08-04T19:11:04.088Z

Oh you are on to something! Thank you so much