datascript

Immutable database and Datalog query engine for Clojure, ClojureScript and JS
oconn 2020-11-12T13:56:23.050100Z

Is there a way to get the following transaction to pass without first transacting {:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")} in this example?

(def schema
  {:user/id {:db/unique :db.unique/identity}
   :user/bots {:db/valueType :db.type/ref
               :db/cardinality :db.cardinality/many}
   :bot/id {:db/unique :db.unique/identity}
   :bot/created-by {:db/type :db.type/ref
                    :db/cardinality :db.cardinality/one}})

;; Transaction fails with:
;; "Cannot add #datascript/Datom [5 :user/id #uuid "a9d9494b-9c96-4e14-8bc5-999a11651358" 536870922 true] because of unique constraint: (#datascript/Datom [7 :user/id #uuid "a9d9494b-9c96-4e14-8bc5-999a11651358" 536870922 true])"

[{:user/bots [{:bot/id (uuid "5ae358f2-5a0c-49ff-9868-bb4f6aa5e98d")
               :bot/created-by {:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")}}]
  :user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")}]

;; Transaction succeeds

[{:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")}
 {:user/bots [{:bot/id (uuid "5ae358f2-5a0c-49ff-9868-bb4f6aa5e98d")
               :bot/created-by {:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")}}]
  :user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")}]

2020-11-12T18:37:58.050600Z

I note that :bot/id is malformed; the key is :db/valueType, not :db/type. but that's probably unrelated.

2020-11-12T18:40:19.050800Z

to answer your actual question: use a lookup ref instead. the duplicated {:user/id (uuid "a9d9...")} is making it try to create two users with the same ID.

2020-11-12T18:41:09.051Z

so the top-level :user/id (uuid "a9d9...") is good. then inside the :user/bots use a lookup :bot/created-by [:user/id (uuid "a9d9...")].

2020-11-12T18:42:03.051200Z

also, stepping back and looking at the design more broadly: don't model relationships in both directions.

2020-11-12T18:42:24.051400Z

this is what the VAET index is for: reverse ref lookups.

2020-11-12T18:45:02.051600Z

if the bots have :bot/created-by as a ref to the user, you can write queries like

(q '[:find [?bot ...] :in $ ?id
      :where [?user :user/id ?]
            [?bot :bot/created-by ?user]]
  db (uuid "a9d9..."))
to get all the bots created by that user. you don't need a :user/bots attribute at all.

oconn 2020-11-12T18:49:03.052Z

;; "Nothing found for entity id [:user/id #uuid "a9d9494b-9c96-4e14-8bc5-999a11651358"]"
{:user/bots [{:bot/id (uuid "5ae358f2-5a0c-49ff-9868-bb4f6aa5e98d")
                              :bot/created-by [:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")]}]
                 :user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")}

oconn 2020-11-12T18:53:05.052300Z

Corrected the schema and updated the ref but got ^. Transacting {:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")} first works as well using the ref.

2020-11-12T19:29:55.052500Z

@oconn see the remarks about the data modeling. I encourage you to drop the :user/bots field outright. then you can transact

[{:user/id (uuid "a9d9...")}
 {:bot/id  (uuid "5ae3...")
  :bot/created-by [:user/id (uuid "a9d9...")]
 }]
and it should work in one shot.

2020-11-12T19:30:17.052700Z

(and lose nothing in queryability)

oconn 2020-11-12T19:35:28.052900Z

Yeah, that’s the next path I was going to walk. The reason I’m trying to get it to work this way is because I’m using a graph api and that’s the way the data was requested. It would be nice to directly transact the response.

2020-11-12T19:38:31.053100Z

well, you'll have to massage it somewhat to use the lookup ref anyway, so it's already not a drop-in.

oconn 2020-11-12T19:43:23.053300Z

Rough draft idea to solve for that;

(defn- xform-metadata
  [[entity-key entity-value]]
  (let [entity-name (name entity-key)]
    (if (and (#{"created-by" "updated-by" "deleted-by"} entity-name)
             (some? (:user/id entity-value)))
      [entity-key [:user/id (:user/id entity-value)]]
      [entity-key entity-value])))

(defn- clean-transaction
  "Cleans a datascript transaction before insertion"
  [transaction]
  (walk/postwalk
   (fn [x]
     (if (map? x)
       (->> x
            (remove (fn [[_entity-key entity-value]]
                      (case entity-value
                        nil true
                        :com.wsscode.pathom.core/not-found true
                        false)))
            (map xform-metadata)
            (into {}))
       x))
   transaction))

oconn 2020-11-12T19:44:22.053500Z

Which would get run on each transaction. Not 100% how I feel about this quite yet..

2020-11-12T19:45:27.053700Z

hm. it's hard to say without seeing the input data, it feels a bit awkward, relative to writing specific code to destructure a particular input map and build a transaction from it.

2020-11-12T19:46:27.053900Z

and you could add a further step to the transformation to select-keys, so you only get the ones you want and not eg. :user/bots.

oconn 2020-11-12T20:01:30.054100Z

Kinda doing something similar when requesting data. Maybe this will help describe the use case a little more.

;; Pathom query to server
{[:user/id user-uuid]
 (into [{:user/bots bots-db/bot-keys}]
   (remove #{:user/bots} users-db/user-keys))}
Where bot-keys & user-keys are essentially the result of running keys on the datascript schema for those models. Because it’s a graph query (and the data is returned in the exact shape it’s requested) - it would be really nice to transact it as is (with as little transformation as possible - preferably none). The API for this query returns
[{:user/bots [{:bot/id (uuid "5ae358f2-5a0c-49ff-9868-bb4f6aa5e98d")
               :bot/created-by {:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")}}]
  :user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")}] 
without any transformation so it’s so close! Seems {:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")} needs to be converted to [:user/id (uuid "a9d9494b-9c96-4e14-8bc5-999a11651358")] - and possibly have to pull :user/bots out and transact that second?

oconn 2020-11-12T20:02:20.054300Z

Thanks again for taking time and looking through this @braden.shepherdson

2020-11-12T20:03:45.054500Z

well, you can model the relationship the other way if you want (`:user/bots` is a list of bots that are assumed to have been created by their owning user) but it's a little more awkward that way.

2020-11-12T20:04:36.054700Z

then the only transformation you need to make is to dissoc :bot/created-by outright.