Clojurians Log v2

Athens Research uses posh but it looks like they’re having performance issues https://github.com/athensresearch/athens/pull/784

(cljs.pprint/pprint
 (let [conn (ds/create-conn)]
   (ds/transact! conn [{:db/id -1
                        :name "Fred"
                        :friend {:db/id -2
                                 :name "Rich"
                                 }}])
   (ds/datoms @conn :eavt)))

(#datascript/Datom [1 :friend {:db/id -2, :name "Rich"} 536870913 true]
 #datascript/Datom [1 :name "Fred" 536870913 true])

Here’s the return from ds/transact!

{:db-before {},
 :db-after {1 :friend, 1 :name},
 :tx-data
 [#datascript/Datom [1 :name "Fred" 536870913 true]
  #datascript/Datom [1 :friend {:db/id -2, :name "Rich"} 536870913 true]],
 :tempids {-1 1, :db/current-tx 536870913}, 
 :tx-meta nil}

From the docs on transact! :

; ref attributes can be specified as nested map, that will create nested entity as well
      (transact! conn [{:db/id  -1
                        :name   \"Oleg\"
                        :friend {:db/id -2
                                 :name \"Sergey\"}])

:friend must be declared as :db.type/ref

Ok, thanks.

Thanks for setting me on the right track. I didn’t know I needed a schema. I haven’t figured out how to pass it in though. I’ve tried passing it on creaton:

(ds/create-conn {:schema
                             [{:db/ident :friend
                                :db/valueType :db.type/ref
                               :db/cardinality :db.cardinality/one}]})

(ds/transact! conn [ {:db/ident :friend
                         :db/valueType :db.type/ref
                         :db/cardinality :db.cardinality/one}])

Oh.

(cljs.pprint/pprint
 (let [conn (ds/create-conn {:schema
                             [{:db/ident :friend
                               :db/valueType :db.type/ref
                               :db/cardinality :db.cardinality/one}]}
                            )]

   (ds/schema @conn)))

{:schema
 [{:db/ident :friend,
   :db/valueType :db.type/ref, 
   :db/cardinality :db.cardinality/one}]}

Oh.

(ds/create-conn {:friend
                             {:db/ident :friend
                              :db/valueType :db.type/ref
                              :db/cardinality :db.cardinality/one}})

it's one of the reasons why I wouldn't recommend it as a catch-all for UI app state

@lilactown It’s actually kinda slow even with very little data on React Native; I am not sure exactly why. Maybe I don’t fully understand how to use it. If you try to use the indices directly, it’s faster; but any use of (d/q ...) is often quite slow.

But otherwise the functionality is very nice, so I’m sticking with… but have to resort to some hacks to make the UI snappy.

One idea I’m about to experiment with is treating it as a remote database; basically assume it’s a remote database, and show loaders/spinners everywhere;

That obvi. won’t make it faster, but maybe it will “feel” faster 🙂

The “hacks” involve having an extra app-state the I render out of, rather than querying directly from Datascript; querying datascript multiple times per second is a big NO-GO

Are you using it with re-frame? Re-frame will run all the queries for every transact, which is what the posh library tried to solve. I was planning to use datascript too but if there is such a perf problem, I'll have to think of something else. Has anyone tried running reframe + datascript in a web worker?

Something like this https://github.com/yetanalytics/re-thread

@kslvsunil no, I use React directly; no extra libs

If I paint myself in the corner, at least I have a chance of understanding how I got there! haha 🙂

Oh dang, then it's going to much much slower with re-frame.

No idea, never used re-frame/reagent.

I wonder how roam research uses datascript

Only om.next back in the day.

And while I haven’t used them, it’s hard to say whether they are faster or slower; but it will most likely be more difficult for me to analyze the whole thing.

Web worker is something I’ve thought about;

It will make it “faster” in the sense that it won’t block your JS UI thread, but the results will still often take 100+ ms to arrive;

So you’re definitely in the realm of what I wrote just above: you’d have to treat DataScript as a fully remote database and never even pretend that it’s local (it will definitely be all async)

Basically, anytime things take more than 16ms in JavaScript, you run into UI smoothness trouble (1000 / 60fps = 16 ms)

Yup agree completely to "It will make it 'faster' in the sense that it won’t block your JS UI thread, but the results will still often take 100+ ms to arrive;". There is also lilactown's https://github.com/lilactown/autonormal which I was recommended in the asami channel. Guess I'll start spiking out with autonormal. All I want is a well structured normalized app state, which I could probably tame by having some discipline. But the problem is every time I want a new state slice, I'll have to think about the structure of it, it's nesting level, yada yada, which is time consuming and burns a lot of cognitive energy unnecessarily

autonormal won't give you the full power of datalog, but on the other hand because it's not tuple-based it should be much faster

I benchmarked it early on against datascript and mapgraph and it was orders of magnitude faster than DS. I've added some features after that that may have slowed it down a bit. I should redo those benchmarks and publish them

React Native could probably get away with treating datascript as a "remote" database since JS runs on a separate thread from the actual UI, but the browser cannot; if transacting takes 250ms then that's 250ms that the UI thread isn't doing anything else (like responding to clicks or key presses)

oh I see you mentioned webworker @raspasov. yes. then it's measuring the cost of serialization

I think that the idea of shoving all app state into a single store is naive anyway. DS in a webworker for domain state might be good enough and open up other novel things like sharing that state across browser windows