Clojurians Log v2

Hello, I’m playing around with the default generated app. If I use put from the REPL to replace the “user” document the changes are reflected immediately in the DB Contents tab (which is awesome). If I define my own “assoc” transaction function to change the value of a single key of the document, this is not picked up automatically by the frontend (a reload is necessary). Is this expected? This is what my function looks like:

(crux.api/submit-tx
 (:biff.crux/node @biff.util/system)
 [[:crux.tx/put
   {:crux.db/id :assoc
    :crux.db/fn '(fn [ctx eid k v]
                   (let [db (crux.api/db ctx)
                         e  (crux.api/entity db eid)]
                     [[:crux.tx/put (assoc e k v)]]))}]])

(crux.api/submit-tx
 (:biff.crux/node @biff.util/system)
 [[:crux.tx/fn :assoc #uuid "4ca6d9b6-e338-4e53-81e1-e2701c845821" :user/bar "IT'S ALIVE!"]])

Up until now I had assumed that subscriptions would pick up on transaction function results normally (i.e. a reload shouldn't be required). However after thinking about it for a second, it occurs to me that actually I'll need to add special handling for them. Just made an issue: https://github.com/jacobobryant/biff/issues/102

In the mean time, for this particular transaction function, you can instead use

(biff.crux/submit-tx
  sys
  {[:user #uuid "4ca6d9b6-e338-4e53-81e1-e2701c845821"]
   {:db/merge true
    :user/bar "IT'S ALIVE!"}})

oh, thanks a lot! As you can see I naively used crux.api instead of biff.crux in my example

ha no worries. Using crux.api/submit-tx directly is fine also, it's just a bit lower level. So far I use biff.crux/submit-tx exclusively myself, but if you wanted to, say, set a custom valid time, or use a more sophisticated transaction function, you'd need to use crux.api

Also at scale, if the match operations from biff.crux were resulting in a lot of contention, you could use transaction functions like the one you defined in the hot spots.

I only have a superficial understanding of both crux and biff for now, but I’m keeping your explanations in my notes for when it makes sense 🙏

biff enables a similar workflow to posh, but instead of focusing on the frontend it allows you to go back all the way to the backend, right? https://github.com/denistakeda/posh

yep

but posh does a lot of work to parse the datalog subs and figure out whether the documents that are changed match it, from reading biff’s code it seems that it’s a simpler approach maybe

oh you put it in an empty DB and see if it matches, nice (live commentary as I’m reading 😄 )

Yeah, the key difference is that Biff doesn't actually provide subscriptions for datalog queries. Since the subscribable queries don't allow joins, it's a lot easier to see if a given transaction affected a given query. side note about posh, my understanding is that its datalog parsing is just a heuristic approach; i.e. it'll work some/most of the time but there's no guarantee that it'll actually work for a given query. I tried using it a couple years ago and ran into a query that wasn't getting updated and I think the heuristic thing was the problem. though I'm not really familiar with posh's internals.

haha yeah. Thanks to the no-joins thing, the empty db thing works

yes, I remember digging into posh’s documentation etc and it gave me the impression that the datalog parsing is very hard to get right and that it was very hit-and-miss, so that put me off from using it

by the way, ElasticSearch percolators also answer the question “if I show you a document, tell me which queries it will match”

on the front end, biff's alternative to posh (`defderivations`) is kind of dumb and inefficient. fine for small apps (I haven't run into any problems with it myself), but after a certain size, UI refreshes will be too slow. I'd like to replace it with odoyle-rules, which is basically a non-heuristic version of posh (see https://github.com/jacobobryant/biff/issues/57)

I’ve watched Zach’s presentation on this library, but I’m finding it hard to visualize how it would help, I seriously need to read biff’s documentation more closely. By the way it seems like the crux people are on your case, 2 comments on the new ticket already 🙂

> yes, I remember digging into posh’s documentation etc and it gave me the impression that the datalog parsing is very hard to get right and that it was very hit-and-miss, so that put me off from using it yeah... it's an interesting project for sure but from what I can tell it's not something you would want to actually use in production... unless maybe you restricted yourself to using a subset of datalog that it understands? (kind of like how biff works I guess) to a lesser extent I feel the same way about datascript actually, at least for using it to drive a reactive UI--I never figured how to do it without just re-running all the queries on each refresh. Though I did find an interesting reddit comment from a while ago where someone described pattern matching on the datoms or something in a way that worked out pretty well for him

I haven't actually tried using odoyle-rules, but my understanding is roughly that the RETE algorithm which it implements allows you to find out what changed for arbitrary datalog queries, with certain constraints that are acceptable on the front end

I have a Java Swing project (where react is not available anyway) and I ended up with a more “primitive” approach of having the same function update the datascript state and also call the ui-updating function(s).

makes sense

in any case, I feel it’s time we left CRUD behind, so I applaud your efforts 👏

Funnily enough I was reading about Lucene's Monitor just yesterday, very cool stuff