Clojurians Log v2

i have been doing some thinking and prototyping around something i'd love feedback/thoughts on

a couple years ago i had a discussion with micha about fusing cells and values, based on the insight that cells mostly contain maps

and most formula code looked pretty much like relational queries if you squinted

but because the formulas are opaque functions, cells aren't aware of relationships between the formula code and changes to the value, if any

so we rely on clojure.core/= here to propagate change. the problem is formulas might run based on some change, but produce the same result

granted, propagation will stop there

so lately i have been noodling around with an in-memory datalog-esque model

you have a db of triples that you can query with datalog, but he twist is, you register queries with a callback and your callback is invoked when your query results change

this arrangement leverages the fact that a datalog query embodies a view into the db, and so when data goes in or is removed, we know exactly which queries will be affected, and thus need to be rerun

so there is still the data-based diffing, but less general, restricted to datalog

my original idea was for registered query callbacks to receive diffs of results, so they could choose to accumulate data themselves

in the context of hoplon, a diff about aparticular entity being added or removed could be used to make a precise dom modification

anyway, would love to hear any thoughts on this general scheme, particularly if others have been experimenting with datalog + hoplon in any way, such as via datascript ( i remember maybe @mynomoto did some of this in the past?)

this isn't exactly the same scheme, but maybe it's close enough to be relevant. I've been experimenting with a macro that can keep track of how values are derived. for example:

(let [state  {:todos
              [{:complete? false
                :description "first"}
               {:complete? false
                :description "second"}
               {:complete? true
                :description "third"}]}
      todos (:todos state)
      my-views (for [todo todos]
                 (hstack
                  (checkbox (:complete? todo))
                  (textarea (:description todo))))]
  (apply vstack my-views))

> but because the formulas are opaque functions, cells aren't aware of relationships between the formula code and changes to the value, if an so if the cells analyzed the code within the cell, then you might be able to automatically derive the relationships and have more efficient change/update tracking

right i think that's exactly what the datalog scheme is

except the analysis is of datalog and not clj

maybe analyzing a clj(s) subset would have the same affordances and come more naturally tho

i can definitely see the appeal

rather than datascript, I use specter under the hood which has its own query language

in theory, you could have the same code as above and have it use datascript or specter under the hood without changing the code.

@alandipert Yeah, I did some experiments but never used on a production app. I just used a cell instead of a atom when creating the datascript database. For the toy project I was experimenting with perf was never a problem but it could be as every query ran again on every database change.

If the query has a way to know if it should or not run again based on what changed on the database that would be awesome. I was experimenting with some indirection to run less queries and use those in several places but that as all the things has trade-offs.

the observation i feel might be helpful is just that, whenever a new fact comes in, if no query refers to any eav of the fact, then no query need run

anyway thanks much for the ideas folks, i'll keep playing with it and report back if i have any interesting results

The way that datalog works it would be interesting to check if you could use different formula cells for each step of filters on queries. That way as soon as a filter returns the same results as before it would stop propagation.

But that would cause more queries to run when the db change but less work keeping track of when a query should run again.

Ignore what I just suggested, it doesn't work for multiple reasons. If the query needs to run it probably has to run as a whole.