@quoll getting somewhere with the datalog-in-JS, finally figured out how to synchronize the result sets with the dom nodes - https://gist.github.com/alandipert/ce0fa7dcb6cd74c4f5ea6679fe29dce1
ended up going bottom-up and diffing results as you suggested. seems to work nicely
the ANY.VALUE thing is a weird discovery, it seems to be a way to do disjunction based on query param, without changing the query
...being a sentinel value understood by the join loop to match any value
was curious if you'd run across anything like that in your travels, or if it's otherwise a well know thing
I have not used JS very much, so I havenāt seen it before, no
oh, ANY.VALUE is just the name of the sentinel value. when you pass that particular value in as a query parameter, it matches anything. sort of like a user-supplied wildcard
... as opposed to a (static) wildcard in the query text itself
OK, I was busy and Iāve thought about it a little now.
I think I see what youāre saying. So you have a pattern of:
[_.e, "status", _.status]
where the _.status
value is supplied as a parameter, but in this case you have a parameter thatās a wildcard.
If Iām getting that rightā¦ I havenāt seen that parameterized before
In Asami, the find
and where
clauses wouldnāt change, but if you didnāt want to restrict the status value, youād have to remove the use
clause (called :in
in Asami)
yes i think that's right, thanks for grokking
it came up for me because i need to (re-)parameterize queries asynchronously, and it didn't seem right for the query pattern itself to change within the "extent" of the subscription established by view
how's asami going btw?
Up and down. I keep having things pull me away from it, but I keep getting back to it š
This week, Iām turning it into a document store š
though today Iām taking the time to write wiki pages
interesting, like a document-flavored frontend layer? or a rethink of the storage etc
New element to the storage
right now, if you store objects in it, it disassembles the objects into triples, and stores those. If you ask for the object back it reassembles it
Which works correctly, and itās why I can do round-trips on JSON documents
But it takes time to rebuild an object to return
so Iām storing the object as well
nice, with an invalidation/rebuild scheme?
yes
in case you are really trying to mess the system up by updating triples that belong to objects
To enable this, all sub-entities now have ownership attributes that connect them to their root object
If you scroll down a little from https://github.com/threatgrid/asami/wiki/Entity-Structure#structures-and-queries, youāll see it in an example ball-and-stick diagram
All the light lines are the ownership attributes
neat
well, Iām still writing it
itās much easier in-memory, since the objects themselves just go into a map. The on-disk version is trickier because I have to serialize everything
hm, the example reminds me of stuff i've done in datomic in the past, converting trees to sets of tuples. the document-in interface seems broadly useful. then i see you have something like datomic pull api on the other end. i like it :thumbsup:
The facilities are already there. This will just make it faster (I hope!)
very cool. yeah i don't think datomic has an option to cache entities or anything
This is why I can load a JSON file in 2 lines of code, and start querying it on line 3
https://github.com/alandipert/intension is how i used to do it
ohā¦ and because of misbehaving JSON, I removed some of the type checking. So now entities and attributes can be anything. This is a semantic nightmare, but turns out to be useful. š
heck yeah, json compat on any level brings a lot of reach and applicability
you know, i suppose what i'm working on is very similar
updating dom nodes surgically in response to a stream of query changes. it's almost exactly document maintenance with datalog backend
well, modulo all the ways asami is different, and javascript š
I was asked about one file, and ended up exploring it with things like:
(def conn (d/connect "asami:<local://tmpdata>"))
(def tx @(d/transact conn (json/parse-string (slurp "cb.json"))))
(d/q '[:find (count ?a) . :where [?e ?a ?v]] db)
(d/q '[:find [?a ...] :in $ ?bad-string :where [?e ?a ?v] [(?bad-string ?a)]]
conn #(and (string? %) (re-find #" " %)))
Which is all easy in other systems as wellyes but we like datalog :simple_smile:
what does the ...
in the :find
do?
But then I wanted to find the entities that contain an attribute with a space in themā¦
(->> (d/q '[:find [?ancestor ...]
:where [?ancestor :tg/entity true]
[?ancestor ?a+ ?e]
[?e ?attr ?v]
[(bad-string? ?attr)]] db #(and (string? %) (re-find #" " %)))
(map #(d/entity db %)))
Normally a query returns a sequence of bindings, where each binding is a sequence of values that align with the :find
clause
If the :find
clause only contains one element, then you get back a sequence of 1 element sequences. e.g.
(["foo"] ["bar"] ["baz])
If you want to, you can send that to a (map first ā¦)
but itās an extra stepgotcha, i like that
By specifying [?ancestor ā¦]
then you get back a seq of values instead
?a+ is another interesting one, do i understand right that deals with arbitrary depth?
Also, if youāre getting back a single value, itās annoying to get [[value]]
. Even if you use the [?arg ā¦]
syntax you still get [value]
. So you use .
Not only is it arbitrary depth, itās with arbitrary attributes!
yeah that's a really good one
you have some awesome extensions going on, i must say. šÆ
this operation is typically reserved for a single attribute. Like:
[?person :has-child+ ?child]
Will return a list of all descendantsdon't mind if i cram a few of these into JS syntax
Itās open source for a reason!
Following arbitrary attributes to arbitrary depth is fraught! If your graph has a spoke and hub structure (like DNS), then you could traverse the entire graph
ā¦ if you happened to start at the wrong spot š
can you have cycles? this could help you find them
It detects cycles
also, if you have ?a
in your :find
, do you get the product of the lvars and every ?a in the tree?
no. I actually cheat a bit
the ?a
value does not bind to a single value per binding, as youād normally expect
instead, it binds to a vector containing a path
So you can actually select the path, if thatās what you want
Iām sure that this will break in some corner cases. I havenāt tested that all that much. But for simple queries it provides a neat way to find paths
yeah i think affordances like this are a net win, and i'm sure i don't have nearly the experience working with this stuff that you do
https://github.com/threatgrid/asami/wiki/Querying#path-querying
...because these things can be understood in terms of the underlying model, they're not arbitrary like with many other query tools
The issue isnāt calculating it. Itās figuring out how to make these calculations show up in query results in a way that makes sense, and does not require an excessive number of special cases
it makes sense to me so i think you're crushing it
regarding :find syntax, any reason you didn't use e.g. :find ?ancestor
to get the flat sequence back?
In that query, the ?ancestor
node was locked (by virtue of the [?ancestor :tg/entity true]
constraint. Only the ?e
was floating. That ends up getting locked by the pair of constraints: [?e ?attr ?v] [(bad-string? ?attr)]
so the sequence of nodes leading from the root attribute down to the bad attribute was actually in the ?e
If thatās what youāre asking about?
i'm asking about the syntax choice of [x ...] over x
wasn't sure if ... has other uses/meanings, or if simply x
is otherwise fraught
well, I find it easier to work with [:a :b :c :d]
than with [[:a] [:b] [:c] [:d]]
..although i'm grateful for the words "locked" and "floating" to describe the lvar status :thumbsup:
I can get from the latter to the former with (map first)
but thatās not repl friendly
oh, i agree with you, i'm asking in particular about the grammar choice of the query map
Ah!
https://docs.datomic.com/on-prem/query/query.html#find-specifications
ah yes, ty
Alsoā¦ Iām going to keep saying it: I really wish it wasnāt called āDatalogā
seconded
i'm out for now, it was nice catching up š
good night!