Clojurians Log v2

@quoll getting somewhere with the datalog-in-JS, finally figured out how to synchronize the result sets with the dom nodes - https://gist.github.com/alandipert/ce0fa7dcb6cd74c4f5ea6679fe29dce1

ended up going bottom-up and diffing results as you suggested. seems to work nicely

the ANY.VALUE thing is a weird discovery, it seems to be a way to do disjunction based on query param, without changing the query

...being a sentinel value understood by the join loop to match any value

was curious if you'd run across anything like that in your travels, or if it's otherwise a well know thing

I have not used JS very much, so I haven’t seen it before, no

oh, ANY.VALUE is just the name of the sentinel value. when you pass that particular value in as a query parameter, it matches anything. sort of like a user-supplied wildcard

... as opposed to a (static) wildcard in the query text itself

OK, I was busy and I’ve thought about it a little now. I think I see what you’re saying. So you have a pattern of: [_.e, "status", _.status] where the _.status value is supplied as a parameter, but in this case you have a parameter that’s a wildcard.

If I’m getting that right… I haven’t seen that parameterized before

In Asami, the find and where clauses wouldn’t change, but if you didn’t want to restrict the status value, you’d have to remove the use clause (called :in in Asami)

yes i think that's right, thanks for grokking

it came up for me because i need to (re-)parameterize queries asynchronously, and it didn't seem right for the query pattern itself to change within the "extent" of the subscription established by view

how's asami going btw?

Up and down. I keep having things pull me away from it, but I keep getting back to it 🙂

This week, I’m turning it into a document store 😜

though today I’m taking the time to write wiki pages

interesting, like a document-flavored frontend layer? or a rethink of the storage etc

New element to the storage

right now, if you store objects in it, it disassembles the objects into triples, and stores those. If you ask for the object back it reassembles it

Which works correctly, and it’s why I can do round-trips on JSON documents

But it takes time to rebuild an object to return

so I’m storing the object as well

nice, with an invalidation/rebuild scheme?

yes

in case you are really trying to mess the system up by updating triples that belong to objects

To enable this, all sub-entities now have ownership attributes that connect them to their root object

If you scroll down a little from https://github.com/threatgrid/asami/wiki/Entity-Structure#structures-and-queries, you’ll see it in an example ball-and-stick diagram

All the light lines are the ownership attributes

neat

well, I’m still writing it

it’s much easier in-memory, since the objects themselves just go into a map. The on-disk version is trickier because I have to serialize everything

hm, the example reminds me of stuff i've done in datomic in the past, converting trees to sets of tuples. the document-in interface seems broadly useful. then i see you have something like datomic pull api on the other end. i like it :thumbsup:

The facilities are already there. This will just make it faster (I hope!)

very cool. yeah i don't think datomic has an option to cache entities or anything

This is why I can load a JSON file in 2 lines of code, and start querying it on line 3

https://github.com/alandipert/intension is how i used to do it

oh… and because of misbehaving JSON, I removed some of the type checking. So now entities and attributes can be anything. This is a semantic nightmare, but turns out to be useful. 🙂

heck yeah, json compat on any level brings a lot of reach and applicability

you know, i suppose what i'm working on is very similar

updating dom nodes surgically in response to a stream of query changes. it's almost exactly document maintenance with datalog backend

well, modulo all the ways asami is different, and javascript 🙃

I was asked about one file, and ended up exploring it with things like:

(def conn (d/connect "asami:<local://tmpdata>"))
(def tx @(d/transact conn (json/parse-string (slurp "cb.json"))))

(d/q '[:find (count ?a) . :where [?e ?a ?v]] db)
(d/q '[:find [?a ...] :in $ ?bad-string :where [?e ?a ?v] [(?bad-string ?a)]]
  conn #(and (string? %) (re-find #" " %)))

yes but we like datalog :simple_smile:

what does the ... in the :find do?

But then I wanted to find the entities that contain an attribute with a space in them…

(->> (d/q '[:find [?ancestor ...]
            :where [?ancestor :tg/entity true]
                   [?ancestor ?a+ ?e]
                   [?e ?attr ?v]
                   [(bad-string? ?attr)]] db #(and (string? %) (re-find #" " %)))
     (map #(d/entity db %)))

Normally a query returns a sequence of bindings, where each binding is a sequence of values that align with the :find clause

If the :find clause only contains one element, then you get back a sequence of 1 element sequences. e.g.

(["foo"] ["bar"] ["baz])

gotcha, i like that

By specifying [?ancestor …] then you get back a seq of values instead

?a+ is another interesting one, do i understand right that deals with arbitrary depth?

Also, if you’re getting back a single value, it’s annoying to get [[value]]. Even if you use the [?arg …] syntax you still get [value]. So you use .

Not only is it arbitrary depth, it’s with arbitrary attributes!

yeah that's a really good one

you have some awesome extensions going on, i must say. 💯

this operation is typically reserved for a single attribute. Like:

[?person :has-child+ ?child]

don't mind if i cram a few of these into JS syntax

It’s open source for a reason!

Following arbitrary attributes to arbitrary depth is fraught! If your graph has a spoke and hub structure (like DNS), then you could traverse the entire graph

… if you happened to start at the wrong spot 🙂

can you have cycles? this could help you find them

It detects cycles

also, if you have ?a in your :find, do you get the product of the lvars and every ?a in the tree?

no. I actually cheat a bit

the ?a value does not bind to a single value per binding, as you’d normally expect

instead, it binds to a vector containing a path

So you can actually select the path, if that’s what you want

I’m sure that this will break in some corner cases. I haven’t tested that all that much. But for simple queries it provides a neat way to find paths

yeah i think affordances like this are a net win, and i'm sure i don't have nearly the experience working with this stuff that you do

https://github.com/threatgrid/asami/wiki/Querying#path-querying

...because these things can be understood in terms of the underlying model, they're not arbitrary like with many other query tools

The issue isn’t calculating it. It’s figuring out how to make these calculations show up in query results in a way that makes sense, and does not require an excessive number of special cases

it makes sense to me so i think you're crushing it

regarding :find syntax, any reason you didn't use e.g. :find ?ancestor to get the flat sequence back?

In that query, the ?ancestor node was locked (by virtue of the [?ancestor :tg/entity true] constraint. Only the ?e was floating. That ends up getting locked by the pair of constraints: [?e ?attr ?v] [(bad-string? ?attr)]

so the sequence of nodes leading from the root attribute down to the bad attribute was actually in the ?e

If that’s what you’re asking about?

i'm asking about the syntax choice of [x ...] over x

wasn't sure if ... has other uses/meanings, or if simply x is otherwise fraught

well, I find it easier to work with [:a :b :c :d] than with [[:a] [:b] [:c] [:d]]

..although i'm grateful for the words "locked" and "floating" to describe the lvar status :thumbsup:

I can get from the latter to the former with (map first) but that’s not repl friendly

oh, i agree with you, i'm asking in particular about the grammar choice of the query map

Ah!

https://docs.datomic.com/on-prem/query/query.html#find-specifications

ah yes, ty

Also… I’m going to keep saying it: I really wish it wasn’t called “Datalog”

seconded

i'm out for now, it was nice catching up 👋

good night!