asami

Asami, the graph database https://github.com/threatgrid/asami
mdallastella 2021-06-03T10:10:39.061100Z

Morning. I'm trying to write a simple task orchestrator using asami as temporary storage, since a job workflow is basically a DAG. I was thinking to use loom.alg/topsort both to check that a job has no cycle and to get the tasks without requirements (so I can run them first). This is what I got so far:

(defn graph-test
  [conn]
  (let [t1 {:db/ident :t1 :task/name "Task 1"}
        t2 {:db/ident :t2 :task/name "Task 2"}
        j1 {:db/ident :j1 :job/name "Job 1"}]
    @(d/transact conn [t1 t2 j1])
    @(d/transact conn [{:db/ident :j1 :tasks [{:db/ident :t1}
                                              {:db/ident :t2}]}
                       {:db/ident :t1 :requires {:db/ident :t2}}])
    (->> conn
         d/db
         d/graph
         a/topsort)))
But the result is unclear to me, I get both node properties, entities and ids. Is there a better way to do this? What am I missing? Thanks.

quoll 2021-06-03T10:18:22.063Z

The structure looks OK, but I honestly have no idea what topsort does! I’ll have to learn more about it before I can say, sorry

mdallastella 2021-06-03T10:21:22.063100Z

Sure, I was hoping that someone had the same problem and how solve it

mdallastella 2021-06-03T10:21:51.063200Z

Just a shot in the dark 😅

quoll 2021-06-03T10:51:09.065300Z

I haven’t done a lot with Loom, sorry. I still need to update it for the on-disk storage (it only works with in-memory stores)

mdallastella 2021-06-03T11:14:06.065400Z

Don't worry, it's probably something I can implement myself

2021-06-03T15:31:24.067200Z

@mdallastella just throwing out some general related knowledge i have in case it's useful. usually a topo sort operates on a set of pairs, parent/child or dependency/depender. in asami or a graph db this is a parent child relationship. so you'd want to get the set of parent/child pairs from asami, then feed that to the topo function. in your case probably it would be pairs of EIDs

👍 1
2021-06-03T15:32:27.067500Z

e.g. `'[:find [?dependency ?depender] :where [?depender :depends-on ?dependency]]`

2021-06-03T15:32:47.067800Z

... then feed those pairs to a topo algo

mdallastella 2021-06-03T15:33:34.068400Z

Thanks @alandipert, it's something I was thinking of

2021-06-03T15:34:07.068900Z

...it might also be possible to define the topology order in terms of a datalog query, maybe with aforementioned extensions quoll demonstrated for navigating deep transitive relationships between entities. but i've never done that myself, in any datalog

mdallastella 2021-06-03T15:36:33.069Z

There's a little bit of research around the web, but probably it's out of the scope of a datalog engine

quoll 2021-06-03T22:35:03.070Z

Hi everyone. I’ve written an https://github.com/threatgrid/asami/wiki/Introduction for people who are new to it. If anyone gets a chance, I would love some feedback please! 🙂

❤️ 2