rdf

2020-10-20T08:42:21.175200Z

@quoll: I’ve been taking a quick look at asami, and I think if I’d known of this 2 years ago I perhaps wouldn’t have written matcha (which really just started as a quick but very useful hack). What I like (for my needs) is that it’s schemaless/open-world, which means like RDF I can add the schema either before or after as facts in the database. I’m currently looking for something to spike out a toy project that will let me implement an RDF-lite in clojure, but with RDFS like inferencing on domains/ranges subproperties/classes etc. My understanding is that I could plug naga in to asami, and add the rdfs-like rules as datalog horn clauses? I can see that the rdfs rules would be trivial to express this way. I’m guessing these rules aren’t stored in the database itself as facts, but sit alongside it? My main questions are more about asami itself… I skimmed through the excellent docs but couldn’t see a discussion of these points, I’m sure I can easily do some more digging to find out though… 1. What is the main difference between :db/id and :db/ident? I’m guessing :db/id is a user assigned id, and :db/ident is a database assigned one? Does every statement get assigned a :db/ident regardless of being given a :db/id or is :db/id used as a :db/ident? 2. I’m assuming the boolean at the end of every stored statement represents addition/retraction? 3. Is it possible to transact triples into asami, i.e. to represent the data as 3/tuple vectors?

quoll 2020-10-21T13:34:32.182600Z

If I’d known about Datascript in 2016 then I wouldn’t have started Asami… probably. Actually, Asami was just supposed to be in-memory storage and indexing with counts and inner-joins. So maybe I would have. :woman-shrugging:

quoll 2020-10-21T13:36:03.182800Z

Yes, Naga runs rules that are outside of the database. It’s funny that you ask this, because the first rules engine I ever wrote stored the rules in the graph 🙂

2020-10-21T13:37:57.183Z

Yeah, I think I saw you say something like that about datascript in another thread… I could be wrong but datascript seems to require you to specify a schema upfront. In particular property cardinality; which I’d rather leave open like in RDF. It looks like Asami has more of this flavour than datascript/datomic?

quoll 2020-10-21T13:38:04.183200Z

Over time I found that people wanted to edit and add to rules, and having these separate made it easier to manage. Also, they wanted to see rules in a document. It’s possible to parse the document, store it in the graph, then read it back out, and emit it for editing, so there’s no complexity there. But it just didn’t seem worth it

quoll 2020-10-21T13:38:21.183400Z

Asami is definitely like RDF in this way

👍 1
quoll 2020-10-21T13:38:40.183700Z

Actually, the differences between RDF and Asami are minimal…

quoll 2020-10-21T13:39:27.183900Z

Asami allows ANY data type in any column. RDF requires: • Subject: URI or Blank Node • Predicate: URI • Object: URI, Blank Node, Literal

2020-10-21T13:40:47.184100Z

> Asami is definitely like RDF in this way Yeah that’s actually what I like about it over datascript… I mean tbh I’d much prefer knowing cardinalities are 1 or many in RDF; so it’s not religous argument for me… I just think Asami would suit my needs better.

quoll 2020-10-21T13:40:58.184300Z

RDF also has specific support for Literals. Language tagged, plain (which are now inferred to be xsd string literals), and XSD types. Asami doesn’t care

2020-10-21T13:41:12.184500Z

yeah I did the same thing in matcha (allowing any value in any position)

quoll 2020-10-21T13:41:45.184700Z

I apologize… I was going to try to go through each thing that you asked. I hope I don’t get mixed up in the conversation instead 🙂

quoll 2020-10-21T13:42:59.184900Z

A lot of RDF was just ensuring that things were the correct types. Moving to Scala helped there a bit. But it was a pain. Since starting Asami I decided that I didn’t care. Sure enough, someone started using strings as predicates to connect things. Asami didn’t care :rolling_on_the_floor_laughing:

👍 1
quoll 2020-10-21T13:43:43.185100Z

About your 3 points:

quoll 2020-10-21T13:52:38.185400Z

1. :db/id is a pseudo property that actually refers to an entity. If you describe an entity with triples, then it’s not needed. e.g.

:node-20 :name "Mary"
:node-20 :age 32
But if you describe it as an entity (i.e. a Clojure map), then how do you refer to the entity node?
{:name "Mary"
 :age 32}
This will allocate a node value that could be anything (typically you won’t actually care). But you may specifically want to refer to :node-20 because you picked that value out of the graph earlier, or something. That can be done with the pseudo property:
{:db/id :node-20
 :name "Mary"
 :age 32}
That ensures that the :node-20 keyword will be the subject value for all of those triples. :db/ident is an actual real property. It just has a special status when it comes to selecting entities. So:
{:db/ident :mary
 :name "Mary"
 :age 32}
will lead to triples like this:
:node-55 :db/ident :mary
:node-55 :name "Mary"
:node-55 :age 32
The special status is that it can be used to refer to an entity. You can query for entities by that value, and you can insert other entities that refer to that value, so that you get a link to it. In this way it acts a bit like :db/ident

quoll 2020-10-21T13:53:50.185700Z

2. Yes, a true is an assertion, a false is a retraction. It’s just to look like Datoms. It’s not actually stored anywhere 🙂

quoll 2020-10-21T13:56:48.185900Z

Errrr…. that would be trivial. Until recently, that’s HOW you put data into Asami. But the transact call was supposed to look like Datomic, so it doesn’t accept that. Ironically, the transact function filters the input for 3 things: 1. Maps. These are entities and are turned into triples for insertion (with some retractions if modifications were requested) 2. :db/add These are stripped down into triples and inserted. 3. :db/retract These are stripped down into triples and removed.

quoll 2020-10-21T13:57:21.186100Z

I could add an option to look for triples, if that’s important to you

2020-10-21T13:59:46.186300Z

1. ok that’s pretty much what I thought after playing with it a little… map syntax is kinda like coining a blank node. > In this way it acts a bit like :db/ident  Did you mean to say “In this way it acts a bit like :db/id here?”

2020-10-21T14:05:13.189200Z

> I could add an option to look for triples, if that’s important to you It’s not super important at the minute… was just wondering what the best way to reingest the naga results as a connection

2020-10-20T10:17:38.177100Z

Is it possible (and is there an example of) to use an asami store, with some naga rules, with arbitrary asami queries as a clojure library?

quoll 2020-10-21T14:00:07.186500Z

There aren’t really many examples. The Naga CLI was actually written to be an example of using Naga. It probable doesn’t do well for that anymore now that the Asami API has changed so much. It still works though… https://github.com/threatgrid/naga/blob/main/src/naga/cli.clj

2020-10-20T10:32:45.178Z

@quoll: It looks like naga performs inference at index time not query time, is that right?

quoll 2020-10-21T14:00:34.186800Z

I actually don’t understand what you’re asking.

quoll 2020-10-21T14:00:52.187Z

Data is indexed as it is inserted, since it’s the indexes which store the data.

quoll 2020-10-21T14:01:08.187200Z

Naga runs inferences by executing a query, and inserting the results as triples

2020-10-21T14:01:58.187400Z

Thanks you’ve answered the question 🙂 Was essentially just checking that it worked by materialisation, rather than via query rewriting.

quoll 2020-10-21T14:21:58.197700Z

Yes, it does. I’d like to do query rewriting one day. But if that’s what you need, then may I suggest Stardog? 🙂

2020-10-21T14:30:58.204300Z

I’m already using Stardog 🙂 It’s not a requirement either; just a question from a curious mind 🙂

👍 1
2020-10-20T10:34:22.178800Z

actually I can see that it does :thumbsup:

2020-10-20T10:37:59.179400Z

👀 ahh there’s a query function on the Storage protocol

2020-10-20T10:40:02.179800Z

looks like that’s a lower level interface though…

quoll 2020-10-21T14:02:09.187600Z

Yes. This was before Asami became its own thing. However, you can pull the Asami graph back out of Naga and convert it into a Connection

quoll 2020-10-21T14:02:24.187800Z

I will document this, and probably add a function to make it trivial

2020-10-21T14:03:17.188600Z

yeah that’s essentially what I was looking at doing

quoll 2020-10-21T14:05:52.189400Z

If you have an AsamiGraph called asami-graph that you used for Naga, then convert it by saying: (asami.core/as-connection (:graph asami-graph) "my:graph:uri")

quoll 2020-10-21T14:08:04.189600Z

Everything wraps a Graph object. Naga wraps it using the AsamiGraph record. This gives it transactions, query/insert, etc. But the graph that it’s wrapping is just the :graph field. Asami Connection objects wrap the graph as well. And you can manually do that with the as-connection function. This wraps it, and assigns a URI for it (since connections have a URI)

quoll 2020-10-21T14:15:37.192400Z

https://github.com/threatgrid/asami/issues/95 https://github.com/threatgrid/asami/issues/96 I’ll look at something for Naga too

2020-10-20T10:59:17.182400Z

I can do stuff like this:

(def rules [(r/r "grand-parent" [?a :grand-parent ?gp] :- [?a :parent ?c] [?c :parent ?gp])])

  (def axioms
    [[:amy :parent :adam]
     [:amy :parent :mary]
     [:mary :parent :sandra]
     [:adam :parent :john]])

  (def prog (r/create-program rules axioms))

  (sr/register-storage! :memory asami-store/create-store)
  (let [[store results :as output] (e/run  {:type :memory} prog)]
    (store/resolve-pattern store '[?s ?p ?o]))
But is it possible to configure naga as a store itself, and register it with asami? Or do I need to pass the set of results from the query pattern back into a new asami store?