Clojurians Log v2

@quoll picking up on this thread: https://clojurians.slack.com/archives/C09GHBXRC/p1613605836089200?thread_ts=1613605226.087200&cid=C09GHBXRC I get that some users might be more familiar with having a “mutable” database/connection, and that this mirrors datomic… However I find this sort of thing makes naga/asami harder to use for these usecases. It’s much less obvious about how to wire things together as we have to learn asami connection APIs; for negative value compared to something more value oriented e.g.

(->> asamival/empty-db 
       (asamival/transact {:tx-data [[:db/add :foo :bar :baz]]) 
       (nagaval/materialise-inferences program) 
       (asami/q '[:find ,,,,]))

I also ran into these issues fwiw 🙂. I avoided by going to the lower-level API underneath, but I found that easier as I have used naga before the conn concept was introduced and knew where to poke.

Yeah not done any digging yet, but good to know.

@quoll: For fun I had a go at defining rdfs entailment in naga. As I think your skos.rlog definition was missing some rdfs rules e.g:

rdf:type(YYY,XXX) :- rdfs:domain(AAA, XXX), AAA(YYY, ZZZ).    /* rdfs2 */
rdf:type(ZZZ,XXX) :- rdfs:range(AAA, XXX), AAA(YYY, ZZZ).     /* rdfs3 */

rdf:type(A, rdfs:Datatype) :- A(B,C), clojure.core/keyword?(A) .
rdf:type(B, rdfs:Datatype) :- A(B,C), clojure.core/keyword?(B) .
rdf:type(C, rdfs:Datatype) :- A(B,C), clojure.core/keyword?(C) .

I guess such a thing might be better expressed in the naga representation than the pabu one though… As pabu rules could then be kept compatible with other datalogs… where as it would be reasonable to assume if you’re using the clojure rules representation you have access to clojure… Actually do you have something like this already? I seem to recall you mentioned negation and (or ,,,)

:thinking_face: hmm looks like mulgara might have had some special predicate/hacks for this sort of thing too e.g mulgara:UriReference

Actually I’m curious, it looks like mulgara “fixes” a frustration I’ve had in the past with the rdfs entailments… i.e. in rdfs IIRC literally everything is an rdfs:Resource, which means that Literals and URIs are kind of indistinguishable. It looks like mulgara’s entailments might try and keep these distinct? Is that what I’m looking at here @quoll? https://github.com/quoll/mulgara/blob/36ee68b9cccaca26f55a39d37511fb4664b004e0/rules/rdfs.dl#L38-L39 i.e: - 4a says any subject is a resource (therefore must be a URI as you can’t speak of literals) - 4b (appears to) say that any object is a resource iff its an object and of type uri?

ok actually going to have to tear myself away and stop digging any further, and do some real work. 😢

I’ll confess that I haven’t been looking at making that skos stuff work. It was lifted out of Mulgara. But since I know Naga/Asami can do everything Mulgara can do, then I figured it just needed a little weaking, if anything.

Also, it’s skos, so it’s not supposed to infer rdfs. A lot of rdfs inferences are trivial. For instance, I found that the inferences of everything being an rdfs:Resource to be useless from a practical perspective, because if it’s in the database, then it’s a resource.

As for these rules, I don’t believe that they are valid.

Nor consistent

If you’re doing RDF (which Asami can approximate, but isn’t trying to be), then resources can only be datatypes if they’re IRIs (or URIReferences)

However, if you’re in Clojure, then keywords are basically QNames, and can be used as such. In that case, your inferences are OK, and are consistent, but I don’t believe they’re valid?

Actually, it’s hard to make something invalid when you’re only in RDFS, so I guess it’s valid, but it isn’t useful. e.g. This will infer that <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> (QName rdf:type or as a keyword :rdf/type) is an rdfs:Datatype. That makes the following syntactically correct, but nonsense: "foo"^^<rdf:type>

Yes. Rules are actually just a :where clause, and a projection to groups of triples for assertion. Pabu is just a parser that takes Datalog and creates such a thing, but because the parsing hasn’t been looked at for a long time, it has limited syntactic capabilities.

If I were to expand Pabu, I should really do it with instaparse. At the moment it uses Parsatron, which was really fun and cool to play with at the time. I just needed a quick way to parse something, and I remembered a previous colleague of mine had ported Parsatron from Haskell parser combinators, so I grabbed it quickly, and 45 minutes later the first version of Pabu existed. It was really only that well thought out 🙂

SPARQL does specifically allow for extensions like this, so hopefully it’s not a “hack” 🙂

URIReference, but yes.

RDF does not allow literals as a subject or predicate, and that avoids the type error that would result. Asami lets you do this, which actually allows for some interesting data expressions. Like:

12 :math/factor 6
12 :math/factor 4
12 :math/factor 3
12 :math/factor 2

Originally, everything was done via a graph protocol, which is reasonably small:

(defprotocol Graph
  (new-graph [this] "Creates an empty graph of the same type")
  (graph-add [this subj pred obj tx] "Adds triples to the graph")
  (graph-delete [this subj pred obj] "Removes triples from the graph")
  (graph-transact [this tx-id assertions retractions] "Bulk operation to add and remove multiple statements in a single operation")
  (graph-diff [this other] "Returns all subjects that have changed in this graph, compared to other")
  (resolve-triple [this subj pred obj] "Resolves patterns from the graph, and returns unbound columns only")
  (count-triple [this subj pred obj] "Resolves patterns from the graph, and returns the size of the resolution"))

originally, graph-transact and graph-diff weren’t there. graph-diff came about because people wanted to see what had changed after Naga had run on it. I consider it optional, as nothing in Asami uses it.

graph-transact is much more recent. Originally, there were 2 functions in the query namespace (not a great place for them, but :woman-shrugging: )

(defn add-to-graph
  [graph data]
  (reduce (fn [acc d] (apply graph/graph-add acc d)) graph data))

(defn delete-from-graph
  [graph data
  (reduce (fn [acc d] (apply graph/graph-delete acc d)) graph data))

The new graph-transact function does exactly this… applying deletions first, then assertions. It also takes a transaction ID, which is not (currently) used in the in-memory database

Anyway, I believe that the `Graph` protocol is the API you want @rickmoynihan

What I found was that no one I worked with felt comfortable learning Asami. So I wrapped that Graph API in a Database/Connection façade, and voilà! They started using it! 🙂

But it’s all still a Graph under the covers

If you have a Connection, then you get the most recent Database using (asami.core/db connection) If you have a Database, then you get the graph for it using (asami.core/graph database)

You can then do whatever you want with the graph (querying it the same as you do a Database… in fact, the q function only works on graphs, and calls graph on a database to get the graph). When you’re done, you can make it look like a Connection/Database again by calling (asami.core/as-connection graph)

Optionally, you can give it a URI for the connection to associate it with:

(asami/as-connection graph "asami:<mem://my-data>")

This will actually replace any registration of prior connections at that URI, which is probably something to be aware of, and also really useful

So, referring back to that original code:

(-> index/empty-graph
    (graph/graph-transact 0 [[:foo :bar :baz]] nil)
    asami/as-connection
    (naga-engine/run my-program)
    db
    graph
    (graph-q '[:find ?e ?a ?v :where [?e ?a ?v]]))

Should this all go in the Wiki?

:thumbsup: Yes I’m aware of most of that (though you’re right to pick me up on misunderstanding rdfs:Datatype and rdfs1 — it’s about literals/datatype-uris and not quite what I thought — the classes of things you can speak of (i.e. BNodes/IRIs)). Re: RDFS I know it’s not part of skos… I just want a combination of some rdfs inferences (mostly domain / ranges, subproperties etc), and likely also some or all of the skos you have too. Though at this stage I’m really just tinkering, rather than having concrete plans for any of this — so I was mainly wanting to write the rdfs rules out in as an excuse to play with naga. The suggestion above came out of that tinkering. My suggestion really wasn’t about RDF though, it was primarily do you think it might be useful to support arbitrary clojure predicates in naga logic programs like this?

:thumbsup: yup I’m aware of this. matcha is similar.

Thanks for posting all the above… I’ll try and digest it all later 🙇

true 🙂

Also… I just remembered: Yes, you’re right. The datomic dependency is entirely optional and should be in a separate profile. I’ve been too lazy to do this.

When you say that it brings in a lot, that’s only because of Datomic. Remove that, and there’s VERY little to come in. Most of it is code that I’ve written myself

Quick answer: yes

They need to be identified as such (and not as edges to be searched for in the database), and then they get turned into filters instead

a trivial way to do this might be to look for namespacing with a / character, instead of a : character

yeah that’s what I was thinking actually

though might be better not to put it in pabu?! And just have it with naga?

BTW, I just checked in a pabu modification that allows for -- comments 🙂

awesome! 🙂

Pabu is just a parser that generates Naga rules

It takes a string and returns Naga rules

Incidentally something else I noticed in the README. /* */ comments aren’t iso prolog comments, prolog uses %. Some implementations e.g. swi do additionally support C style comments blocks; but as far as I know they’re non standard.

I was about to show my manager this thing I’d just gotten working, except it was only configurable in code, and I knew him well enough to know that he would immediately ask me to try changing things to see if they worked. That would work best if I hard a parser for rules. Which was why I put Pabu together so quickly. It was never supposed to last. But when I saw how it could run all sorts of simple Prolog code… well, I kept it 🙂

I don’t know Prolog very well, so that’s good to know, thank you

I’m not suggesting you add a 3rd comment form — but might % not be a better choice, as it will probably make your datalog a proper subset of prolog syntax… and thus work properly in emacs prolog-mode etc.

I think sicstus might also support C style ones

errr, well… I’ve already done it, and am just running the regression tests now 😜

oh well the ship has sailed 🚢 👋

I like the -- style, because I kept seeing it in various places. Also SQL

plus, it should make those mulgara rules work

not the mulgara:UriReference function though. I’ll give that one some thought

Yes I quite like them visually… just a shame to miss out on free syntax highlighting support

and automatically commenting out blocks

not a big deal though

A big exception there is Plumatic Schema.

I’m not sure if I should keep that or not. It was so useful during development, and people looking at my code said that they really appreciated seeing it. That said, it does nothing at runtime

@rickmoynihan this morning you inspired me to clean up my builds Try depending on Naga 0.3.12 and tell me what you think about the dependencies

Surprisingly, one of Asami’s plugin dependencies (`cider-nrepl`) was included in the Asami release. I have no idea why this would happen. But I don’t use it anyway, so I’ve removed that, and it’ll propagate through later

BTW, I don’t know if it was clear earlier… there is already external predicate support in Naga (via Asami). So the Pabu-style rules:

rdf:type(A, rdfs:Datatype) :- A(B,C), clojure.core/keyword?(A) .
rdf:type(B, rdfs:Datatype) :- A(B,C), clojure.core/keyword?(B) .
rdf:type(C, rdfs:Datatype) :- A(B,C), clojure.core/keyword?(C) .

(r [?a :rdf/type :rdfs/Datatype] :- [?b ?a ?c] [(keyword? ?a)])
(r [?b :rdf/type :rdfs/Datatype] :- [?b ?a ?c] [(keyword? ?b)])
(r [?c :rdf/type :rdfs/Datatype] :- [?b ?a ?c] [(keyword? ?c)])

I figure it was worth calling out since I’ve been explicit in stating my RDF background, and that a lot of design was influenced by Mulgara

Oh absolutely. I’m very grateful for all your explanations.

Ok this is fantastic and is exactly what I was asking for, I didn’t realise you were saying this, so it’s amazing to see you’re already doing it! :thumbsup: I can imagine this will be very handy.