Clojurians Log v2
datalog
I've been thinking... is there a way to have a datalog database AND also have SQL?
I do love the idea of using a DB with time-travel baked in, and datalog with the pull syntax is really great to use, but SQL really gets the job done in multiple situations...
you could store your data in relational form in SQL, then query and load it into an in-memory datascript DB in your clojure code
There are many issues to be taken into account when trying to represent table data as a graph, and vice versa. There are https://ontosql.inria.fr/rdfdb-stores-rdf-graphs-into-relational-databases/, along with several commercial systems. There are some reasonable approaches that can be made, but it’s difficult to cover the general case.
If you want to represent a graph as SQL (bearing in mind that SQL is a datalog as well), then you need to consider that it will be viewing the data as a virtual table. The approach that has the most general semantics is to consider entities as rows in a table, and every attribute as columns. Does the table structure get defined ahead of time, or just for the life of the query? Entities don’t have to have certain attributes, meaning those columns will be null. Theoretically, every entity could be in every virtual table, but that could mean that some entities have null for every column.
It’s easy for some queries, but harder for others.
e.g. SELECT name, age, employer WHERE age > 21;
could be translated to:
:find ?name ?age ?employer :where [?e :name ?name] [?e :age ?age] [?e :employer ?employer] [(> ?age 21)]
But there are questions that come up even while doing that. SQL is supposed to have a FROM clause. So is there some kind of defined table layout that is added like a schema? What about people without an employer? Those entries are skipped in Clojure semantics, but included in SQL. But what about aggregate queries (count, avg, and so on)? The semantics start getting harder.
Note that I’m not saying that it can’t be done. It HAS been done (I have friends who have built successful businesses around it). But it takes a lot of work, and there are a lot of frustrating edge cases. The point is that it’s usually better to keep your data in a database that is better suited to its shape, and the associated query languages are usually more appropriate for it.
No, I haven’t seen this at all, though it seems like it must be possible. Most databases are built using tree-based indexes (usually B-trees). When we built Mulgara we were inspired by the TUX2 filesystem which used immutable trees, though we were unaware of the prior work in this area. Okasaki cites Myers (Mye82, Mye84) and Sarnak & Tarjan (ST86a) as having pioneered these implementations. We spent a lot of time and effort trying to clean up and re-use tree nodes that were “no longer in use”, particularly in the early days when space was at more of a premium. Over time, I decided that disk was cheap, and I should just leave that old stuff “hanging around”, to be cleaned up at some later date. It wasn’t until I saw Datomic that I realized that we already had this historical record (or “time traveling”). It just had to be indexed. I always meant to do it, but never did… until I wrote Asami (Asami is heavily based on Mulgara, but in pure Clojure). I think that most tree-based databases could be changed to be historical. If they’re record/table based (RDBMS), then it may result in greater fragmentation, and may need a new index to return flat records, but this could be improved by threads in the background. It just needs someone willing to try 🙂
I think postgres now has some built in historical table feature
either that or it's a plugin
@quoll correct me if I'm wrong, but datalog is not turing-complete right? Because I remember pure SQL, even without PL/SQL or TSQL extensions, to be...
As for postgres, it's a plug-in (or a trigger system). I tried to use it but it's less than ideal to be honest 😞
yes, but many datalog implementations have extensions
datalog does not allow you to declare new things, but SQL has INSERT/SELECT
SQL turing completeness apparently came with persistent stored modules. I see a https://stackoverflow.com/questions/900055/is-sql-or-even-tsql-turing-complete
I agree with this 100%… Was just discussing a similar topic in another channel last night: https://clojurians.slack.com/archives/C0904S2QJ/p1616486680004500?thread_ts=1616482516.003700&cid=C0904S2QJ @quoll have you seen a solution that is able to store a graph data model in SQL + does time-travel/history in the Datomic style?
Storing the graph model (1) + retrieving it in the same shape (2) + time travel (3); That would be the holy grail 🙂
