datomic

Ask questions on the official Q&A site at https://ask.datomic.com!
onetom 2020-10-22T07:34:38.106100Z

To make Ion deployment more comfortable, we started using the datomic.ion.dev/push and friends directly. Those function however seem to shell out to call clojure and expect it to be available on the PATH. Since we are using nix-shells to manage our development environments, we deliberately have no clojure on our PATH by default, only within direnv managed shells. Would it be possible to just call the necessary functions directly from the JVM process which runs the datomic.ion.dev/push function? It seems like it's only doing some classpath computation, which should be possible to do directly with clojure.tools.deps...

cmdrdats 2020-10-22T09:44:37.107200Z

We've got a datomic on-prem transactor running, saving the data in mysql - we got this, and it died... how would I go about diagnosing the cause and fixing so it doesn't die?

cmdrdats 2020-10-22T09:45:07.107600Z

we're running datomic-transactor-pro-1.0.6165.jar

favila 2020-10-22T09:51:34.107800Z

super-high-level, the transactor tried to update one of the top-level mutable rows in the database (“pods”) and it couldn’t so it self-destructed. If this happened immediately on txor startup, I would check the connection parameters and credentials are correct and the mysql user has the right permissions (it needs SELECT UPDATE DELETE on table datomic_kvs). Assuming the transactor started correctly and connected to mysql correctly and this happened randomly later, I don’t know. It could be something transient on mysql itself.

favila 2020-10-22T09:52:19.108Z

I would actually look at mysql logs first

cmdrdats 2020-10-22T09:59:07.108200Z

hmm - ok, makes sense - this happened randomly much later

cmdrdats 2020-10-22T10:00:24.108400Z

I do know we've had weird random connection issues on our mysql hosts that we've had to workaround with reconnecting.. so that seems the most likely explanation, thanks for the info!

2020-10-22T17:02:56.111600Z

i'm seeing some old materials around datomic that suggest you can query vanilla clojure data structures, such as vectors of vectors. eg:

(d/q '[:find ?first ?height
       :in $a $b
       :where [$a ?last ?first ?email]
              [$b ?email ?height]]
     [["Doe" "John" "<mailto:jdoe@example.com|jdoe@example.com>"]
      ["Doe" "Jane" "<mailto:jane@example.com|jane@example.com>"]]
     [["<mailto:jane@example.com|jane@example.com>" 73]
      ["<mailto:jdoe@example.com|jdoe@example.com>" 71]])
;; =&gt; #&lt;HashSet [["Jane" 73], ["John" 71]]&gt;
this does not run for me. is there a version of this that does work?

motform 2020-10-22T17:05:37.114100Z

Is it possible to express a recursive query with the pull api where the data looks like :c/b m..1-&gt; :b/a m..1 -&gt; :a syntax starting from a/gid ? I.e. walking down refs (not components) that point “upward” from the top of hierarchy. I can easily do it from the bottom up, from c/gid, but I guess I just don’t get how to reverse the query. EDIT: Never mind, I just realised that this is what _ is for.

favila 2020-10-22T17:07:12.114200Z

This is on-prem with the peer api; if you are using the client api, you need to include a “real” database as a parameter (first parameter?) even if you don’t use it because that is how the client api finds a machine to send the query to

favila 2020-10-22T17:07:51.114400Z

on-prem runs the query in-process; client (typically-not necessarily) sends the query over the network to another process

2020-10-22T17:11:02.114800Z

i see, thank you. it would be terrific to be able to use generic data structures as databases, seems like that would have been the clojure way of doing things as opposed to locking you in to a nominal type

favila 2020-10-22T17:11:44.115Z

the client api doesn’t provide a query engine, so, they’re kind of at cross purposes

favila 2020-10-22T17:13:05.115200Z

to be clear: this works with the client api just fine, but you have to send it to something that can evaluate it

2020-10-22T17:14:58.115400Z

can you say more, i'm not sure i follow, yet. this works - this being, using generic data structures as the db?

favila 2020-10-22T17:15:12.115600Z

(d/q '[:find ?first ?height
       :in $ $a $b
       :where [$a ?last ?first ?email]
              [$b ?email ?height]]
     (d/db client-connection)
     [["Doe" "John" "<mailto:jdoe@example.com|jdoe@example.com>"]
      ["Doe" "Jane" "<mailto:jane@example.com|jane@example.com>"]]
     [["<mailto:jane@example.com|jane@example.com>" 73]
      ["<mailto:jdoe@example.com|jdoe@example.com>" 71]])

2020-10-22T17:15:26.115800Z

i am using datomic.client.api, it's not on-prem

favila 2020-10-22T17:15:47.116100Z

I’m saying that this should work

favila 2020-10-22T17:15:51.116300Z

note I added a client db, but I didn’t use it in the query

2020-10-22T17:16:17.116600Z

oh, interesting.

favila 2020-10-22T17:16:18.116800Z

all of those arguments will be sent to the server (probably not in-process), the query will run, and you will get the result

favila 2020-10-22T17:16:33.117Z

the server that is backing the db object

2020-10-22T17:16:50.117200Z

yeah, that works!

2020-10-22T17:24:17.117400Z

i wonder if there's a way to use this for testing? maybe not, since my production query will necessarily be referring to $ (ie not $a or $b)

2020-10-22T17:25:04.117600Z

it would be great if i could throw together a very simple database out of generic data structures and exercise my production query on that, instead of a real db

favila 2020-10-22T17:32:39.117800Z

You could, but real databases normalize entity references to entity ids for you (e.g. it knows :some-attr-keyword is eid 123). Without that you would have to construct your query or data carefully so that the comparisons are exact

favila 2020-10-22T17:33:04.118Z

also many query helper functions only work on a real database because they use indexes directly

favila 2020-10-22T17:33:08.118200Z

(e.g. d/datoms)

favila 2020-10-22T17:33:23.118400Z

I’m pretty sure get-else would fail, for example

2020-10-22T17:37:23.118600Z

perhaps it's more practical to use a real db in tests, then.

2020-10-22T18:22:18.120400Z

it looks like the two argument comparison predicates such as &lt; work perfectly well to compare instants when inside of a datomic query but not in normal clojure. it's confusing because the documentation says that most of the datomic query functions are the same as those found in clojure.core. anyone have any insight?

schmee 2020-10-22T18:23:34.121Z

@michael740 &lt; and some other common predicates are the exception: https://docs.datomic.com/cloud/query/query-data-reference.html#range-predicates

1🙌
2020-10-22T18:26:00.121500Z

the documentation does not indicate that &lt; works with strings, inst, etc.

2020-10-22T18:26:37.121700Z

i glad they do, though!

jaret 2020-10-22T21:12:34.124800Z

Hi All! I wanted to announce the release of the Datomic Knowledgebase: http://ask.datomic.com/

3👍4🙌
jaret 2020-10-22T21:22:19.125100Z

For anyone wondering... we will be migrating over all of the pendo/receptive requests we've received in the past. So if you don't see something you've requested with me or on our old portal feel free to re-ask or check back again in a week or so.

kenny 2020-10-22T21:23:34.125600Z

Curious when this should be used over the forum.

jaret 2020-10-22T21:25:48.125800Z

The big gain from the forums, which is still the place to have discussions about Datomic Applications and to see announcements -- is to have the upvote button for features and harder questions.

jaret 2020-10-22T21:28:16.126Z

Our previous tool Pendo/receptive had several limitations. It just wasn't as accessible as we wanted it to be to get that feedback loop on what features are important to the community.

jaret 2020-10-22T21:29:50.126200Z

I'll definitely be cross linking/posting from forum posts going forward if we get to a point where a feature is the best answer for whatever is being discussed.