Clojurians Log v2

Hey folks, I want your opinion. I need to put in my datomic schema an ordered list of refs to other entities. Since datomic doesn't support that natively, how would you do that? The difficult part is probably how to make sure the ordering stays consistent. Are composite tuples a good way of modeling that? :thinking_face:

What I had in mind was something like suggested here: https://forum.datomic.com/t/handling-ordered-lists/305/5

Hey! Our team is using Datomic Analytics for some dashboards. One metric is based on the depth of trees in our system. I noticed the query sometimes breaks, I think in datomic-land. The result of the query is:

java.sql.SQLException: Query failed (#20210309_090647_00001_tab9g): bytes is negative

WITH RECURSIVE t(lvl, parent, db__id) AS (
  SELECT 1, parent, db__id
  FROM mydb.node
  WHERE parent IS NULL
  UNION ALL
  SELECT lvl + 1, t.parent, t.db__id
  FROM mydb.node
  JOIN t
    ON mydb.node.parent = t.db__id
)
SELECT * FROM t LIMIT 1;

The node table is simply db__id, parent, and name. where parent refers to another node (or nil)

It sometimes happens. So running the query multiple times yields different results. We're currently on datomic 1.0.6222.

Spotted another odity; binding the same attribute multiple times doesn't work (in datomic analytics):

java.sql.SQLException: Query failed (#20210309_101720_00072_tab9g): Symbol name already has assignment "name_20", while adding "name_24"

WITH RECURSIVE t(lvl, parent, db__id, name, root_name) AS (
  SELECT 1, r.parent, r.db__id, r.name, r.name
  FROM mydb.node AS r
  WHERE r.parent IS NULL
  UNION ALL
  SELECT lvl + 1, c.parent, c.db__id, c.name, root_name
  FROM mydb.node as c
  JOIN t
    ON c.parent = t.db__id
)

SELECT * FROM t ORDER BY t.lvl

https://docs.datomic.com/cloud/client/client-api.html#timeouts says that timeouts are normally returned as ::anom/unavailable, but we seem to get a lot of ::anom/interrupted exceptions where the text content is Datomic Client Timeout. Is there some bad way of interacting we're likely to be doing that would cause us to get interrupted rather than unavailable?

If I have a to-many ref attribute, is there an easy way to query for that being empty?

I suppose maybe missing? is the easiest way.

I'm not quite sure how to use missing? in an or clause, though, because of the unification requirement. Hmm.

@jeroen.dejong how consistently can you reproduce? Once you can consistently, can you upgrade to the latest release and then try to repro again?

This ball is entirely in prestosql’s court.

Fair enough! It was hard to judge this one. I simply wrapped the value in a array, which did the trick.

I could not repro consistently. But I noticed an error in my query. Its recurring until OOM. Sometimes returning "out of resource " (expected). Sometimes returning the bytes-error. I'll upgrade the cluster later this week anyway and see if I can reproduce.

What is the size of your analytics gateway and is it pointed at a query group?

I'm using on prem, right now it's a test setup with 10GB ram per query and 16GB max object heap. As a reference; when I fixed the query it consumed 1.5GB, the entire datomic DB is under a gig

Ahh, I was thinking you were on cloud.

I think that discussion is pretty sound. you end up with a "table" of [order, ref] pairs. there's not really a better way to do it and keep it queryable. abusing tuples for a dynamically-sized list is not likely to end well.

https://forum.datomic.com/t/datomic-1-0-6269-now-available/1798

Both are retriable...

Hi. I'm trying to follow the Datomic Ions tutorial, but am stuck on the https://docs.datomic.com/cloud/ions/ions-tutorial.html#configure-connection I'm not sure how to determine the :endpoint : I have hacked up

:endpoint "<http://entry.mbrainz-stu.us-east-1.datomic.net:8182>"

datomic -r eu-west-1 client access 

Ah found it. It was in the Outputs of the Compute Stack

hmmm how do I specify an aws region in

clojure -A:ion-dev '{:op :push}'

{:command-failed "{:op :push}",
 :causes
 ({:message
   "Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region.",
   :class SdkClientException})}

a somewhat low-tech

export AWS_REGION=eu-west-1

@benha See push docs: https://docs.datomic.com/cloud/ions/ions-reference.html#push 🙂

Add :region "my-region-1"

this seems an unfortunate error message upon the deploy

{:command-failed
 "{:op :deploy, :group vorpal-ion-starter-Compute-A2UL4PZOZHGU, :uname \"benjy-1\"}",
 :causes
 ({:message
   "A Lambda function name, which is the concatenation of the query group and the function name specified in the ion-config.edn file must be <= 64 characters.\nThe following names are too long for Lambda: vorpal-ion-starter-Compute-A2UL4PZOZHGU-get-items-by-type-lambda-proxy",
   :class RuntimeException})}

I have control of the first 18 chars of that compute group name but not the last 20

Yeah: https://docs.datomic.com/cloud/ions/ions-reference.html#lambda-config. I've hit this before too 😞

Perhaps they should add a warning note to the tutorial on this point.

so the CodeDeployApplicationName should be no more than 11 characters long

do I have to tear the whole stack down and recreate it?

that really ought to be in the tutorial if so

I don't believe you can change the name without recreating it.

In https://docs.datomic.com/cloud/operation/planning.html#naming, they recommend keeping names under 24 characters.

https://docs.datomic.com/cloud/getting-started/start-system.html#details

yeah just saw that. Ah well, thats a sign to pack it in for the evening

thanks for your help

Sure thing. Sorry the result isn’t ideal 😞

its all about the journey