datomic

Ask questions on the official Q&A site at https://ask.datomic.com!
tatut 2021-05-11T04:52:19.053900Z

in datomic cloud analytics, how do I configure username/password to the trino connector (https://trino.io/docs/current/security/password-file.html) ? the datomic documentation only talks about catalog .properties file and not the overall config.properties

Joe Lane 2021-05-12T04:21:48.087300Z

It's a bit late, but I'm not clear here what you're trying to do? Datomic Analytics (as of today) still runs presto 348, which is a pre-trino version.

tatut 2021-05-12T04:29:15.087500Z

ok, the same config should work there… trino is just a name change

tatut 2021-05-12T04:30:13.087700Z

I’m trying to expose an analytics endpoint via load balancer to our customer, and need to configure options for the connector

tatut 2021-05-12T04:31:04.087900Z

the datomic-cli analytics sync seems to only copy the catalog and metaschema and doesn’t have a way to do other config changes for presto/trino

Joe Lane 2021-05-12T04:33:34.088100Z

Trust me, it's more than just a name change 🙂 Is "our customer" a 3rd party? What options exactly are you trying to configure for the connector?

tatut 2021-05-12T04:34:13.088300Z

the username/password authentication and tls proxy config

tatut 2021-05-12T04:35:18.088500Z

customer is 3rd party

Joe Lane 2021-05-12T04:38:49.088700Z

How many databases will your system have?

tatut 2021-05-12T04:39:35.088900Z

2 that need analytics access

Joe Lane 2021-05-12T04:43:06.089100Z

I'm going to have to sleep on this one.

Joe Lane 2021-05-12T04:43:21.089300Z

Let's reconnect tomorrow?

tatut 2021-05-12T04:43:42.089500Z

sure, I think we will try just ssh’ing and modifying the config.properties in our dev test environment and see that happens

Joe Lane 2021-05-12T04:45:48.089700Z

Something more useful for me would be a sample configuration / project showing that you can expose a secure presto/trino server with the config you need to your customer, taking datomic cloud out of the picture entirely (just for the sample project).

Joe Lane 2021-05-12T04:47:07.089900Z

I'm sure ssh'ing will get it to work once, but it will probably not continue to work upon access gateway restart.

tatut 2021-05-12T05:30:47.090200Z

added 2 lines to /opt/presto-config/config.properties.template

http-server.authentication.type=PASSWORD
http-server.process-forwarded=true
and then added password-authenticator.properties and password.db to /opt/presto-data/etc folder… that worked. Don’t know if that survives restart of the gw

jcf 2021-05-11T13:27:41.059200Z

Hi all! 👋 Hope everyone is doing well today. Is there anything special I need to do to get at Math/abs in a Datomic client query? I seem to remember things like this just working, but this would have been with the peer API…

[:find ?va
 :where 
 [_ :foo/long ?v]
 [(.doubleValue ?v) ?vd]
 [(Math/abs ?d) ?va]]
When I try to execute my query I get an exception, so maybe I need to declare a dependency in my query. I'll perusing the docs now.
1. Caused by clojure.lang.ExceptionInfo
   Unable to load namespace for java.lang.Math/abs
   #:cognitect.anomalies{:category :cognitect.anomalies/not-found,
                         :message
                         "Unable to load namespace for java.lang.Math/abs"}
               require.clj:   53  datomic.core.require/anomaly!
               require.clj:   51  datomic.core.require/anomaly!
               require.clj:   67  datomic.core.require/default-resolver/fn
               require.clj:   64  datomic.core.require/default-resolver
               require.clj:   57  datomic.core.require/default-resolver
               require.clj:   79  datomic.core.require/resolve!
               require.clj:   74  datomic.core.require/resolve!
               datalog.clj: 1342  datomic.core.datalog/resolve-qualified-fn
               datalog.clj: 1336  datomic.core.datalog/resolve-qualified-fn
                 query.clj:  448  datomic.core.query/resolve-qualified-fns
                 query.clj:  445  datomic.core.query/resolve-qualified-fns
                 query.clj:  465  datomic.core.query/parse-query
                 query.clj:  452  datomic.core.query/parse-query
                 query.clj:  469  datomic.core.query/load-query
                 query.clj:  468  datomic.core.query/load-query
I'm assuming I'll need to add type hints to prevent reflection too; hoping primitive types are all good…

Joe Lane 2021-05-11T13:31:07.059400Z

Try typehinting that first, it may not be able to find the right method without it.

jcf 2021-05-11T13:32:41.059600Z

@jcf I have a ^long and a ^double hint in my query, and I'm seeing the same exception.

Joe Lane 2021-05-11T13:33:28.059800Z

paste it again with these new hints?

jcf 2021-05-11T13:33:59.060Z

Can't type hint a primitive local… I thought that might be a problem.

Joe Lane 2021-05-11T13:35:28.060200Z

Show me the query with the hints

jcf 2021-05-11T13:36:27.060400Z

'[:find (sum ?va)
  :with ?e
  :where
  [?e :transfer/amount ?v]
  [(.doubleValue ?v) ?vd]
  [(Math/abs ^double ?vd) ?va]]

jcf 2021-05-11T13:36:56.060600Z

I get a result when I ditch the use of Math/abs and sum the ?vd.

Joe Lane 2021-05-11T13:37:10.060800Z

Can you show that query as well

jcf 2021-05-11T13:37:51.061Z

'[:find (sum ?vd)
  :with ?e
  :where
  [?e :transfer/amount ?v]
  [(.doubleValue ?v) ?vd]]

jcf 2021-05-11T13:38:06.061200Z

That gives me back a negative double.

jcf 2021-05-11T13:38:34.061400Z

clj-kondo is warning me about reflection. That's a great library!

Joe Lane 2021-05-11T13:39:36.061600Z

Now go for the minimal repro.

'[:find ?va
  :where
  [(ground 42.0) ?vd]
  [(Math/abs ^double ?vd) ?va]]

jcf 2021-05-11T13:40:48.061800Z

Same exception with your minimal repro.

jcf 2021-05-11T13:41:02.062Z

1. Caused by clojure.lang.ExceptionInfo
   Unable to load namespace for Math/abs
   #:cognitect.anomalies{:category :cognitect.anomalies/not-found,
                         :message "Unable to load namespace for Math/abs"}
               require.clj:   53  datomic.core.require/anomaly!
               require.clj:   51  datomic.core.require/anomaly!

jcf 2021-05-11T13:42:17.062200Z

A call to Math/abs works outside of the query, which is what made me wonder if I need to whitelist the Math namespace, but I think everything in java.lang is available by default.

Joe Lane 2021-05-11T13:42:29.062400Z

'[:find ?va
  :where
  [(ground 42.0) ?vd]
  [(java.lang.Math/abs ^double ?vd) ?va]]

jcf 2021-05-11T13:43:05.062600Z

Adding java.lang doesn't help.

Joe Lane 2021-05-11T13:43:35.062800Z

Hmm.. Can you open a support case for this so I can look into it?

jcf 2021-05-11T13:44:20.063Z

Where's the place to open support cases these days? http://support.datomic.com?

Joe Lane 2021-05-11T13:44:34.063200Z

Yep, same as always

Joe Lane 2021-05-11T13:46:19.063400Z

We've also got this handy format that prevents roundtrips https://docs.datomic.com/cloud/tech-notes/writing-a-problem-report.html

jcf 2021-05-11T13:52:33.063600Z

@lanejo01 want me to log an issue for this Zendesk error too? I can't create a password because of some janky iframe stuff from the looks of it.

Joe Lane 2021-05-11T13:53:32.064Z

Yes, that's weird.

Joe Lane 2021-05-11T13:53:49.064200Z

Please include your browser details

Joe Lane 2021-05-11T13:54:42.064400Z

and if possible a .har file network recording (or the firefox equivalent) of the network requests made.

jcf 2021-05-11T13:58:05.064600Z

The support email that gets sent out links to a different doc on what info to provide with support requests, and it's from 2016: https://support.cognitect.com/hc/en-us/articles/215581538-Information-to-provide-with-a-support-request

jcf 2021-05-11T13:58:48.064900Z

It doesn't mention providing version numbers, which is probably more helpful with a problem on top of an in-memory database. 🙂

jcf 2021-05-11T14:00:22.065100Z

Quick fix for the Zendesk iframe jank is to open the iframe in a new tab, and then submit the form.

jcf 2021-05-11T14:08:03.065300Z

I've logged the support request. Thanks, @lanejo01! 🙇

Joe Lane 2021-05-11T14:08:30.065500Z

Thank you @jcf!

Yarin Kessler 2021-05-11T15:04:19.067100Z

• Hi all. So I was going through the ion-starter tutorial, and ran into the following error at https://docs.datomic.com/cloud/ions/ions-tutorial.html#test-your-connection :

- Downloading: com/datomic/ion/0.9.50/ion-0.9.50.pom from datomic-cloud 
- Downloading: com/datomic/ion/0.9.50/ion-0.9.50.jar from datomic-cloud 
- Error building classpath. Could not find artifact com.datomic:ion:jar:0.9.50 in central (<https://repo1.maven.org/maven2/>)
◦ Here’s the project’s deps.edn for reference: https://github.com/Datomic/ion-starter/blob/master/deps.edn ◦ I was able to resolve this by adding full S3 access permissions to my IAM Datomic user, based on this tip from https://clojurians-log.clojureverse.org/datomic/2021-03-14/1615742989.291900. However, I’m not clear on why that helped. Why would giving full access to my S3 account help with locating an external jar? I’m completely new to Java/Maven/tools.deps ecosystem so feel free to ELI5. Thanks!

alexmiller 2021-05-11T15:08:00.068800Z

the ion jars are provided in a Maven repository hosted on s3. while the bucket is public, you must have IAM creds with access to S3 to read it

Joe Lane 2021-05-11T15:08:50.069300Z

FWIW, you shouldn't need S3 full access.

Yarin Kessler 2021-05-11T15:20:42.072100Z

So my Datomic user was set up according to instructions here: https://docs.datomic.com/cloud/getting-started/configure-access.html#authorize-user Is there a reason that setup doesn’t have the necessary S3 creds baked in?

Yarin Kessler 2021-05-11T15:21:01.072300Z

Yea, I figured I don’t need full access, but I don’t know what specifically I do need? Still confused as to why giving access to MY S3 account would affect access to a bucket outside of my account.

Joe Lane 2021-05-11T15:28:52.072500Z

The issue that you haven't given your user access to read ANY S3 buckets, even public ones.

Yarin Kessler 2021-05-11T15:43:15.072700Z

Say I were to grant universal read access via an AmazonS3ReadOnlyAccess policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": "*"
        }
    ]
}
What that means to me is that I have granted full read access to my buckets. What would differentiate a policy that gave read access to my buckets vs a policy that gave read access to outside public buckets?

Joe Lane 2021-05-11T15:46:03.073Z

You should restrict the resource arn I believe.

Yarin Kessler 2021-05-11T16:19:36.073200Z

Tried this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": "arn:aws:s3:::datomic-releases-1fc2183a/maven/releases/*"
        }
    ]
}
But AWS doesn’t allow it. You can’t set arn to point to outside resource AFAICT. Which means there’s no way to say “You can read S3 public buckets but not my buckets”. Which honestly aligns with my original instinct that external public buckets are public resources and so applying permissions against them makes no sense. So I’m still massively confused.

Pragyan Tripathi 2021-05-11T20:57:54.080700Z

I started learning datomic/datalog today. I have following pull query that works:

(d/pull db '[*] [:block/id #uuid 0000000-0000-0000-000]) ;; based on unique
Now I want to another resolver that returns a vector filtered based on block/tags I couldn’t figure out how to write pull query for that:
(d/pull db '[*] [:block/tags :button])
The sample data looks like following:
[{:db/id "block-button-id"
  :block/id (uuid-from-string "block-button-1")
  :block/tags [:button]
  :block/display "Button Block 1"
  :block/description "Button Block 1"
   :block/value 101155069755482}
  {:db/id "block-button-2"
   :block/id (uuid-from-string "block-button-2")
   :block/tags [:button]
   :block/display "Button Block 2"
   :block/description "Button Block 2"
   :block/value 101155069755483}]
Apologies if it is a trivial question, I would appreciate any help in learning it.

Joe Lane 2021-05-11T21:12:58.082400Z

Hi @pntripathi9417, I think you're looking for a query, not a pull.

(d/q '[:find (pull ?b [*])
       :where 
       [?b :block/tags :button] db)

👍 1
naomarik 2021-05-11T23:39:51.085900Z

Anything exist that's more updated than this? https://github.com/dazld/awesome-datomic