datomic

Ask questions on the official Q&A site at https://ask.datomic.com!
steveb8n 2020-10-15T00:17:43.004900Z

@onetom not sure what problem you are describing here. one useful tool is to watch the deploy in the AWS console in “Code Deploy”. That can provide useful info

onetom 2020-10-15T00:18:34.005Z

thanks! i never looked at that console yet, only the various cloudwatch logs.

steveb8n 2020-10-15T00:22:28.005200Z

sure. are you also enabling api-gw level logging (as well as lambda/ion logs)? I have debugged many issues with that level of detail

onetom 2020-10-15T01:45:30.007Z

is there a way to deploy ions from a clojure repl? i tried this:

(datomic.ion.dev/-main
    (pr-str
      {:op :push
       :uname "grace"
       :region "ap-southeast-1"
       :creds-profile "gini-dev"}))
but it quits my repl after executing the operation. it would be nice to just expose the function which gets called with that map provided as a command line argument and return the printed result map, so i can just grab the :deploy-command (or rather just the map itself, which describes the next operation)

joshkh 2020-10-15T13:14:37.010Z

here's what i use, which let's me deploy from the command line to a provided group name via an alias:

$ clj -Adeploy-to-aws some-group-name
https://gist.github.com/joshkh/3455a6905517a814b4623d01925baf0e

2020-10-19T00:20:15.063100Z

There is. 🙂 You can use the functions push and deploy in the namespace datomic.ion.dev Here's some code from my dev ns on a project. I think you can visually extract the important bits and ignore the project-specific ones:

(defn deploy-unrepro-build!
  ([]
   (deploy-unrepro-build! nil))
  ([system-config-overrides]
   (deploy-unrepro-build! system-config-overrides
                          (str "dev-" (java.util.UUID/randomUUID))))
  ([system-config-overrides uname]
   (let [system-config (system/get-config system-config-overrides)]
     (ion/push {:uname uname})
     (ion/deploy {:group (:pawlytics/deployment-group system-config)
                  :uname uname}))))

(defn deploy-rev-build!
  ([rev] (deploy-rev-build! rev nil))
  ([rev system-config-overrides]
   (let [system-config (system/get-config system-config-overrides)]
     (ion/push {:rev rev})
     (ion/deploy {:group (:pawlytics/deployment-group system-config)
                  :rev   rev}))))

(defn deploy-current-rev-build!
  ([]
   (deploy-current-rev-build! nil))
  ([system-config-overrides]
   (deploy-rev-build! (-> (shell/sh "git" "rev-parse" "HEAD")
                          :out
                          str/trim-newline)
                      system-config-overrides)))

2020-10-19T00:21:11.063300Z

warning though: repro builds don't check that the working directory is clean like they do using the clj command.

onetom 2020-10-19T11:40:55.066300Z

thanks everyone! I will give these a try!

Erwee 2020-10-15T08:06:54.009900Z

Hey, coming from a typical app, if you have heavy read operations, you could spin up a sql read only replica and point your data guys there, safely knowing you wont topple your prod db. How is this generally solved in the on prem postgresql storage datomic world? Something like memcache wont offer much value, its one off huge queries being run.

favila 2020-10-16T23:42:03.035400Z

If the segment churn on your db is low, consider keeping a valcache volume around and remounting it for this big query job

joshkh 2020-10-15T13:14:37.010Z

here's what i use, which let's me deploy from the command line to a provided group name via an alias:

$ clj -Adeploy-to-aws some-group-name
https://gist.github.com/joshkh/3455a6905517a814b4623d01925baf0e

donyorm 2020-10-15T18:15:49.011400Z

I'm trying to use dev-local, but I'm getting this error: Unable to load client, make sure com.datomic/client-impl-local is on your classpath. Is there a way to add client-impl-local directly to the classpath? I'm not seeing a maven dependency online anywhere

kenny 2020-10-15T18:48:52.012800Z

It’s probably a transitive dep. Are you on the latest Clojure CLI version @donyorm ? I know there was an issue with an older Clojure CLI version and dev-local.

favila 2020-10-15T18:56:32.016500Z

How big is your table in bytes and what is your current read/write to Postgres? The load on storage is purely IO—Postgres is basically used as a dumb key-value store. It seems very unlikely (but not impossible) that this is going to be a problem.

donyorm 2020-10-15T18:56:41.016700Z

Yeah I have the same issue with the newest CLI. I tried looking for the dependency transitively with -Stree, but it wasn't there

Erwee 2020-10-15T19:01:23.016900Z

We were in DDB, but it became too expensive, so we moved to PostgreSQL. DB is 50GB on clean restore, up to 100 GB, as we need to vacuum the storage

Erwee 2020-10-15T19:02:43.017100Z

Its a theoretical at this point, postgresql is more than capable, im just curious what the options are, or what other folks are doing for this kind of thing 🙂

kenny 2020-10-15T19:06:37.018400Z

Hmm. Well you should not need to add that dep to your cp. Can you paste your -Stree you used to launch your REPL?

2020-10-15T20:50:56.018800Z

Little data modeling question: Say that I have a category with tags, and these tags are component/many of this category. Now, I’d like to add a composite (tuple) key to this tag entity that says [tagName, category] is unique, but there is no explicit relation from tag -> category. Do I have to reverse this relation / lose the component-ness to add this composite key?

donyorm 2020-10-15T20:57:48.019Z

Here's what I got

kenny 2020-10-15T21:39:36.019400Z

Hmm. Not sure. Guessing someone with deeper knowledge of dev-local needs to help here.

donyorm 2020-10-15T21:40:05.019600Z

Well thanks for the attempt anyway. I appreciate it!

kenny 2020-10-15T21:40:06.019800Z

Only other thing that might help is posting a snippet of what you're doing to get the error.

kenny 2020-10-15T21:41:08.020Z

Fwiw, this is what my dev-local looks like from -Stree

com.datomic/dev-local 0.9.203
  com.google.errorprone/error_prone_annotations 2.3.4
  com.datomic/client-api 0.8.54
  com.google.guava/listenablefuture 9999.0-empty-to-avoid-conflict-with-guava
  com.datomic/client 0.8.111
    com.cognitect/http-client 0.1.105
      org.eclipse.jetty/jetty-client 9.4.27.v20200227
  org.checkerframework/checker-compat-qual 2.5.5
  com.google.guava/failureaccess 1.0.1
  com.google.guava/guava 28.2-android
  com.datomic/client-impl-shared 0.8.80
    com.cognitect/hmac-authn 0.1.195
  com.google.j2objc/j2objc-annotations 1.3
  com.datomic/query-support 0.8.27
  org.fressian/fressian 0.6.5
  com.google.code.findbugs/jsr305 3.0.2
  org.ow2.asm/asm-all 4.2

kenny 2020-10-15T21:41:50.020200Z

I don't see any dep in my -Stree for com.datomic/client-impl-local

donyorm 2020-10-15T21:45:04.020500Z

Yeah I wonder why it's looking for that

donyorm 2020-10-15T21:46:36.020700Z

oh I used the wrong type of :server-type in the config. I did :local instead of :dev-local . That would do it

1✔️
Brandon Olivier 2020-10-15T22:03:42.022100Z

I’m trying to do a fulltext search on my Datomic instance, but I get this error:

The following forms do not name predicates or fns: (fulltext)
Anybody know why that might be? I’m following straight from what’s in the docs

Brandon Olivier 2020-10-16T15:19:41.024900Z

@lennart.buit This is local. It should be the on-prem version, but I’m connecting via the datomic api client.

2020-10-16T15:20:38.025800Z

Iirc you can’t use fulltext from the client api

Brandon Olivier 2020-10-16T18:30:26.026600Z

That was my suspicion, but I couldn't confirm. So I need to convert my application to use the peer server internally?

Brandon Olivier 2020-10-16T18:30:34.026800Z

or I guess "should", not "need"

2020-10-16T19:34:30.028900Z

That depends on what you’d like to achieve. If your goal is to move to the cloud at some point, you may want to consider sticking with the client API and instead using some other store for your full text needs. Here is a thread on the datomic forums about it: https://forum.datomic.com/t/datomic-fulltext-search-equivalent/874/6

2020-10-16T19:34:59.029200Z

I can’t decide what your architecture should look like, but this is advice I’ve seen before 🙂

2020-10-15T22:15:05.022200Z

Are you using on prem, or cloud?