Clojurians Log v2
datomic
@onetom not sure what problem you are describing here. one useful tool is to watch the deploy in the AWS console in “Code Deploy”. That can provide useful info
thanks! i never looked at that console yet, only the various cloudwatch logs.
sure. are you also enabling api-gw level logging (as well as lambda/ion logs)? I have debugged many issues with that level of detail
is there a way to deploy ions from a clojure repl? i tried this:
(datomic.ion.dev/-main
(pr-str
{:op :push
:uname "grace"
:region "ap-southeast-1"
:creds-profile "gini-dev"})):deploy-command (or rather just the map itself, which describes the next operation)
here's what i use, which let's me deploy from the command line to a provided group name via an alias:
$ clj -Adeploy-to-aws some-group-name
There is. 🙂
You can use the functions push and deploy in the namespace datomic.ion.dev
Here's some code from my dev ns on a project. I think you can visually extract the important bits and ignore the project-specific ones:
(defn deploy-unrepro-build!
([]
(deploy-unrepro-build! nil))
([system-config-overrides]
(deploy-unrepro-build! system-config-overrides
(str "dev-" (java.util.UUID/randomUUID))))
([system-config-overrides uname]
(let [system-config (system/get-config system-config-overrides)]
(ion/push {:uname uname})
(ion/deploy {:group (:pawlytics/deployment-group system-config)
:uname uname}))))
(defn deploy-rev-build!
([rev] (deploy-rev-build! rev nil))
([rev system-config-overrides]
(let [system-config (system/get-config system-config-overrides)]
(ion/push {:rev rev})
(ion/deploy {:group (:pawlytics/deployment-group system-config)
:rev rev}))))
(defn deploy-current-rev-build!
([]
(deploy-current-rev-build! nil))
([system-config-overrides]
(deploy-rev-build! (-> (shell/sh "git" "rev-parse" "HEAD")
:out
str/trim-newline)
system-config-overrides)))warning though: repro builds don't check that the working directory is clean like they do using the clj command.
thanks everyone! I will give these a try!
https://github.com/jacobobryant/trident/blob/master/src/trident/ion_dev/deploy.clj
https://gist.github.com/jacobobryant/9c13f4cd692ff69d8f87b0d872aeb64e
Hey, coming from a typical app, if you have heavy read operations, you could spin up a sql read only replica and point your data guys there, safely knowing you wont topple your prod db. How is this generally solved in the on prem postgresql storage datomic world? Something like memcache wont offer much value, its one off huge queries being run.
If the segment churn on your db is low, consider keeping a valcache volume around and remounting it for this big query job
I'm trying to use dev-local, but I'm getting this error: Unable to load client, make sure com.datomic/client-impl-local is on your classpath. Is there a way to add client-impl-local directly to the classpath? I'm not seeing a maven dependency online anywhere
It’s probably a transitive dep. Are you on the latest Clojure CLI version @donyorm ? I know there was an issue with an older Clojure CLI version and dev-local.
How big is your table in bytes and what is your current read/write to Postgres? The load on storage is purely IO—Postgres is basically used as a dumb key-value store. It seems very unlikely (but not impossible) that this is going to be a problem.
Yeah I have the same issue with the newest CLI. I tried looking for the dependency transitively with -Stree, but it wasn't there
We were in DDB, but it became too expensive, so we moved to PostgreSQL. DB is 50GB on clean restore, up to 100 GB, as we need to vacuum the storage
Its a theoretical at this point, postgresql is more than capable, im just curious what the options are, or what other folks are doing for this kind of thing 🙂
Hmm. Well you should not need to add that dep to your cp. Can you paste your -Stree you used to launch your REPL?
Little data modeling question: Say that I have a category with tags, and these tags are component/many of this category. Now, I’d like to add a composite (tuple) key to this tag entity that says [tagName, category] is unique, but there is no explicit relation from tag -> category. Do I have to reverse this relation / lose the component-ness to add this composite key?
Here's what I got
Hmm. Not sure. Guessing someone with deeper knowledge of dev-local needs to help here.
Well thanks for the attempt anyway. I appreciate it!
Only other thing that might help is posting a snippet of what you're doing to get the error.
Fwiw, this is what my dev-local looks like from -Stree
com.datomic/dev-local 0.9.203
com.google.errorprone/error_prone_annotations 2.3.4
com.datomic/client-api 0.8.54
com.google.guava/listenablefuture 9999.0-empty-to-avoid-conflict-with-guava
com.datomic/client 0.8.111
com.cognitect/http-client 0.1.105
org.eclipse.jetty/jetty-client 9.4.27.v20200227
org.checkerframework/checker-compat-qual 2.5.5
com.google.guava/failureaccess 1.0.1
com.google.guava/guava 28.2-android
com.datomic/client-impl-shared 0.8.80
com.cognitect/hmac-authn 0.1.195
com.google.j2objc/j2objc-annotations 1.3
com.datomic/query-support 0.8.27
org.fressian/fressian 0.6.5
com.google.code.findbugs/jsr305 3.0.2
org.ow2.asm/asm-all 4.2I don't see any dep in my -Stree for com.datomic/client-impl-local
Yeah I wonder why it's looking for that
oh I used the wrong type of :server-type in the config. I did :local instead of :dev-local . That would do it
I’m trying to do a fulltext search on my Datomic instance, but I get this error:
The following forms do not name predicates or fns: (fulltext)@lennart.buit This is local. It should be the on-prem version, but I’m connecting via the datomic api client.
Iirc you can’t use fulltext from the client api
That was my suspicion, but I couldn't confirm. So I need to convert my application to use the peer server internally?
or I guess "should", not "need"
That depends on what you’d like to achieve. If your goal is to move to the cloud at some point, you may want to consider sticking with the client API and instead using some other store for your full text needs. Here is a thread on the datomic forums about it: https://forum.datomic.com/t/datomic-fulltext-search-equivalent/874/6
I can’t decide what your architecture should look like, but this is advice I’ve seen before 🙂
Are you using on prem, or cloud?