Clojurians Log v2

thanks @lanejo01, I'll check it out

hi - we're trying to restore a backup of our datomic database (it's a tiny 9mb db, as a test), and it just seems to be hanging forever.. how do we go about figuring out what we're doing wrong? We've set the log level in the logback.xml for datomic to TRACE - and theres a tiny bit of logging (I'll attach that in thread), but nothing else

I am trying to get started with datomic in a dev-local setup. Any idea why my dev-local db is not found? I can see can see db.log and log.idx files were created.

(ns flow.db
  (:require [datomic.client.api :as d]))

(let [dir     (str (System/getProperty "user.dir") "/var/datomic")
      db-name "flow"
      client  (d/client {:server-type :dev-local
                         :storage-dir dir
                         :system      "dev"})
      _       (d/create-database client db-name)
      conn    (d/connect client {:db-name db-name})]
  conn)

;; Unhandled clojure.lang.ExceptionInfo
;; Db not found: flow
;; #:cognitect.anomalies{:category :cognitect.anomalies/not-found,
;;                       :message "Db not found: flow"}

Can you try testing with the absolute path of dir and just execute d/client without the let? Also does the absolute path already have a "dev" system folder?

@cmdrdats Can you share what command you're using to run backup/restore?

If you'd like feel free to log a case to me by e-mailing <mailto:support@cognitect.com|support@cognitect.com> and we can dive further.

then list-dbs on the client

(d/list-databases client {})

Sure.

(def client (d/client {:server-type :dev-local
                       :storage-dir "/home/jam/src/flow/var/datomic"
                       :system      "dev"}))
(d/create-database client "flow")  ; => true
(d/list-databases client {})   ; => []

file system looks like this:

$ tree /home/jam/src/flow
/home/jam/src/flow
├── deps.edn
├── <http://FLOW.org|FLOW.org>
├── src
│   └── flow
│       └── db.clj
└── var
    └── datomic
        └── dev
            ├── db.log
            └── log.idx

I created var/datomic. dev/ gets created by create-database fn

I tried setting :storage-dir in ~/.datomic/dev-local.edn and it has the same problem. Creates some files but no db found.

What version of Dev-local are you using?

Can you share the .datomic/dev-local.edn file in your home directory?

Might be best to just share your deps.edn and I will try to re-create. So I can see version of client and dev-local.

jam@stick:~/.datomic$ cat dev-local.edn 
{:storage-dir "/home/jam/var/datomic"}

jam@stick:~/opt$ java -version
openjdk version "11.0.8" 2020-07-14

@millettjon I don't see a version of client in your deps? Do you have datomic client in your .clojure/deps.edn? Could you include com.datomic/client-cloud "0.8.102"

Also worth testing with the latest dev-local and use 0.9.225

Ok. I was following instructions here: https://docs.datomic.com/cloud/dev-local.html and didn't know about that additional dep. Unfortunately, adding it didn't make any difference. I will try updating to 0.9.225.

@millettjon I can't reproduce is there anything else I could be missing? Are you starting a new repl to do this? Can you try making a new system name to confirm you are local when a new dir is made? What OS are you using on your system?

What's the best way to report/ask about questions on the official datomic docs? e.g The docs for import-cloud on the docs: https://docs.datomic.com/cloud/dev-local.html#import-cloud Don't match the docstring. I assume the docstring is correct as the documentation page lists the same value for source and dest.

There’s a new ask.datomic site that probably fits the bill

should one be alarmed when periodically seeing an Unable to load index root ref <uuid> exception appear in Datomic Cloud logs? (from com.amazonaws.services.dynamodbv2.model.InternalServerErrorException )

@jaret we're doing this:

/storage/datomic/current/bin/datomic -Xmx1g -Xms1g restore-db "file:/storage/datomic/archive/restore/" "datomic:sql://...?jdbc:<mysql://192.169.56.1:3305/datomic?user=...&amp;password=>..."

look at the args for create-database

needs to be {:db-name "flow"} not "flow"

@millettjon @jaret

!!! Ghadi, great catch!

That's it

calling it with a raw string probably shouldn't return true

No it should not!

Try restoring locally using :dev protocol, just to see if it's underlying storage related. But if it couldn't write to storage due to perms I would expect an error not a hang.

Thanks!

I'll fix that!

Thanks for catching it.

Do you often delete DBs? In general, you can see this error if a client or node is asking for a deleted db. The error would also correlate to an outage but these calls are retried so if you don't see this error often or repeatedly it's probably not a major issue.

As always, it is my support-person duty to recommend upgrading to the latest Cloud release CFT and if you'd like me to look more closely at your system logs I'd be happy to poke around with a Read-Only CloudWatch account. If you want to go down that path, log me a case at <mailto:support@cognitect.com|support@cognitect.com> and I can take a look.

When i try to use dev-tools to import a cloud locally it complains that it can't use the endpoint to connect/"name or service is not known". I'm connected and can query the databases though so i'm not sure what the issue is.