datomic

Ask questions on the official Q&A site at https://ask.datomic.com!
greg 2021-06-15T12:02:42.213400Z

Hi, I'm trying to build a first app using datalog/datascript. In the db I want to store FX exchange rates. For each pair (eg. GBP/USD, GBP/EUR), for every day, one value. I'm wandering how to design schema for such application. I'm struggling with what should be marked as an entity. Currency, currency pair, currency pair for given date, or all actually? I'd much grateful for some ideas or sample schemas that you think might make sense. Thanks

Joe Lane 2021-06-15T13:28:52.213700Z

Hi @grzegorz.rynkowski, are exchange rates directional? e.g. [#inst "2021-01-01" :GBP :USD 10] but [#inst "2021-01-01" :USD :GBP 8]?

2021-06-15T14:21:44.214Z

Hi. Can you be more concrete on a wrapper intercepting the protocol function?

2021-06-15T14:21:59.214200Z

How do you achieve this functionality?

greg 2021-06-15T14:45:47.214400Z

@lanejo01 yes, this kind of situation is possible: [#inst "2021-01-01" :GBP :USD 2] [#inst "2021-01-01" :USD :GBP 0.4] In addition there might be more then one source of rates, so there might be: [#inst "2021-01-01" :GBP :USD [:name "BoE"] 2.01] [#inst "2021-01-01" :GBP :USD [:name "HMRC"] 2.03]

kschltz 2021-06-15T17:13:36.215200Z

you could achieve that via a defrecord implementing said protocol

jdkealy 2021-06-15T17:34:31.216100Z

What's the best way to get two strings like this into an instant that i can save in datomic? "2021-06-08T16:30:12" "America/New_York" I tried using clj-time but no dice

(let [datetime-vec (take 8 (parse-timestamp vector date-str))]
     (-> (apply zoned-date-time datetime-vec)
         (with-zone UTC-offset)
         (instant)))
I get
:db.error/wrong-type-for-attribute Value 2021-06-08T16:30:12-04:00[America/New_York] is not a valid :inst for attribute :session/start_time
` even though the output of my function is an instant Trying to insert
#inst "2021-06-08T20:30:12.000000000-00:00"

tvaughan 2021-06-15T17:41:39.216500Z

Perhaps this https://forum.datomic.com/t/java-time/1406 is relevant?

Joe Lane 2021-06-15T17:43:25.216800Z

@jdkealy For now you gotta turn that into a j.u.Date before you persist it in Datomic.

jdkealy 2021-06-15T17:44:16.217Z

what's a j.u.date a google search is giving me linkes to jdate πŸ™‚

jdkealy 2021-06-15T17:47:11.217200Z

i guess, Java Util Date ?

jdkealy 2021-06-15T17:48:26.217400Z

(java.util.Date. (tz/to-instant "2021-06-08T16:30:12" "America/New_York"))

jdkealy 2021-06-15T17:50:55.217600Z

This seems to work

([date-str, UTC-offset]
   (let [datetime-vec (take 8 (parse-timestamp vector date-str))]
     (-> (apply zoned-date-time datetime-vec)
         (with-zone UTC-offset)
         (instant)
         inst-ms
         (java.util.Date.))))

2021-06-15T17:51:00.217800Z

Yeah, we just have a deftype somewhere that forwards most calls to the β€˜original’ db/`connection`/`client`, but amends their implementation:

(deftype MyWrappedDb [orig]
  datomic.impl/Queryable
  (q [_ arg-map] (println arg-map) (datomic.impl/q orig arg-map))
  ...)
That said, I’m not saying you should do this β€” I think its perhaps better to use dev-local to create a memory db for testing, but if you ever feel like you need to intercept calls on db values, this is a way to do so.

jdkealy 2021-06-15T17:51:53.218Z

input

"2021-06-08T16:30:12" "America/New_York"
output
#inst "2021-06-08T20:30:12.000-00:00"
looks legit πŸ™‚

1🎸
Joe Lane 2021-06-15T17:53:49.218200Z

To give better advice I'd need to know more about the rest of the app, but for those fx entities I'd lean towards representing :GBP->:USD as an entity with a composite tuple of :loc/from and :loc/to where the composite tuple represents identity. Then for the rates i'd want to know what the access patterns will be and the growth of the dimensions. The rate entity could be something like:

{:time/at #inst "2021-01-01
 :rate/from-to {:loc/from :GBP :loc/to :USD}
 :rate/source {:source/name "BoE}
 :rate/amount 2.01}
That would then have a composite tuple of
[#inst "2021-01-01" 123 456 2.01]
Where 123 is the directed exchange and 456 is the rate source. You can add additional composite tuples to allow different access patterns in exchange for space. That being said, if the number of datoms stays small ( sub 1-billion) then who needs the extra tuples.

1πŸ‘1πŸ‘
Joe Lane 2021-06-15T17:57:47.218500Z

FWIW, if you care about the TZ when you query obviously you'll need to stash that as another attribute and reconstitute a ZonedDateTime on the way out. Avoid querying with zones if you can, it can make simple queries slow and complicated.

Joe Lane 2021-06-15T18:00:12.218700Z

@i that protocol is an impl detail and subject to change. I'd like to go back to your original problem statement. What problem will you solve when you "mask out these functions and return faked ones."?

Joe Lane 2021-06-15T18:27:40.219300Z

Oh also, you can use (Date/from your-inst)

jdkealy 2021-06-15T19:53:07.219500Z

cool thanks! Yes, I'm saving 3 attrs, the timezone, the original string, the converted instant (just in case i screw something up, i can regenerate them )

jdkealy 2021-06-15T19:53:56.219700Z

I'll only be querying for stuff in the immediate future, so i think I'll just be passing (> (java.util.Inst))

greg 2021-06-15T22:26:12.221600Z

When accessing raw index (listing datoms using ), is there a way of substituting numbers by entities itself? I was checking https://docs.datomic.com/cloud/query/raw-index-access.html, but there is nothing about such a thing. Example: when I preview datoms for an example data set

(d/datoms db-solar-system {:index :eavt})
I receive something like that:
...
 #datom[74766790688845 73 "Sun" 13194139533319 true]
 #datom[74766790688845 74 696000.0 13194139533319 true]
 #datom[74766790688846 73 "Jupiter" 13194139533319 true]
 #datom[74766790688846 74 69911.0 13194139533319 true]
 #datom[74766790688847 73 "Saturn" 13194139533319 true]
 #datom[74766790688847 74 58232.0 13194139533319 true]
...
And I would like to see attribute names instead of its number. Is there some Datomic API to do that?

2021-06-16T10:45:58.222700Z

the numbers are eids, so you can just pull them using d/pull. You could query all attribute names, and use the query-result to map over the 4th entry to make it more efficient. depending your usecase.

2021-06-15T23:41:45.222200Z

I want to avoid connecting to real database for testing some functions that indirectly writes to db.