Ask questions on the official Q&A site at https://ask.datomic.com!
tatut 2021-04-19T05:02:56.324700Z

that looks quite tricky, can’t really say how make the query faster, but if you could update some attr on the root every time you update some descendant (like :root/recursive-modification-time timestamp), you could simply query all roots based on that

tatut 2021-04-19T05:04:19.324900Z

in effect moving the “has some descendant changed” determination from query time to tx time


Hey all. I asked a little while back about doing multi-tenant apps with datomic and a couple of ideas came up. One of those was using a single DB per customer. So the connection string might look like : "datomic:<dev://localhost:4334/customer_id>" This looks interesting - do this mean that all databases backed by the same Postgres instance (for example) share the same DB? In which case, the only different between: "datomic:<dev://localhost:4334/customer_id_1>" and "datomic:<dev://localhost:4334/customer_id_2>" Is some sort of partioning key that Datomic uses to keep data separate within it's backing store?


If that is the case, then is it feasable to "open" a new DB connection per http request? or keep a pool of conns in some sort of LRU cache? From the docs

Datomic connections do not adhere to an acquire/use/release
pattern. They are thread-safe and long lived. Connections are
cached such that calling datomic.api/connect multiple times with
the same database value will return the same connection object.

prnc 2021-04-19T10:25:04.328300Z

Hi 👋 I am looking into application level access restriction/permissions to entities. I couldn’t find any specific guidelines for cloud, that would allow for a separation of concerns between query logic and permissions. In on-prem filter seems to be the useful for that. What are some good ways to go about this for datomic cloud? [It depends ofc, it’s early stage so I don’t have specific requirements, just looking for some general guidelines/mechanisms in datomic that could be leveraged]. Thanks!

prnc 2021-04-20T15:10:11.352Z

Thanks @tatut! Being new to datomic this is quite an interesting problem for me. The filter solution (on-prem only!) seems to be very clean, and it’s what nubank are using. Query logic is separated from access restrictions. Leveraging “database as a value” and thus playing to datomic strengths. The other nice thing about datomic is ofc “queries are data” as mentioned by @lanejo01 I’ve just used this approach to write a boundary query-as-a-user fn, which rewrites the queries to refer to access restriction rule. Somehow it doesn’t really feel right, less separate, less explicit. Maybe it’s just my shoddy impl 😜 We will see how it fares in practice! If anyone has examples of those kinds of things in real world (w/ users, permissions, auth etc) open source code I would love to see them! Cheers!

Joe Lane 2021-04-20T15:16:23.352200Z

How complex is your permission model?

prnc 2021-04-20T15:34:35.352400Z

ATM I’m only concerned with the ~trivial case of: “Alice owns some resources and only Alice can see them” where resources are entity level, so not very complicated and not very granular, but that will probably evolve. It’s a knowledge base type of application so where this will potentially get more interesting is in the cross-sections of public and private information--private knowledge graphs embedded in public ones. So I’m just trying to model this in a nice way with datomic without prior experience with datomic ;) Presently I was mainly concerned with the simple mechanics i.e. where those restrictions should sit (in terms of best practices), so it’s not error prone. So today I’ve just added a rule and a centralised query fn for restricted resources that just adds those access constraints captured by the rule to the :where clause... so far so good I think ;)

tatut 2021-04-19T12:17:20.328400Z

afaict, there is no similar functionality in cloud


In our system we only check the authorization (i.e. access) at the borders of the system. When you're passed that interceptor/layer we don't bother. That way the queries are simpler and faster (at the cost of always running one (or more) queries beforehand.

Joe Lane 2021-04-19T13:14:03.331200Z

Queries are data and compose, programmatically building queries is a normal thing to do :)

jaret 2021-04-19T13:39:34.331400Z

Hi, For on-prem we generally recommend that you run a single logical database per transactor (pair). Some customers use additional DBs for operational tasks, but generally a datomic on-prem system (and license) includes a transactor pair, single DB, associated peer applications. Perhaps we can discuss your needs for separate DBs. Have you evaluated https://docs.datomic.com/cloud/whatis/architecture.html? Cloud is more suitable for per db mutli tenancy. How many DBs do you envision long term? What are the sizes of each DB that you expect? If you'd like to share more details you can shoot a ticket to <mailto:support@cognitect.com|support@cognitect.com> and/or we can arrange a call to discuss.

shields 2021-04-19T13:57:49.332600Z

Good example at the 12 minute mark of how Nubank controls ownership and authorization w/ their schema using rules. https://youtu.be/7lm3K8zVOdY

prnc 2021-04-19T14:03:51.332900Z

I’ve started drafting an impl of what I need with rules—will definitely check out the nubank talk as is seems to go that direction. Thanks everyone!

cjsauer 2021-04-19T18:03:27.335400Z

Has anyone experimented with defining “virtual attributes” in the db schema? Meaning, attributes that are not meant to be involved in the transaction of data, but that only serve as documentation, or to make the system even more self-describing. For example, I might include :product/href in my schema as a card-one string type attribute, but only ever derive its value on-demand. I’m finding that describing the system fully in the schema gives you really great “meta-query-ability”, but I’m wondering if there is any downside to doing this.

cjsauer 2021-04-19T18:05:41.336700Z

I’ve seen quite a few other projects that do this out-of-band in code. But it struck me that this type of metadata really belongs in the schema proper. It could even allow datomic to help you with making backwards compatible changes by warning you if you break schema (?)

Joe Lane 2021-04-19T18:07:11.337600Z

What your describing https://docs.datomic.com/on-prem/best-practices.html#annotate-schema @cjsauer!

Joe Lane 2021-04-19T18:08:11.338100Z

There are limits to how many schema elements you can have, but that https://docs.datomic.com/on-prem/schema/schema-limits.html

Joe Lane 2021-04-19T18:09:37.339100Z

I've seen migration tools built using this capability, diagramming tools, etc.

cjsauer 2021-04-19T18:11:11.340300Z

Aha, cool! For a moment it felt odd to be capturing schema about data that I don’t intend to ever store on disk, but after pondering a bit, why should the durable storage have a monopoly on the schema definition??

Joe Lane 2021-04-19T18:12:02.341600Z

... I mean, it is durable. These attributes are the same data as would be in transactions and are stored in durable storage.

Joe Lane 2021-04-19T18:12:12.341800Z

Datomic is built out of itself

cjsauer 2021-04-19T18:13:02.342700Z

True true. I mean to say that I never intend to actually transact a value of that attribute onto domain data. Of course the schema itself is stored durably.

cjsauer 2021-04-19T18:16:04.345500Z

Another way to put this: essential state shouldn’t have a monopoly on the schema. So it’s not really about its durability. More its use in the system.

Joe Lane 2021-04-19T18:16:58.346300Z

Makes a nice compile target too 😉

cjsauer 2021-04-19T18:17:46.346800Z

Yea definitely

ghadi 2021-04-19T21:19:00.347600Z

txacting data about attributes such a useful thing to do ^^^^^^