Clojurians Log v2
datomic
that looks quite tricky, can’t really say how make the query faster, but if you could update some attr on the root every time you update some descendant (like :root/recursive-modification-time timestamp), you could simply query all roots based on that
in effect moving the “has some descendant changed” determination from query time to tx time
Hey all. I asked a little while back about doing multi-tenant apps with datomic and a couple of ideas came up. One of those was using a single DB per customer. So the connection string might look like : "datomic:<dev://localhost:4334/customer_id>"
This looks interesting - do this mean that all databases backed by the same Postgres instance (for example) share the same DB?
In which case, the only different between:
"datomic:<dev://localhost:4334/customer_id_1>"
and
"datomic:<dev://localhost:4334/customer_id_2>"
Is some sort of partioning key that Datomic uses to keep data separate within it's backing store?
If that is the case, then is it feasable to "open" a new DB connection per http request? or keep a pool of
From the docs
conns in some sort of LRU cache?
Datomic connections do not adhere to an acquire/use/release
pattern. They are thread-safe and long lived. Connections are
cached such that calling datomic.api/connect multiple times with
the same database value will return the same connection object.
Hi 👋
I am looking into application level access restriction/permissions to entities.
I couldn’t find any specific guidelines for cloud, that would allow for a separation of concerns between query logic and permissions.
In on-prem filter seems to be the useful for that. What are some good ways to go about this for datomic cloud?
[It depends ofc, it’s early stage so I don’t have specific requirements, just looking for some general guidelines/mechanisms in datomic that could be leveraged].
Thanks!
Thanks @tatut!
Being new to datomic this is quite an interesting problem for me.
The filter solution (on-prem only!) seems to be very clean, and it’s what nubank are using. Query logic is separated from access restrictions.
Leveraging “database as a value” and thus playing to datomic strengths.
The other nice thing about datomic is ofc “queries are data” as mentioned by @lanejo01
I’ve just used this approach to write a boundary query-as-a-user fn, which rewrites the queries to refer to access restriction rule.
Somehow it doesn’t really feel right, less separate, less explicit. Maybe it’s just my shoddy impl 😜 We will see how it fares in practice! If anyone has examples of those kinds of things in real world (w/ users, permissions, auth etc) open source code I would love to see them! Cheers!
How complex is your permission model?
ATM I’m only concerned with the ~trivial case of: “Alice owns some resources and only Alice can see them” where resources are entity level, so not very complicated and not very granular, but that will probably evolve. It’s a knowledge base type of application so where this will potentially get more interesting is in the cross-sections of public and private information--private knowledge graphs embedded in public ones. So I’m just trying to model this in a nice way with datomic without prior experience with datomic ;)
Presently I was mainly concerned with the simple mechanics i.e. where those restrictions should sit (in terms of best practices), so it’s not error prone.
So today I’ve just added a rule and a centralised query fn for restricted resources that just adds those access constraints captured by the rule to the :where clause... so far so good I think ;)
afaict, there is no similar functionality in cloud
In our system we only check the authorization (i.e. access) at the borders of the system. When you're passed that interceptor/layer we don't bother. That way the queries are simpler and faster (at the cost of always running one (or more) queries beforehand.
Queries are data and compose, programmatically building queries is a normal thing to do :)
Hi, For on-prem we generally recommend that you run a single logical database per transactor (pair). Some customers use additional DBs for operational tasks, but generally a datomic on-prem system (and license) includes a transactor pair, single DB, associated peer applications. Perhaps we can discuss your needs for separate DBs. Have you evaluated https://docs.datomic.com/cloud/whatis/architecture.html? Cloud is more suitable for per db mutli tenancy. How many DBs do you envision long term? What are the sizes of each DB that you expect? If you'd like to share more details you can shoot a ticket to <mailto:support@cognitect.com|support@cognitect.com> and/or we can arrange a call to discuss.
Good example at the 12 minute mark of how Nubank controls ownership and authorization w/ their schema using rules. https://youtu.be/7lm3K8zVOdY
I’ve started drafting an impl of what I need with rules—will definitely check out the nubank talk as is seems to go that direction. Thanks everyone!
Has anyone experimented with defining “virtual attributes” in the db schema? Meaning, attributes that are not meant to be involved in the transaction of data, but that only serve as documentation, or to make the system even more self-describing. For example, I might include :product/href in my schema as a card-one string type attribute, but only ever derive its value on-demand. I’m finding that describing the system fully in the schema gives you really great “meta-query-ability”, but I’m wondering if there is any downside to doing this.
I’ve seen quite a few other projects that do this out-of-band in code. But it struck me that this type of metadata really belongs in the schema proper. It could even allow datomic to help you with making backwards compatible changes by warning you if you break schema (?)
What your describing https://docs.datomic.com/on-prem/best-practices.html#annotate-schema @cjsauer!
There are limits to how many schema elements you can have, but that https://docs.datomic.com/on-prem/schema/schema-limits.html
I've seen migration tools built using this capability, diagramming tools, etc.
Aha, cool! For a moment it felt odd to be capturing schema about data that I don’t intend to ever store on disk, but after pondering a bit, why should the durable storage have a monopoly on the schema definition??
... I mean, it is durable. These attributes are the same data as would be in transactions and are stored in durable storage.
Datomic is built out of itself
True true. I mean to say that I never intend to actually transact a value of that attribute onto domain data. Of course the schema itself is stored durably.
Another way to put this: essential state shouldn’t have a monopoly on the schema. So it’s not really about its durability. More its use in the system.
Makes a nice compile target too 😉
Yea definitely
txacting data about attributes such a useful thing to do ^^^^^^
1💯