Clojurians Log v2
datomic
Hey all. I asked a little while back about doing multi-tenant apps with datomic and a couple of ideas came up. One of those was using a single DB per customer. So the connection string might look like : "datomic:<dev://localhost:4334/customer_id>"
This looks interesting - do this mean that all databases backed by the same Postgres instance (for example) share the same DB?
In which case, the only different between:
"datomic:<dev://localhost:4334/customer_id_1>"
and
"datomic:<dev://localhost:4334/customer_id_2>"
Is some sort of partioning key that Datomic uses to keep data separate within it's backing store?
If that is the case, then is it feasable to "open" a new DB connection per http request? or keep a pool of
From the docs
conns in some sort of LRU cache?
Datomic connections do not adhere to an acquire/use/release
pattern. They are thread-safe and long lived. Connections are
cached such that calling datomic.api/connect multiple times with
the same database value will return the same connection object.
Hi š
I am looking into application level access restriction/permissions to entities.
I couldnāt find any specific guidelines for cloud, that would allow for a separation of concerns between query logic and permissions.
In on-prem filter seems to be the useful for that. What are some good ways to go about this for datomic cloud?
[It depends ofc, itās early stage so I donāt have specific requirements, just looking for some general guidelines/mechanisms in datomic that could be leveraged].
Thanks!
Thanks @tatut!
Being new to datomic this is quite an interesting problem for me.
The filter solution (on-prem only!) seems to be very clean, and itās what nubank are using. Query logic is separated from access restrictions.
Leveraging ādatabase as a valueā and thus playing to datomic strengths.
The other nice thing about datomic is ofc āqueries are dataā as mentioned by @lanejo01
Iāve just used this approach to write a boundary query-as-a-user fn, which rewrites the queries to refer to access restriction rule.
Somehow it doesnāt really feel right, less separate, less explicit. Maybe itās just my shoddy impl š We will see how it fares in practice! If anyone has examples of those kinds of things in real world (w/ users, permissions, auth etc) open source code I would love to see them! Cheers!
How complex is your permission model?
ATM Iām only concerned with the ~trivial case of: āAlice owns some resources and only Alice can see themā where resources are entity level, so not very complicated and not very granular, but that will probably evolve. Itās a knowledge base type of application so where this will potentially get more interesting is in the cross-sections of public and private information--private knowledge graphs embedded in public ones. So Iām just trying to model this in a nice way with datomic without prior experience with datomic ;)
Presently I was mainly concerned with the simple mechanics i.e. where those restrictions should sit (in terms of best practices), so itās not error prone.
So today Iāve just added a rule and a centralised query fn for restricted resources that just adds those access constraints captured by the rule to the :where clause... so far so good I think ;)
afaict, there is no similar functionality in cloud
In our system we only check the authorization (i.e. access) at the borders of the system. When you're passed that interceptor/layer we don't bother. That way the queries are simpler and faster (at the cost of always running one (or more) queries beforehand.
Queries are data and compose, programmatically building queries is a normal thing to do :)
Hi, For on-prem we generally recommend that you run a single logical database per transactor (pair). Some customers use additional DBs for operational tasks, but generally a datomic on-prem system (and license) includes a transactor pair, single DB, associated peer applications. Perhaps we can discuss your needs for separate DBs. Have you evaluated https://docs.datomic.com/cloud/whatis/architecture.html? Cloud is more suitable for per db mutli tenancy. How many DBs do you envision long term? What are the sizes of each DB that you expect? If you'd like to share more details you can shoot a ticket to <mailto:support@cognitect.com|support@cognitect.com> and/or we can arrange a call to discuss.
Good example at the 12 minute mark of how Nubank controls ownership and authorization w/ their schema using rules. https://youtu.be/7lm3K8zVOdY
Iāve started drafting an impl of what I need with rulesāwill definitely check out the nubank talk as is seems to go that direction. Thanks everyone!
Has anyone experimented with defining āvirtual attributesā in the db schema? Meaning, attributes that are not meant to be involved in the transaction of data, but that only serve as documentation, or to make the system even more self-describing. For example, I might include :product/href in my schema as a card-one string type attribute, but only ever derive its value on-demand. Iām finding that describing the system fully in the schema gives you really great āmeta-query-abilityā, but Iām wondering if there is any downside to doing this.
Iāve seen quite a few other projects that do this out-of-band in code. But it struck me that this type of metadata really belongs in the schema proper. It could even allow datomic to help you with making backwards compatible changes by warning you if you break schema (?)
What your describing https://docs.datomic.com/on-prem/best-practices.html#annotate-schema @cjsauer!
There are limits to how many schema elements you can have, but that https://docs.datomic.com/on-prem/schema/schema-limits.html
I've seen migration tools built using this capability, diagramming tools, etc.
Aha, cool! For a moment it felt odd to be capturing schema about data that I donāt intend to ever store on disk, but after pondering a bit, why should the durable storage have a monopoly on the schema definition??
... I mean, it is durable. These attributes are the same data as would be in transactions and are stored in durable storage.
Datomic is built out of itself
True true. I mean to say that I never intend to actually transact a value of that attribute onto domain data. Of course the schema itself is stored durably.
Another way to put this: essential state shouldnāt have a monopoly on the schema. So itās not really about its durability. More its use in the system.
Makes a nice compile target too š
Yea definitely
txacting data about attributes such a useful thing to do ^^^^^^