Clojurians Log v2

Anyone want to discuss this? https://youtu.be/B1-gS0oEtYc I’m really seeing a strong connection of ideas between that and http://www.stuttaford.me/2016/01/15/how-cognician-uses-onyx/ I’m too tired atm to put it all together, but the general trend i’m seeing is that its often sensible to build an append only data log (datomic, kafka) and have the rest of your system just consume and produce to it. This seems to be at the heat of CQRS and FRP, both of which i’m just starting to read about. The really interesting part is that the stream processor (kafka streams, onyx, flink) might just produce the views for other components of the system, there by removing the need for more ridge forms of storage. Which is talked about a bit here.... https://youtu.be/uuv-lnOrD0o And is a key principle of apache samza.

Yeah I really like that talk 🙂 That and the "Turning the database inside out" talks were the ones that really helped me understand the ideas of CQRS

Or maybe I should say Event Sourcing

Don't really see how FRP is related though

FRP has 2 meanings. The common meaning, but also Functional Relational Programming, which has a discussion of immutable events as the basis.

They're also similar enough that 2 can talk about each without realising they're not both discussing the same topic

Huh, first time I've heard of functional relational programming

http://shaffner.us/cs/papers/tarpit.pdf

It's a very good paper, relates well to Clojure

I believe it has been cited as influential to the design of clojure

As a side note, event sourcing can be achieved without any CQRS, it is just identify and collect events. I see CQRS just as a fancy word that says: "please, writes and reads should not intermingle"

Very relevant to this are recent talks by Bobby Calderwood: https://www.youtube.com/shared?ci=rg0OP10t2NQ

Right, I meant to say event sourcing instead of cqrs

I use CQRS with non-event sourcing. Needs using any time you do this: (defn get-user [id] (let [x (get-user-from-db id)] (assoc x :age (calc-on (:dob x)))) else you get endpoints with more data than others (many wtf moments there!)

@dominicm Could you elaborate on why that code fragment leads to a need for CQRS?

@seancorfield People have a tendency to return the result of (jdbc/insert!) (there's a form that returns what you inserted I think?). Which would have less data.

Ah, I’m with you now. Yes, we have an API where many update operations return the updated data — but they all do it via calling the exact same function that the query itself calls. That’s done to avoid a round-trip on the API.

We could separate the Command (update) from the Query (get) but in nearly 100% of use cases all clients would need to issue C.Q. (two API calls).

@seancorfield So you do (map #(assoc % :age …) (jdbc/insert! …))?

We have a render-user function that accepts either a user map (as returned by jdbc/insert!) or a user ID (in which case it starts by fetching the user map) and that is used consistently wherever we specify that we return a (public) user map.

Age isn’t the only thing we calculate and add to the map.

Then the public “get user” API calls render-user too.

For us, it’s just an optimization to avoid clients having to always follow a user update API call with a user get API call.

(mostly this is around jdbc/update! calls since only registration will jdbc/insert! on a user — but it applies to several other entities in the system too)

There are also some APIs where the update (or insert) operation cannot yield calculated data — so the client knows everything about the entity already — and those are pure command APIs (they return just success/failure).