Clojurians Log v2

I am going to store blockchain block informations into datomic. From time to time, I would query the total transaction count in the most recent H blocks. So this is querying against a window of H blocks. How can I perform that with datomic *efficiently ?

is there a reason not to just store a transaction count value with the block entity?

Or are you asking how to do the moving window aggregation (rolling sum of last H blocks)

Yes. Sliding window as normally done in Flink. Wonder how to best achieve that w/ datomic.

I'd like to monitor the time spent in d/q (specifically, to report it to New Relic). I could just put a wrapper function around d/q and hope developers remember to call that instead of calling d/q directly, but is there a better way? Ideally, I'd do something when setting up the connection to ensure all queries get monitored. Is there a protocol I could extend or something?

On client at least, the datomic api is just a set of functions invoking protocol functions on your connection/db value. So what you can do is kinda proxy them: create a deftype for say a db value that forwards all calls to an ‘underlying’ datomic db value. In these wrappers you can then add logic like this

The benefit is that you only have to control where the db or connection value comes from, the functions that get it are oblivious to that these values are ‘decorated’ like this

Yeah, that's the initial route I was looking down, but I couldn't figure out how d/q used the db value. Specifically, I didn't see a q method or similar on there which suggested to me that member functions would be called many times on a single query, so I wasn't able to figure out how I'd know when a query starts and ends. Have you tried doing this?

I guess I could wrap a db value and record which methods are called to maybe guess how it's used.

@bmaddy take a look at https://github.com/ComputeSoftware/datomic-client-memdb/blob/master/src/compute/datomic_client_memdb/core.clj

Specifically LocalDb

@bmaddy I did a small lib to ensure that every transact get audited (some extra tx-data on every transaction in a conn). You can do somethign like that, but to ensure that every (d/db) in your conn will return the "db-with-spy-methods" https://github.com/molequedeideias/conncat

Be careful with that. Depending on your system, it can create lots of spurious transactions if you’re not checking for empty first.

@souenzzo, yeah, exactly what I was thinking in terms of wrapping the connection. Nice to see that it works. Thanks! @kenny Nice, it looks like client-impl/Queryable has a q function right there on it that you can implement. Does anyone konw if there's a similar interface for a peer (I should have mentioned in my question that we're using a peer)?