Clojurians Log v2

Let me first provide some background on my position before replying to async for konserve. I agree that callbacks are more portable and can be made composeable (continuations are a universal framework of computation), but they still require a lot of things to be defined to have similar semantics between libraries and runtimes, i.e. error handling, the way the event loop schedules them (on the JVM a threadpool integration) and how synchronisation happens atomically between two different processes, which is still implicitly in the responsibility of the library maintainer porting code. I think the relationship of the Clojure community to core.async is a bit contradictory in this respect, because these things are language level abstractions that need to be standardized or you always need to manually glue the interfaces together and make sure that the different forms of parallelisms between libraries do not deadlock or screw each other over. In that sense I think Clojure should have got first class async semantics in its compiler instead of bailing out to CPS as the general interface. Even the language of "lean" callbacks, JS, has now done that with async and await. The biggest annoyance with callbacks for me is still the inversion of control flow that is forced on me as a programmer (few programmers like to program directly in CPS style) and most importantly the forced sequential nature of a monadic promise based approach. With core.async I can write down nested compositions of functions doing aggregated immutable reads concisely, which I did a lot for replikativ and kept the codebase very concise and easy to verify that way, e.g. for fetching tree fragments over the wire or mutating durable storage in a critical section, while the callbacks would have cluttered a lot of my code. In that sense I am not happy with having to use callbacks as a library developer. I know this suggestion has been made by the Clojure core team and I think it is debatable that this is a good approach because applications tend to factor out and become libraries. I think it is just evading the problem from their side to be honest and I initially thought 7 years ago when it came out that core.async would have been deeper integrated by now.

Having said all this konserve was initially implemented deliberately with callbacks. By now the control flow in the filestore became a bit more complicated with using the asynchronous NIO API, so we decided to use core.async internally to make the error handling easier and have better understandable sequential code. But if you would like to not have the core.async dependency in general we can definitely remove it from the interface and either factor out the filestore with the dependency on it or maybe @noprompt can help me to port it to promises, if this makes sense.

@noprompt, @alekcz360 Besides removing the core.async dependency we might need to add additional protocols to facilitate low-level block based access to konserve.

Removing core.async shouldn't be an issue. The code is structures well enough for it to be done painlessly. I think it'd be worthwhile. Though providing an optional namespace to provide that kind of interface for current users.

@whilo @noprompt I'd need some guidance with the block based storage bit. I'm not yet familiar with how asami's storage works.

@quoll and everybody else interested, what about aiming for a meeting next Thursday, 26th of November, 11 am EST?

Yes, we should discuss this together, I think.