Clojurians Log v2

Hello everyone, I'm new to clojure and saw the announcement, nice project! couple of things wrt proletarian: • the link to brian goetz article in the readme is 💀 , web archive has it though: https://web.archive.org/web/20201111190527/https://www.ibm.com/developerworks/library/j-jtp05236/index.html 🙂 • Coming from other languages/frameworks, the worker queue model for those short but async jobs makes total sense, but in the application I'm working on, I see sometimes future used to that effect. I get futures are not serialized and the restart logic would have to be implemented in the future itself, but still... is it true that given the concurrency primitives clojure provides, some usecases are solved by the language features? I still prefer the persistent queue model, but I'm trying to wrap my head around the new environment. (sorry if it's too generic of a question but felt like a good opportunity to pick clojurists brains) 😛

Hi @raimonster! Thanks for the heads up about that dead link - I’ve updated the readme with the web archive link you provided. I can also recommend chapter 7 from Brian Goetz’ Java Concurrency in Practice (2006) on this topic. You’re right, future can be, and often are, used for async work. Clojure makes this very easy to do in a safe manner. I’ve only recently converted must futures in my work codebase to Proletarian jobs. The reason is, like you’re alluding to, that the jobs are serialized to durable storage and can be retried (restarted).

Indeed, the inspiration for Proletarian comes from projects such as Delayed Job and Sidekiq in the Ruby world.

yep, I did a bunch of sidekiq/resque in the past, that's why this model makes so much sense to me to the point I feel I'm biased 😛

There’s a nice property that Proletarian has over those projects, though, and it’s that the job can be written to the queue in the same transaction as whatever caused the job to be queued in the first place. For example, an order receipt email can be queued in the same transaction in which the order is being written.

Your business requirements must dictate if this extra machinery is necessary, though. Maybe a future here and there is good enough. It really depends on the use case and what guarantees you want to have.

Proletarian tries really hard to make sure your job is processed at least one. If the computer dies, or an external call fails (and is not catched), or any other failure happens while the job is being processed, it will get processed again, possibly by another machine, or by another thread on the same machine.