proletarian

A durable job queuing and worker system for Clojure backed by PostgreSQL: https://github.com/msolli/proletarian
dominicm 2021-04-25T15:17:01.038400Z

In addition to the job handler being passed in, I think the retry-strategy could also be a function. I had an idea to derive the retry information from the exception, e.g. look at common retry headers from HTTP in the ex-data, or look for cognitect anomalies and how they might impact whether to retry again or not. Something that did fall out of reducing the retry-strategy & job-handler to passed-in functions is that it made the job-type redundant as far as I can tell. This means the notion of a job type can move into the payload in "user space", rather than being baked into the library. From a little hacking these changes seem fairly trivial as it's just making arguments out the handle-job! and retry-strategy functions and passing them to the appropriate places. I think you could retain the existing system by having the enqueue mechanism automatically add a ::job-type key to the payload and use that as the dispatch into the existing multi-methods. I suggest that there's naming updates to move that into some kind of flavour.