Clojurians Log v2

n00b question: is it ill-advised to use onyx synchronous request/response-type workflows?

@devn You can get away with it to a limited capacity if the round-trip time isn't huge with things like batch/bulk functions, but in general you're better off designing for asynchrony.

@michaeldrogalis

err @michaeldrogalis define limited capacity 🙂

I wouldn't block progress for more than a few seconds. I misquoted - by the way. Batch functions are for async. tasks that need to be done fully synchronously.

Other n00b questions include: - If I have a set of parallel steps, can I do something like switch to a fail branch if they don't all succeed?

You can still do it that way, but I wanted to correct it for the record

Flow conditions are the closest thing to help you there, but that sounds like more in the territory of a workflow engine.

that's exactly what it is 🙂

really trying not to reinvent the wheel and looking at all of the options we have

You can still do it in Onyx, but it would look a little differently. Onyx is a streaming engine.

Can I get a bit wider description of the problem to make a suggestion? 🙂

Yeah, sorry, should have started there. 🙂

We have a legacy system written in Ruby that uses resque to manage job states. Most of our customer integrations are asynch, but one of them we're building out right now is synchronous. Some jobs in the workflow are retryable. Some jobs can happen in parallel (for instance, two separate jobs hit two different APIs to pull down relevant data and a future step merges the results). It would be nice if something in the clojure ecosystem existed where I could just specify some clojure functions, tag them as retriable, specify which functions can run in parallel, define failure branches, etc. Think: a subset of Amazon's states language, applied to Clojure functions. We've been looking at AWS step functions, but there are a number of things that don't seem particularly great about them, like the local testing and development story for starters.

Basically, we need state machine with some bells and whistles. We want to keep our options open on deploying the tasks in one app, or as separate reusable tasks that are shared by multiple workflows.

Is that clear as mud? 🙂

Onyx is conceptually close to your description of "some clojure functions, tag them as retriable, specify which functions can run in parallel, define failure branches, etc", except everything is retryable to completion and all tasks in a job run in parallel.

I think if you design it CQRS style, Onyx is a really good fit for what you're trying to do.