Clojurians Log v2

if i want input plugins to be able to pick up on configuration changes without restarting the job, my best bet is to write additional logic on top of it, correct? just to name an example, have a kafka input dynamically change which partitions it subscribes to.

i was thinking of writing a small layer on top of the input plugins that distribute workload over them, fairly similar to onyx’ log. reuse existing virtual peer id’s + a uuid for each workload item. but i need to check whether there are simpler ways of doing this.

@lmergen I can’t quickly comment on whether or not that’s a good approach. But once a task starts in Onyx, it’s configuration is considered immutable.

For example, in onyx-kafka, the consumer is started at the outset of the task. There’s most definitely a way that we can make it subscribe to other partitions over time, but we didn’t intend for that kind of a use case at design time.