onyx

FYI: alternative Onyx :onyx: chat is at <https://gitter.im/onyx-platform/onyx> ; log can be found at <https://clojurians-log.clojureverse.org/onyx/index.html>
sparkofreason 2018-04-16T15:55:34.000186Z

Doesn't look like it, no :reduce in the source that I can find, and definitely letting all segments through.

lucasbradstreet 2018-04-16T17:07:04.000220Z

Yeah, you’re right. It’ll need a bit of a refactoring to separate out the transformed segments from the triggered segments so that when we pass the segments downstream we can correctly identify which ones we want to pass down based on the onyx/type.

lucasbradstreet 2018-04-16T17:08:17.000492Z

Actually it might be enough to just decide in here https://github.com/onyx-platform/onyx-local-rt/blob/0.12.x/src/onyx_local_rt/impl.cljc#L276 since onyx.core/triggered has all of the triggered segments. We need to bring it closer in line with what is in onyx core though.

2018-04-16T23:31:27.000023Z

Hey everyone - I’m writing a pretty straightforward output plugin, and I have a few questions about making network calls asynchronously The output has to make calls that are potentially very long-running (~30 seconds if it times out/errs), but I don’t want that to hold up other calls being made (esp. if the batch size is more than a few), since the results of the calls aren’t related to each other. I originally wrote it using pmap to execute them all in parallel, but I’m wondering if there’s a better way to handle that that will work better with the framework (and since I don’t see pmap being using in the other plugins I’ve checked out). I set up a test environment using the flat-workflow example in examples repo, and obviously changing pmap to map causes the calls to be executed one at a time, which I’d like to avoid

lucasbradstreet 2018-04-16T23:32:13.000096Z

Hi @gseyffert, I have some comments for you, but I’ll have to get back to you on it.

2018-04-16T23:32:22.000274Z

Take your time!