Clojurians Log v2

@lucasbradstreet How could i go about verifying the behavior of the window-state? I have a fairly small program and i’m not sure how to debug it. I’m expecting the :trigger/sync function to fire and it doesn’t. Hopefully i’m not missing something small here. Code for reference: https://github.com/drewverlee/ad/blob/master/src/ad/core.clj

I would try copying the watermark trigger code into your own code base and add some printlns. I’m out and on my phone at the moment, but I see two risky things they. One is that I’m not sure the watermark scheme supports local rt because that functionality required a lot of tracking watermarks across channels. The second problem is that I can’t remember if watermarks are a window state trigger now. It probably is, but I would want to check.

@lucasbradstreet thanks for the input. I’ll see what i can do.

I have a pretty strong feeling we’re not tracking the min watermark across DAG edges in local rt

I noticed that the developer's guide is outdated in regards to the branching policy (https://github.com/onyx-platform/onyx/blob/0.12.x/doc/developers-guide/branch-policy.md). The preferred way of submitting a PR nowadays is to target master, right?

I created a couple of tiny PRs with stuff that I found along the way while upgrading the plugin template

@quimrstorres ah yes, we dropped develop and purely work from master now. It wasn’t worth the overhead.

Updated it. Thanks!

hi, it seems that no one is using onyx-http? I can't get it working and I don't find examples in github

It should work. If you git clone the repo and run lein test do they pass?

If so, check the onyx.log to see if there’s an error

the tests pass but in onyx-starter the messages just seem to go through untouched

only getting Aeron status channel error in the log

You’ve modified onyx-starter to use onyx-http?

yes

etc

Does that job even submit OK? We don’t currently support output tasks that have other tasks downstream

yikes, that must be it

https://github.com/onyx-platform/onyx-starter/blob/master/src/onyx_starter/launcher/submit_sample_job.clj#L25 try capturing the submit-job return here

the success? key will probably be false

Also, note we have a template you can use to start a new project https://github.com/onyx-platform/onyx-template

thanks, I'll use the template when I get the basics working

result {:success? true, :job-id #uuid "085f412c-30d9-7cc1-939e-095f9213961b"}

who knows...anyway I prefer using clj-http directly, also it seems that onyx-http is not designed for my usecase

I was having much success with clj-http

Ok sure. If you’re going to use clj-http you may want to try onyx/batch-fn to fire off a a batch of async requests at a time.

Otherwise you’ll find throughput challenging

thanks for all the help, really helped me as an Onyx beginner 🙂

No worries!