Clojurians Log v2

hey! Would you be willing to accept changes to onyx-kafka which (optionally) commits offset to Kafka? Without that our process breaks, so I'm hoping you would accept that so that we don't need to maintain fork 🙂

Hi! Is there a hot code reload story with Onyx on a local setup? Expecting to reload jobs/fns, not the catalog... Pointers much appreciated, thanks!

@asolovyov You can set Kafka consumer options to enable that like so: :kafka/consumer-opts {"enable.auto.commit" true}

@niamu hm 🙂

@asolovyov that should work. What do you need the commits for?

@fdserr It's just a jar -- standard nrepl reloading story for Clojure applies 🙂

@michaeldrogalis Wunderbar, thanks!

No problem @fdserr!

4 years everyone! Fun times.

https://twitter.com/MichaelDrogalis/status/942819243732512768

@lucasbradstreet at least for our operations guys for monitoring 🙂

Right, was wondering if it was for that. That’s a good use case for it.

You can also look at lag_Value from onyx’s metrics. It’ll tell you how far a peer is behind the head of the partitions, though it does aggregate multiple partitions into one value.

riight, that one is also interesting

also when we make some changes to our jobs we usually make a new release of the project, which means a new tenancy id

which means checkpoints are left in old tenancy, so I think easiest way for us is to use kafka offsets for some time

not sure that's the best practice right now though...

Ah, I see. Resume points is usually the way of picking those back up.

Resume points will allow you to pick them up from a diff tenancy.