Clojurians Log v2

ah, if so then I think the behavior i saw was expected. Why is after-batch still called after the task is done processing all of the batches?

Hi Onyxeers, happy 2018 to all! Is there a plug-in, lib, gist, or article to help me get started with idiomatic in-outing of onyx messages to/from transit over websocket? TIA!

@quan.ngoc.nguyen After-batch is after a single batch, not all the batches.

@fdserr Hi! I don't think there's a recent plugin for web socket IO. There is one for HTTP though.

The Kafka plugin producer silently fails when a message does not adhere to the expected format. Subsequent values are then simply ignored. It seems an exception or warning should be thrown if a segment does not adhere to the expected format.

@kenny As it turns out, Im investigating that bug right now 🙂

We're going to throw here and crash the job, deferring to lifecycle restarts. Tracking down why it's been silenced though.

Another possibly related issue that I just ran into is if an exception is thrown when attempted to serialize the Kafka message, that exception is swallowed. For example when using Nippy to serialize, I attempted to pass in a Datomic Datom as part of the Kafka :message. That silently fails when an exception should be thrown. Normally if you attempt to serialize something Nippy doesn't know how to serialize an ExceptionInfo is thrown.

Sounds like it's right around the same area.

I'll keep you posted when I figured it out, though do let me know if you find the root cause first

@kenny The venerable @gardnervickers will be picking up the bug from here, I need to task switch. He's got a reproducer and isolated it to onyx-kafka.