Clojurians Log v2

Is there any comparison between Kafka Streams and Onyx? I wanted to know the flexibility of Kafka Streams topology built with the Kafka Processor API in comparison with Onyx (which I know is powerful using the workflow abstraction)

Maybe something with the Onyx concepts (catalog, workflow etc) could be made on top of the Kafka Processor API... I don't know, just wandering

There’s previously been some comparison with streams, but it was mostly operational

Comparison on a functional level or performance? I know the streams api might create a lot of in between topics, which could become hard to handle.

I’m sure you could do the same on the kafka API, but it’d be a lot of work.

The main topological difference is that Kafka performs the work for a single partition locally on one machine. Onyx fans out the work in between tasks across a cluster.

No perf comparisons that I know of, but usually it’s dominated by what is being performed. Onyx gives you more knobs and flexibility, but may also require a bit more knowledge of how your job should be split up.

We’ll have a little workshop at our company in some weeks, I will use onyx, but I will suggest, and most likely it will be picked up, is doing the same with spring boot/data.

Thanks, guys o/

That should be an interesting comparison.

@gklijs Maybe you could video stream the workshop =P uhaeuheuehehu jk, here where I am is a needy place of these kind of workshops/talks of not mainstream stuff

They are quit chaotic and I’m also afraid I don’t have enough time to prepare, so don’t really think it would be worth it.

@pfeodrippe really depends on the use case.

A bit of a long shot but my Introduction to Onyx talk is waitlisted for Strata London, if someone drops out then there’s slim chance I’ll be able to do it.

“Learning how to design automatically updating AI with Apache Kafka and DeepLearning4J” is confirmed but I will be name dropping Onyx in as well as an alternative.