Could anyone speak to the latency of Kafka Streams? I'd like to know the amount of milliseconds between publishing a message to a topic until it's received by a consuming Streams app (assuming negligible network latency.) Not finding much on Google
Depends on a lot of things. But using default settings, and having a low througput you have a latency between calling the send method in the producer and it actually be send to the broker going up to 100ms. This is because http://linger.ms is by default 100 in Kafka Streams to help throughput. Apart from that, and assuming connections already made, it should be on the order of a couple of milliseconds.
If latency is more important then throughput you can set it lower. It will also cause more stress for the brokers, because less messages are combined. From what I saw doing benchmarks I wouldn't set it lower then 10.
Some of them can be found here, when 0 the is a lot more load on the brokers, https://graphql.gklijs.tech/results/linger-ms-config/average-kb-cpu
Hmm, sub 40ms is probably find and achievable for my use case. Thanks for the graph and insight! 👍