Clojurians Log v2

not exactly onyx question, but "can I do paginated consumption from kafka topic with gregor?"

say, 10 or less messages at a time

you can set ConsumerConfig/MAX_POLL_RECORDS_CONFIG to allow a maximum per poll, but you will get back less, if there are less ready

sweet! thanks @gklijs

for gregor config will look like {"max.poll.records" "10"}

It’s nice to have, and something which was missing before. I don’t know if a heartbeat is now send when not polling, bu we had problems before when consumers where taking to long, and thought dead, because they took to long after a pol

Could be, I only used java interop so far when working with clojure and kafka

yeah, writing UI for onyx ETL job reports, and if you are lucky to consume closed topic for huge input file – chrome hangs for minutes :opieop:

"max.poll.records" is kafka's setting name. gregor seems to just send it as string w/o any changes. will try it in a few minutes

That’s one of the reason I like the java interop, by using static strings there is less risk of making types + when they might rename/delete it in a newer version your covered.

#clojure-spec :)