Clojurians Log v2

Hi everyone,

is it possible to do something like:

(a/go 
(doseq [m messages*]
      (a/go
        (client/send! c/kafka-producer "topic" id h))

what i mean, is it possible to nest a/go ?, is that a bad habit or something wrong in the example above?

it's possible to nest go blocks, although it may have unexpected results if you don't have the right mental model for what a nested go block implies. the above example might be ok, but it depends on whether messages* blocks when you iterate over it. If iterating over messages* does block, then it probably does not belong in a go block.

otherwise, the above example is fine. however, it probably doesn't have much benefit in practice over:

(doseq [m messages*]
      (a/go
        (client/send! c/kafka-producer "topic" id h)))

why not just do the following?

(doseq [m messages*]
 (client/send! c/kafka-producer "topic" id h))

client/send! is asynchronous ?

i assumed it’s not and use a/go to make it async

why do you want to make it asynchronous?

conceptually, a go block that doesn't interact with any channels is just queuing up some work to do on a thread pool. it doesn't seem like running client/send! from a thread pool versus the current thread makes much of a difference unless there something else going on

also, if client/send! can take long enough to effectively matter, it should be inside thread, as putting it in go will clog up the limited core.async threadpool

@abdullahibra this is a very common error, it behaves fine in unit tests and simple experiments, and has severe consequences under load

also, given that kafka enforces ordering internally, writing to one topic from N concurrent threads will be slower than doing them one at a time

Perfect, thanks guys 🙂