jackdaw

https://github.com/FundingCircle/jackdaw
Travis 2019-04-15T21:07:49.001500Z

Hey guys. I think we may have ran into a bug but trying to confirm. We are trying to use the streams/kstreams api to setup a stream on multiple input topics. However when doing this it does not seem to be losing the serializer that is defined with this topic. I think I have tracked it to this code but I am not sure

(kstreams
    [_ topic-configs]
    (clj-kstream
     (let [topic-names (clojure.core/map :topic-name topic-configs)]
       (.stream streams-builder
                ^Collection topic-names)))

Travis 2019-04-15T21:08:16.002Z

if this is the code being used its just setting the name but doing nothing with the serializer

Travis 2019-04-15T21:14:42.002800Z

Actually maybe this is an issue with kafka streams in general. Not sure you can consume from multiple topics and these topics each have different formats

1
2019-04-16T14:13:55.005500Z

There is a trick though. If you use the byte serde, you can use your own logic to find the appropriate serde for a given topic. Check out the test-machine for code that does this.