jackdaw

https://github.com/FundingCircle/jackdaw
xi 2019-07-22T13:08:05.001800Z

Anyone here please? Having trouble understanding how Jackdaw maps onto the Java API. In streams.clj I can see a bunch of calls to protocols, and I can see the protocols, but... I don't get how they join up. For example, the protocol method is kstream but on the Java API that's called stream. Something must be doing the mapping? I've had a poke through the source and I just don't see it 😐

xi 2019-07-22T13:09:06.002300Z

Also not sure how to materialize a join of two KTables to a store

xi 2019-07-22T13:10:43.003Z

Great to have this library for free but well, the doc can be rather sporadic. A lot of methods take functions and there's no detail of the signature required.

2019-07-22T14:13:30.006300Z

Hey @dbernal, Yes you can use KafkaStreams with the avro serde but you need to pass it as an argument to the .stream or .table function rather than rely on the default configured in the application config. Here's where this is done in the example app... https://github.com/FundingCircle/jackdaw/blob/master/examples/word-count/src/word_count.clj#L76 This is because if you let kafka look it up by name, it won't find dynamically created Clojure classes of the type created by the avro-serde.

👍 1
2019-07-22T14:29:28.012800Z

Hey @slack1274, https://github.com/FundingCircle/jackdaw/blob/master/src/jackdaw/streams/interop.clj is where the rubber meets the road. I can't remember why stream was changed to kstream. Wherever you see a topic-config map that is referenced in the docs, the keys for this map are spec'd in "src/jackdaw/specs.clj". This should be better documented. When I get a chance, I'm going to add a high-level guide to "configuring" to go alongside the existing guides.