jackdaw

https://github.com/FundingCircle/jackdaw
2020-11-19T15:27:59.050Z

I have a topic that I am using the json serde on and am writing records with string keys. When I use it as a source for another stream the records show up as keywordized keys. Is there something I can configure to make the next stream consume as stringified keys? To be clear, in the following example intermediate topic and final topic both use a json serde. Records of the form {"test record" 3} are being written to the intermediate topic and when peeked on they show up as {:test record 3}. I want to preserve the string keys as well as have json being written to the topic.

(-> (js/kstream builder intermediate-topic)
    (js/peek (fn [[_ r]] (pp/pprint r)))
    (js/to final-topic))

2020-11-19T15:34:24.052700Z

Afraid not. Here's the definition of the json serde though. I'd recommend having serdes defined locally in your project anyway. As you can see they're very simple and having your copy means you can easily tweak them as required https://github.com/FundingCircle/jackdaw/blob/master/src/jackdaw/serdes/json.clj#L36

2020-11-19T15:36:50.053500Z

Ah, https://github.com/FundingCircle/jackdaw/blob/master/src/jackdaw/serdes/json.clj#L34. You are right, it is pretty simple. Thanks!

2020-11-19T15:37:20.053700Z

I lost the debate around how much should be packed into jackdaw. Originally it was just a thin wrapper around the streams API but there was a desire to make it a "one-stop shop for kafka apps"