jackdaw

https://github.com/FundingCircle/jackdaw
2018-12-20T08:50:53.005100Z

@andrea.crotti see above... 🙂

2018-12-20T08:53:47.008Z

@truemped I work for funding circle but I didn't work on this library (I do mostly clojurescript actually), I mentioned my colleagues that we have a question though

2018-12-20T08:54:17.008300Z

cool, thanks!

2018-12-20T08:54:54.009600Z

I though I read your name in the commit logs. Probably not...

2018-12-20T08:54:59.009700Z

Feel free to open github issues of course which at least don't disappear

2018-12-20T08:55:12.010300Z

I just did a couple of prs this week for the first time

2018-12-20T08:55:34.010700Z

I'm thinking the error is in my code somehow or I'm using it wrong.

2018-12-20T08:56:41.011900Z

Could be but could also be just something missing in the library since we never had that need so far

2018-12-20T08:57:51.013400Z

Just fyi he biggest contributors are in the pdt timezone

2018-12-20T12:22:15.016400Z

Hey @truemped. The value you're getting in the reducer might not be what you expect. I think while it's trying to serialize the reduction to your intermediate topic, it's throwing a serialization error because something that expects a String is getting a window instead.

2018-12-20T12:25:45.016600Z

ah, right

2018-12-20T12:47:08.019500Z

Actually looking at the test in here https://github.com/FundingCircle/jackdaw/blob/master/test/jackdaw/streams_test.clj#L685 ....do you have something similar in your code to this line that extracts the key out of the window?

(k/map (fn [[k v]] [(.key k) v]))

2018-12-20T13:34:08.020200Z

no, not really

2018-12-20T13:36:35.020800Z

how do the arguments to reduce look like? I thought it was just the value map

2018-12-20T13:37:43.021200Z

I'm also not using group-by but group-by-key

2018-12-20T13:50:49.022500Z

ah, I see. Up until (k/map (...)) the key is still a windowed key. My output-topic is non-windowed and should only receive the last value per time-window. So I need to transform the key to a non-windowed key... right?!

2018-12-20T13:50:50.022700Z

🙂

2018-12-20T14:51:17.023Z

works like a charm! Thank you @cddr

2018-12-20T15:43:25.023600Z

Ah sorry I went awol. Seems like you figured it out though 🙂

2018-12-20T19:39:28.024600Z

yes, essentially like I said. Everything works fine up until I try to (k/to ...) a topic without windowed keys. For this I was missing the ('k/map ...) part.