Clojurians Log v2
Clojure programming
Channels
Apps
jackdaw
https://github.com/FundingCircle/jackdaw
2019-09-08T01:17:29.006700Z
Hi, I am playing around with Jackdaw and Willa (and new to them and Kafka) and trying to build a TimeWindowed workflow and having some troubles. How do I define an entity and topology for aggregating by a time window? Here is my attempt munging the Willa example:
(def workflow
[[:input-topic :aggregate-stream]
[:aggregate-stream :output-topic]])
(def entities
{:input-topic {::w/entity-type :topic
:topic-name "timed-readings-topic"
:replication-factor 1
:partition-count 1
:key-serde (serdes.edn/serde)
:value-serde (serdes.edn/serde)}
:aggregate-stream {::w/entity-type :ktable
:willa.core/group-by-fn (fn [[k v]] (:device-number v))
:willa.core/window (TimeWindows/of (* 2 60 1000))}
:output-topic {::w/entity-type :topic
:topic-name "timed-readings-output-topic"
:replication-factor 1
:partition-count 1
:key-serde (serdes.edn/serde)
:value-serde (serdes.edn/serde)}})No method in multimethod 'coerce-to-ktable' for dispatch value: class jackdaw.streams.interop.CljTimeWindowedKStream exception. Am I doing this right? or on the right path?
davewm 2019-09-13T14:56:00.010200Z
hi @pdmct, that looks like a bug in Willa to me - I’ll take a look this evening if I have chance 🙂
davewm 2019-09-13T14:58:20.010400Z
could you create an issue in the Willa github repo please? Just conscious that your message will disappear in a few days, and it’d be good to have a more permanent record of this problem
2019-09-14T02:37:35.000100Z
no probs
2019-09-08T02:53:14.007500Z
Changing entity-type to KStream seems to have fixed this.