I gave it another run. Chrome starts up with this:
Test output: https://gist.github.com/lbradstreet/8a481771fde676ac8d166dc89fd0312f
If I load the page manually, I see: https://imgur.com/a/PvvDiQk
cc/@manuel
@lucasbradstreet this is what I get on lein test
: https://gist.github.com/manuel-uberti/a91db5a83e4bcaffc12c0eb14964b82e
https://github.com/onyx-platform/onyx-kafka#attributes
:kafka/deserializer-fn
what is the recommended way to add some state (not serializable) in onyx ?
The way I’ve found so far would be to set a global state and get the global state inside the deserializer function.
I’ve also tried to create my own deserializer in my own task, but I have the same issue. You must use a namespace keyword. When you use a namespace keyword you can’t inject a state.
I can use lifecycle, create a new task for deserialization and inject the state using the lifecycle
I could also create a new def at runtime with the state, but all those solutions seems bad
(defn build-catalog [kafka-zookeeper kafka-topic bootstrap-servers]
[{:onyx/name :kafka-in
:onyx/type :input
:onyx/medium :kafka
:onyx/plugin :onyx.plugin.kafka/read-messages
:onyx/max-peers 1
:onyx/batch-size 50
:kafka/zookeeper kafka-zookeeper
:kafka/topic kafka-topic
:kafka/deserializer-fn ::deserializer-avro
:kafka/offset-reset :earliest}
(defn set-deserializer [schema-registry]
(defn deserializer-avro [data]
(avro/decode schema-registry data)))