Clojurians Log v2

Hey Onyx team. I haven’t used Clojure or Onyx in about 6 months, but it’s exciting to see all the continued progress. I have a small question, which is that at my new workplace, we’re heavily invested into Kubernetes. It seems to me that K8 would conflict with the way Onyx already manages its own cluster with Zookeeper. But I’m curious if this has been looked at.

Onyx can run just fine on K8

We have run it for a bit on there. We run both Kafka and zk in K8. Onyx is just a set of pods

Essentially

Right. We don’t do anything magical with ZooKeeper outside of the testing only server that can be startup in the env, so all you need to do is startup ZooKeeper containers in k8s and point Onyx at them. :)

I figured there would be problems with Zookeeper/Onyx and K8 fighting to manage bringing nodes (pods) up and down. But awesome. I’ll dig in more then.

no zookeeper + k8s is not a problem. onyx would probably look a bit differently if it was designed from the ground up for k8s, but i’ve been running it without problems for quite some time

https://github.com/kubernetes/charts/tree/master/incubator/zookeeper

there’s even a ZK chart already

Most of the problems we’ve hit with k8s and kafka/zookeeper were due to how well they run as StatefulSets, and were related to DNS resolution issues that are slowly being fixed in upstream ZooKeeper and Kafka.

i think it’s mostly Kafka being problematic with broker / node discovery, right ?

e.g. https://issues.apache.org/jira/browse/ZOOKEEPER-1846?attachmentSortBy=fileName and https://issues.apache.org/jira/browse/ZOOKEEPER-1506?attachmentSortBy=dateTime on the zookeeper side

ah interesting

Kafka was fine in that respect. Actually I think it was mostly related to these issues so we needed to patch in a ZooKeeper client with a fix into Kafka.

so that would also be related to the Curator discussion we had last week ?

That one was more for SSL for us. But yes, I think we need to upgrade it for that reason too.

ok, good to know