Clojurians Log v2

I remembered that the "not in a catalog" way was explained in Onyx Template Webcast (starting at 4:43: https://youtu.be/5hFV_W9-10s?t=283). The mechanism is called "task bundles" and are introduced as "nice units of composition". They "group together schema checks with the tasks themselves". You can also specify tasks by parametrizing them. At 7:45 (https://youtu.be/5hFV_W9-10s?t=465) they show an example. And at 8:45 (https://youtu.be/5hFV_W9-10s?t=525) they introduce "bundle modifiers", that can be used e.g. to add logging to a task easily.

actually, under the hood, i believe add-task does add things to the catalog

it's just a convenience function when you want to compose things rather than define them as data

Yes, @lmergen, it is done like that: https://github.com/onyx-platform/onyx/blob/0bc25bc0292971557734ecd496b81e75fbb94989/src/onyx/job.clj#L41

I'm trying to run a job with multiple peer nodes (each with 5 virtual peers) but I'm having a hard time getting it to run correctly. The nodes seem to join the cluster and create their peers but once I start the job it hangs and kills the virtual peers and then starts them again. This repeats indefinitely until one I kill one of the peers. And if the job has enough peers to run from just the one peer node it runs correctly until completion. Any ideas on what I could be doing wrong in my set up? Currently I have 2 machines running the start-peer command as a java process that runs forever and I submit the job through my dev laptop.

why is the onyx dashboard unmaintained? https://github.com/onyx-platform/onyx-dashboard/commit/55f4a6b0fe35bfda1f228b7476b54248eae3e3a6

isn’t it a crucial piece of the onyx platform?

My guess would be that it was a first pass at the platform that became pyrostore or something? Why would it be crucial? It has nothing to do with actual data processing, right?

ok maybe crucial was a strong word 🙂 but it’s part of the getting started instructions https://github.com/onyx-platform/onyx-starter#dashboard

just read about pyrostore. looks like it was part of the distributed masonry/confluent acquisition.project page is down right now http://pyrostore.io/

http://www.onyxplatform.org/jekyll/update/2017/02/08/Pyroclast-Preview-Simulation.html ah yeah i can definitely see how they leveraged the dashboard

Dashboards are sadly what people pay money for...