Clojurians Log v2

hey there. I am working on a project with a number of grouped tasks and when trying to query the state of said tasks (via the onyx-peer-http-query server), I’m not receiving anything back in the :contents when not supplying the groups in my query params.

I’m curious if this is expected behavior and if so, where should I go to retrieve the group-ids relevant to my tasks (something I also can’t seem to figure out, lol).

Onyx version?

We did fix an issue like that in peer-http-query in the last month or so

https://github.com/onyx-platform/onyx-peer-http-query/issues/5 hmm, probably not this issue

Sounds like you’re just getting empty results instead.

If you could fork http-peer-query and demonstrate the issue via https://github.com/onyx-platform/onyx-peer-http-query/blob/0.12.x/test/onyx/health_test.clj, that’d be great.

we’re using 0.12.

will take a look at forking right now

K. Thanks. We may have seen something similar. Hopefully it’s easily reproducible. If not, let us know and we can dig in further

k, so it looks like when I don’t specify a groups query param, the health test fails with the following:

FAIL in (health-test) (health_test.clj:137)

expected: (get-in (clojure.edn/read-string (:body response)) [:contents "A stealthy fo"])
  actual: nil

  values: (get-in
           {:contents {}
            :grouped? true
            :window {:window/aggregation :onyx.windowing.aggregation/count
                     :window/id :my-window-id
                     :window/range [5 :minutes]
                     :window/storage-strategy [:incremental :ordered-log]
                     :window/task :my/inc
                     :window/type :fixed
                     :window/window-key :event-time}}
           [:contents "A stealthy fo"])

I simply commented out this line: https://github.com/onyx-platform/onyx-peer-http-query/blob/0.12.x/test/onyx/health_test.clj#L133

and ran the tests

Oh, I realised that it was by design (probably bad design)

just to clarify, my expectation is that if I don’t specify groups, that I should get contents for all the groups

Yeah, that’s really how it should work.

:thumbsup:

k, cool, thanks. I can fix that tonight.

that would be awesome. thanks so much!

you guys are awesome!

How do you find the usability of that feature? I ummed and ahhed about providing a way to do it without supplying the allocation version, but I was worried that users would get at the state over an entire cluster in an inconsistent way. That said, it would make it simpler to play with.

overall its been very useful. just had a curveball with a grouped task. all of the other tasks i have aren’t, so it’s been really great to be able to query the state and see what’s going on in my windowed tasks

the params weren’t too difficult to get at either, which made it a really usable feature, imo

Great 🙂

Yeah, it’s been really nice for us too. Even if you only use it for debugging.

Can you please give onyx-peer-http-query 0.12.0.1-20171215.010158-3 a try?

If it works for you I’ll release 0.12.0.1

well, unfortunately, i’d need to redeploy my onyx setup, which takes some time (and also my manager’s sign off 😄), so i’d probably need to test that in the morning.

Understood 😄

Well, let me know whenever and I’ll cut you a release.

that’s awesome. thanks so much! will ping you tomorrow for sure

hey @lucasbradstreet, good news. I tested out your change and it looks great! without specifying group keys I get back everything instead of nil. if I specify the groups, it still works as well. kudos! thanks so much for getting that out so quickly. :thumbsup:

Great. Thanks for the update 🙂.

0.12.0.1 is going through our release process now.

💥 awesome