Clojurians Log v2

@dm3 @lmergen I have something to show you regarding the Hystrix clone in Clojure. Would you be interested to review it? It’s an extraction of something that has been running partially in production without obvious flaws

yes, i would love to take a look!

sure 🙂

btw, would anyone here find a Clojurescript Manifold port useful?

Sure, I think this would be great! Cljs support is core.async's main advantage over Manifold

I definitely have async code/patterns that I would like to reuse in the browser or potentially on Node

I'm trying to port the let-flow macro - the hardest part

have everything else pretty much working

awesome!

I have been trying to avoid let-low and stick to chain and that works up to some point. So I think even a Manifold cljs without let-flow may be very useful.

yeah, I think let-flow is more important in Clojurescript as the concurrency patterns are more limited

and you compete with go

I don't use it in Clojure either

hmm, you think users will need d/loop a lot in cljs, replacing go-loop?

I have d/loop working

yeah, that should replace go-loop

@dm3 @lmergen I’ve invited you via Github. It is missing some documentation. I think it’s best to start in examples

I don't see anything

@dm3 .. it will be interesting to see how core.async and Manifold patterns compare in cljs/the browser - and if Manifold could win by having more functional syntax. Also I think it would compete with RxJS (https://github.com/funcool/beicon) and https://github.com/funcool/promesa

yes, it's much simpler than RxJS

in terms of concepts at least

true

@dm3 you should have an invitation via email (assuming you are dm3 on github) or you can accept the invitation here https://github.com/millmobile

@dm3 what type of apps are you planning to use Manifold for in the browser? For general UIs it seems everyone uses Re-frame nowadays - Om also droped core.async ..

I use Hoplon 🙂

I don't really know where I'd use it. Probably everywhere I used core.async before

which is mostly event buses, web sockets, etc.

it's much less useful in the browser than it is on the backend

If you are into reactive programming/Hoplon, have you considered Pulsars Rx namespace?

If you would create manifold.cljs it would allow my Hystrix clone to run in the browser 🙂

At least nice for a demo 😉

interesting how you avoided using any java.util.concurrent constructs 🙂 was this on purpose?

yes partially I guess. Maybe there are better ways for some parts, please let me know 🙂 But e.g. for the semaphores it was convenient to use a Manifold stream ,because you can increase the semaphore limit dynamically. E.g. with blocking queues this would not be possible (also a limitation of Hystrix itself)

I’ve updated the README btw, maybe it gives some extra information

@jeroenvandijk sounds great! didn’t know you were behind AdGoji btw, everything starts to make sense 🙂

Cool 🙂

looks quite solid actually

you said it’s already running in production ?

you’re using it for your RTB i see ?

Yes I have been running it on a single node from time to time. Unfortunately the node is always gone the next day (it could also be some unlucky autoscaling), so I’m afraid the reporting part leaks somewhere. I haven’t dug into that yet. I wanted to get the (hystrix) reporting running and basic features first. That alls seems to work fine and also the performance seems to be comparable to what we have running on the other nodes. So needs a bit more testing before I dare to declare it to be stable

and yes this is running in our bidder code

yeah, i see

the example is slightly different than what is running in production though 🙂

interesting design choice behind the global registry

i’m not sure whether that’s the best approach

oh wait, it’s on a per-component basis

yes i guess it is a bit confusing if you look at the code. The idea is that each component is responsible for what is readable and writable. This makes train wrecks less likely and moves the complexity to the components. Maybe it is an overkill for a small library like this

yeah, it’s good enough i think

I do think it has more use cases though. Configuration often becomes a hassle to manage

yes that’s true, and this way it integrates well into component

for me the biggest concern is that i can easily map such a context to a single endpoint — this works in this case

the approach i’ve personally been taking was implement this as an aleph.http.client middleware

Yes I think the rest of the code doesn’t need to know the configuration is decentralised e.g. if you call registry-snapshot you will get a normal map. Maybe it easier to break code that is depending on it because it so easy to change the location of components this way

@lmergen If you want we can discuss more options (e.g. the Kibana idea) also in person and see if there is an option to work on it together

@jeroenvandijk sounds like a great idea! i have quiet a lot of these “itches” i want to scratch and plenty of ideas 🙂

Is there a way to do streaming (bulk) uploads with aleph.http?

I know you can pass in :raw-stream? true but that looks like it only works on the response, not the request.

ah, looks like I can pass in an arbitrary stream coercable via byte-streams

@andreas-thoelke getting back on the Pulsar Rx - no I haven't used it. I've played with Plasar actors though. I've never used a proper dataflow impl on the backend yet, although some use cases I had seemed very suitable.