Clojurians Log v2

If I understand correctly, we can compile Clojure code into LLVM bytecode with GraalVM right?

If this is the case, we could also take this code and convert into Webassmbly?

I dunno whether that's possible and if so, how straightforward it is to do. But, I'm pretty sure it won't be practical in terms of bundle size (assuming browser use) because we'd have to ship the runtime (stuff like GC). May I know the use case you have in mind?

I just want to write a simple numerical function to optimize some computation on the front end

It would be a rolling average function like m_n = m_{n-1} + x_n/n - x_{n-w-1}/n

I could try and write it into C++, but it was more an idea.

I heard Rust has a good compile-to-wasm story. That might yield a much smaller output.

Thanks

I will try. I never coded in rust though haha

if you go for rust, i highly recommend checking out this chapter before diving in: https://www.oreilly.com/library/view/programming-rust/9781491927274/ch04.html

I just want to write (reductions + coll) in rust, I hope it won’t translate into 100s of lines!

Probably not. Rust is pretty concise. BTW we can take this to #other-languages or #off-topic because it isn't relevant to #graalvm or Clojure.

Is it safe to assume that if my Clojure app uses only the JDK (and Clojure's std lib) it should be straight forward to compile it using native-image?

yes, but it depends™

I'm not sure of all the edge cases, but it's way more likely that you'll be good than if you incurred a bunch of random dependencies

I thought that will be the case - all I have to do is to periodically fetch data from an HTTP endpoint, parse the json and send some of the data over to a statsd server - seems like all of these should be doable with just the JDK - JSON is probably the harder part, but I've noticed that babashka has cheshire as a dependency so It Should Work:tm:

clojure.data.json should be fine

the http part is trickier

plain slurp seems to be working based on my testing via the repl

@lukaszkorecki writing this as a simple babashka script should also work. equally fast startup, no compilation needed

https://github.com/borkdude/clj-http-lite and https://github.com/borkdude/babashka.curl are two http clients that work with graalvm

@borkdude can babashka do loops, as in (while true (do (something) (Thread/sleep 10)) - I haven't had the time to play with it yet

@lukaszkorecki yep

This Changes Everything

the udp/statsd part will be fun

@lukaszkorecki I've included babashka.curl inside bb. curl can do amazing stuff, talk http over unix sockets for example. don't know about the udp stuff you're talking about

Oh, it would be a simple statsd client - something I wrote before many times (in Common Lisp, Chicken Scheme, C etc etc )

Nothing complicated

is that doable with a socket? babashka also includes http://java.net.Socket etc

Yeah, http://java.net.Socket should be enough

statsd itself has a clojure implementation based on it https://github.com/statsd/statsd/blob/master/examples/statsd.clj

@lukaszkorecki cool. and if you decide you want to run as a native image anyway, someone made a migration script from bb to native-image: https://github.com/MnRA/nativity 🙂

yeah, I saw that yesterday - looks really neat

I'm really trying to avoid writing Go right now ;-)

hmm, I don't think DatagramSocket etc is in bb right now: https://github.com/borkdude/babashka/blob/36c163a4449e43cc99be2ce5b32794d854d4ab68/src/babashka/impl/classes.clj#L56 but it's just a matter of adding it to that list.

I'll take a look tomorrow if I can get that library working with bb

Oh thanks - I'm drastically short on time, otherwise I'd look into it

@lukaszkorecki was fun listening to the podcast (the REPL) btw!

thanks! I tried my best to stay on topic(s)

@lukaszkorecki Are you on Mac or Linux?

@lukaszkorecki With a few of those Datagram classes added, this now works:

$ ./bb -e '(load-file "/tmp/statsd.clj")' -e "(require '[statsd-client :as c]) (c/increment :foo)"

This is a linux binary: https://6696-201467090-gh.circle-artifacts.com/0/release/babashka-0.0.85-SNAPSHOT-linux-amd64.zip And this is a macos binary: https://6697-201467090-gh.circle-artifacts.com/0/release/babashka-0.0.85-SNAPSHOT-macos-amd64.zip with these classes added. Let me know if this is useful to you. If yes, I'll keep 'em in for the next release 😃.

Linux works, nobody's going to run this on a mac

thanks a lot @borkdude - I'll see if I can stitch it all together today!

@lukaszkorecki If this is going to run on Linux alpine, let me know, then you'll need the static binary

target container size is not that important, it's more about the memory/cpu footprint of the tool itself

so I'll most likely end up with a Debian image

if it all works - I'm going to write about how we are monitoring our services in ECS with graphite and statsd

so 🤞

@lukaszkorecki Did an additional test with https://www.npmjs.com/package/statsd-logger

$ statsd-logger
Server listening on 0.0.0.0:8125
StatsD Metric:  foo:1|c

Really cool - I already got a POC - just looking at CPU/RAM usage

$ ./bb -e '(load-file "/tmp/statsd.clj")' -e "(require '[statsd-client :as c]) (while true (c/increment :foo) (Thread/sleep 100))"

$ statsd-logger
Server listening on 0.0.0.0:8125
StatsD Metric:  foo:1|c
StatsD Metric:  foo:1|c
StatsD Metric:  foo:1|c
...

@lukaszkorecki If I put (System/gc) at the end of the iteration, it runs pretty consistently with 11 MB of memory (reported in Activity Monitor on a Macbook Pro)

Nice! Let me wire polling the HTTP server and reporting metrics via UDP to see how it behaves

thank you for the quick turnaround @borkdude

It works 🎉 but leaks memory somewhere, I need to dig into it - thank you for your help @borkdude

The fix was (obviously) to move the loop out of bb and leave it to the shell 🤷