graalvm

Discuss GraalVM related topics. Use clojure 1.10.2 or newer for all new projects. Contribute to https://github.com/clj-easy/graal-docs and https://github.com/BrunoBonacci/graalvm-clojure. GraalVM slack: https://www.graalvm.org/slack-invitation/.
2020-04-14T11:18:09.083700Z

If I understand correctly, we can compile Clojure code into LLVM bytecode with GraalVM right?

2020-04-14T11:20:26.084500Z

If this is the case, we could also take this code and convert into Webassmbly?

jaihindhreddy 2020-04-14T11:22:42.086400Z

I dunno whether that's possible and if so, how straightforward it is to do. But, I'm pretty sure it won't be practical in terms of bundle size (assuming browser use) because we'd have to ship the runtime (stuff like GC). May I know the use case you have in mind?

2020-04-14T11:32:44.087200Z

I just want to write a simple numerical function to optimize some computation on the front end

2020-04-14T11:34:03.089100Z

It would be a rolling average function like m_n = m_{n-1} + x_n/n - x_{n-w-1}/n

2020-04-14T11:35:03.089900Z

I could try and write it into C++, but it was more an idea.

1๐Ÿ‘
jaihindhreddy 2020-04-14T11:48:50.091100Z

I heard Rust has a good compile-to-wasm story. That might yield a much smaller output.

2020-04-14T11:55:24.091400Z

Thanks

2020-04-14T11:55:35.091800Z

I will try. I never coded in rust though haha

2020-04-14T12:07:16.092400Z

if you go for rust, i highly recommend checking out this chapter before diving in: https://www.oreilly.com/library/view/programming-rust/9781491927274/ch04.html

2020-04-14T13:37:04.093800Z

I just want to write (reductions + coll) in rust, I hope it wonโ€™t translate into 100s of lines!

jaihindhreddy 2020-04-14T15:07:37.093900Z

Probably not. Rust is pretty concise. BTW we can take this to #other-languages or #off-topic because it isn't relevant to #graalvm or Clojure.

lukasz 2020-04-14T16:13:15.094800Z

Is it safe to assume that if my Clojure app uses only the JDK (and Clojure's std lib) it should be straight forward to compile it using native-image?

ghadi 2020-04-14T16:22:00.095Z

yes, but it dependsโ„ข

ghadi 2020-04-14T16:22:38.095800Z

I'm not sure of all the edge cases, but it's way more likely that you'll be good than if you incurred a bunch of random dependencies

lukasz 2020-04-14T16:25:32.097500Z

I thought that will be the case - all I have to do is to periodically fetch data from an HTTP endpoint, parse the json and send some of the data over to a statsd server - seems like all of these should be doable with just the JDK - JSON is probably the harder part, but I've noticed that babashka has cheshire as a dependency so It Should Work:tm:

ghadi 2020-04-14T16:26:34.097800Z

clojure.data.json should be fine

ghadi 2020-04-14T16:26:47.098300Z

the http part is trickier

lukasz 2020-04-14T16:27:02.098900Z

plain slurp seems to be working based on my testing via the repl

borkdude 2020-04-14T16:27:09.099300Z

@lukaszkorecki writing this as a simple babashka script should also work. equally fast startup, no compilation needed

borkdude 2020-04-14T16:28:01.101Z

https://github.com/borkdude/clj-http-lite and https://github.com/borkdude/babashka.curl are two http clients that work with graalvm

lukasz 2020-04-14T16:28:18.101700Z

@borkdude can babashka do loops, as in (while true (do (something) (Thread/sleep 10)) - I haven't had the time to play with it yet

borkdude 2020-04-14T16:28:25.101900Z

@lukaszkorecki yep

lukasz 2020-04-14T16:28:32.102100Z

This Changes Everything

lukasz 2020-04-14T16:28:40.102300Z

the udp/statsd part will be fun

borkdude 2020-04-14T16:29:13.102900Z

@lukaszkorecki I've included babashka.curl inside bb. curl can do amazing stuff, talk http over unix sockets for example. don't know about the udp stuff you're talking about

lukasz 2020-04-14T16:29:52.103500Z

Oh, it would be a simple statsd client - something I wrote before many times (in Common Lisp, Chicken Scheme, C etc etc )

lukasz 2020-04-14T16:30:01.103800Z

Nothing complicated

borkdude 2020-04-14T16:30:10.104200Z

is that doable with a socket? babashka also includes http://java.net.Socket etc

lukasz 2020-04-14T16:30:21.104600Z

Yeah, http://java.net.Socket should be enough

lukasz 2020-04-14T16:30:54.105500Z

statsd itself has a clojure implementation based on it https://github.com/statsd/statsd/blob/master/examples/statsd.clj

borkdude 2020-04-14T16:31:02.105800Z

@lukaszkorecki cool. and if you decide you want to run as a native image anyway, someone made a migration script from bb to native-image: https://github.com/MnRA/nativity ๐Ÿ™‚

lukasz 2020-04-14T16:31:27.106400Z

yeah, I saw that yesterday - looks really neat

lukasz 2020-04-14T16:31:38.106800Z

I'm really trying to avoid writing Go right now ;-)

borkdude 2020-04-14T16:32:43.107600Z

hmm, I don't think DatagramSocket etc is in bb right now: https://github.com/borkdude/babashka/blob/36c163a4449e43cc99be2ce5b32794d854d4ab68/src/babashka/impl/classes.clj#L56 but it's just a matter of adding it to that list.

borkdude 2020-04-14T16:33:02.108100Z

I'll take a look tomorrow if I can get that library working with bb

lukasz 2020-04-14T16:35:24.109400Z

Oh thanks - I'm drastically short on time, otherwise I'd look into it

borkdude 2020-04-14T16:36:02.109700Z

@lukaszkorecki was fun listening to the podcast (the REPL) btw!

lukasz 2020-04-14T16:36:22.110Z

thanks! I tried my best to stay on topic(s)

borkdude 2020-04-14T17:49:45.110500Z

@lukaszkorecki Are you on Mac or Linux?

borkdude 2020-04-14T17:53:27.111500Z

@lukaszkorecki With a few of those Datagram classes added, this now works:

$ ./bb -e '(load-file "/tmp/statsd.clj")' -e "(require '[statsd-client :as c]) (c/increment :foo)"
(where statds.clj is the example you linked to earlier)

borkdude 2020-04-14T17:54:44.112300Z

This is a linux binary: https://6696-201467090-gh.circle-artifacts.com/0/release/babashka-0.0.85-SNAPSHOT-linux-amd64.zip And this is a macos binary: https://6697-201467090-gh.circle-artifacts.com/0/release/babashka-0.0.85-SNAPSHOT-macos-amd64.zip with these classes added. Let me know if this is useful to you. If yes, I'll keep 'em in for the next release ๐Ÿ˜ƒ.

lukasz 2020-04-14T17:58:49.112700Z

Linux works, nobody's going to run this on a mac

lukasz 2020-04-14T17:59:09.113200Z

thanks a lot @borkdude - I'll see if I can stitch it all together today!

borkdude 2020-04-14T17:59:45.113600Z

@lukaszkorecki If this is going to run on Linux alpine, let me know, then you'll need the static binary

lukasz 2020-04-14T18:01:54.114300Z

target container size is not that important, it's more about the memory/cpu footprint of the tool itself

lukasz 2020-04-14T18:02:03.114600Z

so I'll most likely end up with a Debian image

lukasz 2020-04-14T18:03:03.115200Z

if it all works - I'm going to write about how we are monitoring our services in ECS with graphite and statsd

lukasz 2020-04-14T18:03:13.115400Z

so ๐Ÿคž

borkdude 2020-04-14T19:50:34.116Z

@lukaszkorecki Did an additional test with https://www.npmjs.com/package/statsd-logger

$ statsd-logger
Server listening on 0.0.0.0:8125
StatsD Metric:  foo:1|c

lukasz 2020-04-14T19:53:58.116800Z

Really cool - I already got a POC - just looking at CPU/RAM usage

borkdude 2020-04-14T19:54:13.117100Z

$ ./bb -e '(load-file "/tmp/statsd.clj")' -e "(require '[statsd-client :as c]) (while true (c/increment :foo) (Thread/sleep 100))"
$ statsd-logger
Server listening on 0.0.0.0:8125
StatsD Metric:  foo:1|c
StatsD Metric:  foo:1|c
StatsD Metric:  foo:1|c
...

borkdude 2020-04-14T19:57:54.118400Z

@lukaszkorecki If I put (System/gc) at the end of the iteration, it runs pretty consistently with 11 MB of memory (reported in Activity Monitor on a Macbook Pro)

lukasz 2020-04-14T19:58:37.119100Z

Nice! Let me wire polling the HTTP server and reporting metrics via UDP to see how it behaves

lukasz 2020-04-14T19:58:50.119500Z

thank you for the quick turnaround @borkdude

lukasz 2020-04-14T22:26:28.120900Z

It works ๐ŸŽ‰ but leaks memory somewhere, I need to dig into it - thank you for your help @borkdude

lukasz 2020-04-14T22:28:29.121400Z

The fix was (obviously) to move the loop out of bb and leave it to the shell ๐Ÿคท