Clojurians Log v2
graalvm
Hey there,
I created my first tool using GraalVM 🎉
https://github.com/jorinvo/prometheus-pushgateway-cleaner
The usecase is pretty specific but I thought the setup might be interesting for others to look at :)
The tool is intended to run in a Docker container and I keep it running constantly that’s why having low memory overhead is nice.
The Docker image is built and deployed automatically through a Github Action.
Also, the project uses tools.deps and not lein.
The trickiest part was that first I wanted to pack the binary in a FROM scratch or plain Alpine container.
But the DNS lookup of clj-http-lite failed. So instead I am using an Alpine image with glibc included now.
I also tried using https://github.com/gnarroway/hato because I was hoping to use the Java 11 HTTPClient, but here Graal doesn’t even compile.
(It fails with an error that seems to be coming from clojure.spec… Error: unbalanced monitors: mismatch at monitorexit )
Maybe someone knows a way to do HTTP calls in a GraalVM binary in a FROM scratch Docker image 🙂
@hi895 the simplest way is to build within the docker container and copy the DNS dynamic .so
Have a look at this https://github.com/leafclick/pgmig/blob/master/Dockerfile
Thanks @katox! That looks interesting 🙂 For now I think I stick with the alpine-glibc approach but if the image size is really critical, that is definitely a neat solution!
Dev image size is critical? While building? Because this target image really just contains the shared libraries and the final graal native binary, nothing else.
I meant, copying all the shared libraries is a great solution to get a small FROM scratch image. With alpine the image is probably a little bigger.
The reason for me to still stick with alpine for now is simply that the Dockerfile has less lines of code and feels easier to maintain.. but it’s really cool that all it takes to run from scratch is copying those libraries 🙂
Note that it is not totally safe to build a graal image on one system and run it against shared libraries of another. Slight incompatibilities or memory corruption would be hell to debug. Unfortunately you can't use static linking if you need DNS resolving.
hm I do use --static … https://github.com/jorinvo/prometheus-pushgateway-cleaner/blob/master/deps.edn#L22
or you refer to something else?
Yes, you can't use static linking, therefore -static.
See https://github.com/oracle/graal/issues/571
Interesting, because that is not the issue I get. I use --static . DNS lookup works as long as glibc is included in the final image.
If glibc is not included, I also don’t get a SEGFAULT. I get a UnknownHostException.
Yes, it is in the linked issues, see https://github.com/oracle/graal/issues/1151
Yes, that is exactly the issue I also came across 🙂
I talked about the issue with Oracle guys. In the end they marked the whole "static linking with glibc" as an usupported case. Not just by them but GNU as well. So I ended up with something that's more or less the same for me minus some initial hunting of dynamically linked shared libraries. But if you stick with a stable distribution like stable debian they are very unlike to ever change.
That sounds like a good compromise, yes 🙂 Sad to hear the don’t plant to support this 😕 The easy of creating static binaries is something I miss from writing Go 😉
also thank you @borkdude for #babashka! The project is a great resource for graal stuff :)
@hi895 Error: unbalanced monitors: mismatch at monitorexit => 1.10.2-alpha1 fixes that
this is a compatible version of hato: https://github.com/borkdude/hato - but compilation takes long (8 minutes)
np 🙂
That’s awesome! Thank you 🙂 But 8 minutes compilation time is a 🙈 For that I rather use the little bit bigger Docker image 😄
@hi895 depending on what you need to do with http, babashka is leaning more towards this approach these days: https://github.com/borkdude/babashka.curl
"just use curl" with a sprinkle of clojure.
clj-http-lite works fine for my usecase I was just hoping to find a way to run FROM scratch . Requiring curl doesn’t help toward that goal. The current 17mb image is small enough though I guess 🙂
right 🙂
I don't fully get what this tool is about, but I love that you made something and it worked for you
yes, really cool that this works 🙂 The usecase is very specific, yes 😆
I am curious which part of hato was not compatible? It seems like for me the normal hato just compiled and the Docker image runs. Compilation time was 480,528.08 ms btw 😄
It has conditional requires.
yeah, that's what I said, 480s = 8 minutes 🙂
ok, unfortunately hato does have the same problem that DNS resolving doesn’t work in a from scratch image.. so I will stick with the current solution 🙂