Clojurians Log v2

Java 17 is dropping support for AOT 😮 How we will compile Clojure programs with GraalVM?

This doesn't have to do anything with Clojure AOT. Clojure AOT is just regular byte code generation, like javac does with .java programs.

It seems they are removing some experimental tools like jaotc which isn't used by anyone in production.

Good to know 🙂 Thank you

What may cause this:

Fatal <error:com.oracle.svm.core.util.VMError>$HostedError: com.oracle.svm.core.util.VMError$HostedError: fierycod.holy_lambda.util$call.invokeStatic(Object, Object): has no code address offset set.

I have seen this a couple of times. Are you perhaps compiling with leiningen?

Nope. With depstar. On leiningen I have never seen it. Actualy I've switched from leiningen to depstar and then bizzare things with AOT started to happen.

@ericdallo had it too with clojure-lsp. After switching from lein to regular (compile '<http://my.app|my.app>) it went away

you can post your uberjar and make an issue at oracle/graal about it

Yeah, I had the same and I'm not remembering how we fixed it 😅, but I think it was indeed related with lein

it was "fixed" by changing from lein aot to regular (compile '<http://my.app|my.app>)

Oh, yeah, I needed to migrate to depstar to fix that

It's some random bug 😄 I'm using depstar and I've just purged all the classes, recompiled three times and on third Graalvm compilation was successful. Btw are you using native configuration to produce executable?

Hello. I believe I've heard in defn podcast #50 with borkdude that there's some flag in graalvm's native-image that "forgives" potential usage of reflection. You might want to try using that flag if there's a chance that such code doesn't actually get executed in your specific runtime path. Does such flag rings a bell? Which one was that? (it's a pitty that it's not easy to "grep" a podcast episode 🙂)

@brdloush https://www.graalvm.org/reference-manual/native-image/Reflection/

Thanks @borkdude. I tried using reflection.json that gets created in babashka native image compilation as an example and stripped it down to just java.net.URLEncoder and java.net.URLDecoder. I've chosen these 2 because lein uberjar gives me following reflection warnings:

Reflection warning, clojure/data/xml/jvm/name.clj:35:1 - call to static method decode on java.net.URLDecoder can't be resolved (argument types: unknown, java.lang.String).
Reflection warning, clojure/data/xml/jvm/name.clj:38:1 - call to static method encode on java.net.URLEncoder can't be resolved (argument types: unknown, java.lang.String).

[{
        "name": "java.net.URLEncoder",
        "allPublicMethods": true,
        "allPublicFields": true,
        "allPublicConstructors": true
    },
    {
        "name": "java.net.URLDecoder",
        "allPublicMethods": true,
        "allPublicFields": true,
        "allPublicConstructors": true
    }
]

native-image -H:ReflectionConfigurationFiles=reflection.json --no-fallback -jar target/uberjar/myapp-0.1.0-SNAPSHOT-standalone.jar

Error: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method java.lang.ClassLoader.defineClass(String, byte[], int, int) is reachable

@brdloush Take a look at the other graalvm arguments in babashka. You need a few others.

You need at least:

--initialize-at-build-time

And sometimes --report-unsupported-elements-at-runtime

It depends on what you are doing. If you want to rely on eval then that's not going to work

Well it's not my code which is causing the trouble I guess. I believe those clojure/data/xml/jvm/name.clj:35:1 warnings ^^^ comes from the fact that I'm using one of these libraries:

[com.cognitect.aws/api "0.8.505"]
[com.cognitect.aws/endpoints "1.1.11.1001"]
[com.cognitect.aws/s3 "811.2.889.0"]
[com.cognitect.aws/iam "811.2.889.0"]
[com.cognitect.aws/sts "811.2.889.0"]                 
[throttler "1.0.0"]
[org.clojure/tools.cli "1.0.206"] 

clojure.data.xml works quite well with graalvm, I'm also using it in babashka

it's actually the aws library itself which is problematic since it uses quite a lot of dynamics

but we (me and some others) have managed to "natify" that as well: https://github.com/babashka/pod-babashka-aws

Sounds like I might even switch to babashka for this utility I'm building then 😄

Here are some hints to what is important to get it working with graalvm: https://github.com/BrunoBonacci/graalvm-clojure/tree/master/aws-api-s3

oooh,.... reading the README.md : • Credentials: custom flows are supported, but not by extending CredentialsProvider interface. See https://github.com/babashka/pod-babashka-aws#credentials for options. This might be the issue, I'm actually using (reify credentials/CredentialsProvider that's very likely the problem.

That note only applies to the babashka aws pod, not to using aws-api as a lib in graalvm native

@brdloush You may want to check this as well if you're compiling with graalvm: https://github.com/FieryCod/holy-lambda/blob/master/examples/bb/native/sqs.example/src/sqs/example/core.cljc Avoid dynamic load: https://github.com/FieryCod/holy-lambda/blob/master/examples/bb/native/sqs.example/src/sqs/example/static_load.cljc and use following graalvm args https://github.com/FieryCod/holy-lambda/blob/master/examples/bb/native/sqs.example/bb.edn#L60. 🙂

@brdloush You can automate generation of native configuration. For instance holy-lambda has a in-agent macro which executes some code only in agent context. https://www.graalvm.org/reference-manual/native-image/BuildConfiguration/#assisted-configuration-of-native-image-builds https://github.com/FieryCod/holy-lambda/blob/master/src/fierycod/holy_lambda/agent.clj#L21 All you need is: 1. Add some in-context calls which utilizes aws-api 2. Uberjar application 3. Generate all configs which should be passed to native-image

java -agentlib:native-image-agent=config-output-dir=resources/native-configuration \
			           -Dexecutor=native-agent \
			           -jar target/output.jar

Thanks a lot to both of you guys. Right now I'll continue adding features to the uberjar-only version that needs the jvm, but later I'll definitely give this a try. I was just curious whether I'll be able to easily native-compile that app, but it seems that it needs some non-trivial effort at the moment.