graalvm

Discuss GraalVM related topics. Use clojure 1.10.2 or newer for all new projects. Contribute to https://github.com/clj-easy/graal-docs and https://github.com/BrunoBonacci/graalvm-clojure. GraalVM slack: https://www.graalvm.org/slack-invitation/.
Karol W贸jcik 2021-05-04T15:04:02.182500Z

Java 17 is dropping support for AOT 馃槷 How we will compile Clojure programs with GraalVM?

borkdude 2021-05-04T15:10:18.183100Z

This doesn't have to do anything with Clojure AOT. Clojure AOT is just regular byte code generation, like javac does with .java programs.

borkdude 2021-05-04T15:10:40.183700Z

It seems they are removing some experimental tools like jaotc which isn't used by anyone in production.

1馃憤
Karol W贸jcik 2021-05-04T15:14:00.185Z

Good to know 馃檪 Thank you

Karol W贸jcik 2021-05-04T15:24:06.185600Z

What may cause this:

Fatal <error:com.oracle.svm.core.util.VMError>$HostedError: com.oracle.svm.core.util.VMError$HostedError: fierycod.holy_lambda.util$call.invokeStatic(Object, Object): has no code address offset set.
It's the first time I'm seeing it

borkdude 2021-05-04T15:24:36.185900Z

I have seen this a couple of times. Are you perhaps compiling with leiningen?

Karol W贸jcik 2021-05-04T15:24:59.186300Z

Nope. With depstar. On leiningen I have never seen it. Actualy I've switched from leiningen to depstar and then bizzare things with AOT started to happen.

borkdude 2021-05-04T15:25:44.187Z

@ericdallo had it too with clojure-lsp. After switching from lein to regular (compile '<http://my.app|my.app>) it went away

borkdude 2021-05-04T15:27:28.187500Z

you can post your uberjar and make an issue at oracle/graal about it

ericdallo 2021-05-04T15:27:29.187600Z

Yeah, I had the same and I'm not remembering how we fixed it 馃槄, but I think it was indeed related with lein

borkdude 2021-05-04T15:27:51.188Z

it was "fixed" by changing from lein aot to regular (compile '<http://my.app|my.app>)

1馃憤
ericdallo 2021-05-04T15:27:59.188200Z

Oh, yeah, I needed to migrate to depstar to fix that

Karol W贸jcik 2021-05-04T15:28:49.189300Z

It's some random bug 馃槃 I'm using depstar and I've just purged all the classes, recompiled three times and on third Graalvm compilation was successful. Btw are you using native configuration to produce executable?

Tomas Brejla 2021-05-04T16:15:56.192100Z

Hello. I believe I've heard in defn podcast #50 with borkdude that there's some flag in graalvm's native-image that "forgives" potential usage of reflection. You might want to try using that flag if there's a chance that such code doesn't actually get executed in your specific runtime path. Does such flag rings a bell? Which one was that? (it's a pitty that it's not easy to "grep" a podcast episode 馃檪)

Tomas Brejla 2021-05-04T16:23:57.196Z

Thanks @borkdude. I tried using reflection.json that gets created in babashka native image compilation as an example and stripped it down to just java.net.URLEncoder and java.net.URLDecoder. I've chosen these 2 because lein uberjar gives me following reflection warnings:

Reflection warning, clojure/data/xml/jvm/name.clj:35:1 - call to static method decode on java.net.URLDecoder can't be resolved (argument types: unknown, java.lang.String).
Reflection warning, clojure/data/xml/jvm/name.clj:38:1 - call to static method encode on java.net.URLEncoder can't be resolved (argument types: unknown, java.lang.String).
But even with following reflection.json
[{
        "name": "java.net.URLEncoder",
        "allPublicMethods": true,
        "allPublicFields": true,
        "allPublicConstructors": true
    },
    {
        "name": "java.net.URLDecoder",
        "allPublicMethods": true,
        "allPublicFields": true,
        "allPublicConstructors": true
    }
]
.. I'm still not able to create a native-only no-fallback image:
native-image -H:ReflectionConfigurationFiles=reflection.json --no-fallback -jar target/uberjar/myapp-0.1.0-SNAPSHOT-standalone.jar

Error: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method java.lang.ClassLoader.defineClass(String, byte[], int, int) is reachable

borkdude 2021-05-04T16:25:37.196500Z

@brdloush Take a look at the other graalvm arguments in babashka. You need a few others.

borkdude 2021-05-04T16:26:12.196800Z

You need at least:

--initialize-at-build-time

borkdude 2021-05-04T16:26:43.197100Z

And sometimes --report-unsupported-elements-at-runtime

borkdude 2021-05-04T16:27:07.197500Z

It depends on what you are doing. If you want to rely on eval then that's not going to work

Tomas Brejla 2021-05-04T16:29:17.199400Z

Well it's not my code which is causing the trouble I guess. I believe those clojure/data/xml/jvm/name.clj:35:1 warnings ^^^ comes from the fact that I'm using one of these libraries:

[com.cognitect.aws/api "0.8.505"]
[com.cognitect.aws/endpoints "1.1.11.1001"]
[com.cognitect.aws/s3 "811.2.889.0"]
[com.cognitect.aws/iam "811.2.889.0"]
[com.cognitect.aws/sts "811.2.889.0"]                 
[throttler "1.0.0"]
[org.clojure/tools.cli "1.0.206"] 
I think it's coming from those cognitext.aws which internally use data.xml, which might be causing the native compilation issue.

borkdude 2021-05-04T16:29:51.199800Z

clojure.data.xml works quite well with graalvm, I'm also using it in babashka

1馃憤
borkdude 2021-05-04T16:30:12.200400Z

it's actually the aws library itself which is problematic since it uses quite a lot of dynamics

borkdude 2021-05-04T16:30:42.200900Z

but we (me and some others) have managed to "natify" that as well: https://github.com/babashka/pod-babashka-aws

1鉂わ笍
Tomas Brejla 2021-05-04T16:31:46.202600Z

Sounds like I might even switch to babashka for this utility I'm building then 馃槃

borkdude 2021-05-04T16:33:09.203800Z

Here are some hints to what is important to get it working with graalvm: https://github.com/BrunoBonacci/graalvm-clojure/tree/master/aws-api-s3

Tomas Brejla 2021-05-04T16:33:20.204100Z

oooh,.... reading the README.md : 鈥 Credentials: custom flows are supported, but not by extending CredentialsProvider interface. See聽https://github.com/babashka/pod-babashka-aws#credentials聽for options. This might be the issue, I'm actually using (reify credentials/CredentialsProvider that's very likely the problem.

borkdude 2021-05-04T16:33:58.204900Z

That note only applies to the babashka aws pod, not to using aws-api as a lib in graalvm native

1馃憦
Karol W贸jcik 2021-05-04T16:44:57.209300Z

@brdloush You can automate generation of native configuration. For instance holy-lambda has a in-agent macro which executes some code only in agent context. https://www.graalvm.org/reference-manual/native-image/BuildConfiguration/#assisted-configuration-of-native-image-builds https://github.com/FieryCod/holy-lambda/blob/master/src/fierycod/holy_lambda/agent.clj#L21 All you need is: 1. Add some in-context calls which utilizes aws-api 2. Uberjar application 3. Generate all configs which should be passed to native-image

java -agentlib:native-image-agent=config-output-dir=resources/native-configuration \
			           -Dexecutor=native-agent \
			           -jar target/output.jar

Tomas Brejla 2021-05-04T16:53:57.211300Z

Thanks a lot to both of you guys. Right now I'll continue adding features to the uberjar-only version that needs the jvm, but later I'll definitely give this a try. I was just curious whether I'll be able to easily native-compile that app, but it seems that it needs some non-trivial effort at the moment.