Discuss GraalVM related topics. Use clojure 1.10.2 or newer for all new projects. Contribute to and GraalVM slack:
tvaughan 2021-01-30T01:13:50.074100Z

Not sure if anyone is interested in yet another example of how to compile a native image, but if so Only docker is required. For anyone looking to reproduce this without docker, Clojure 1.10.2 is required and for some reason this needs to be specified in deps.edn. A recent version of depstar is also required.

ericdallo 2021-01-30T16:30:52.078500Z

ok, nrepl fixed using dynaload ๐Ÿ˜„ I tried to use SLF4j, log4j, log4j2 and logback, but all have issues with file appenders with graalvm... so I think timbre will be the only way I fixed all found issues with reflection, need to test all features still. I noticed that the first clojure-lsp classpath scan time using clj-kondo was increased, for a proect with 250 files to scan, while with JVM it takes 90s with graalvm takes 130s, not sure there is anything to improve that though. About the image size, I removed a lot of unnecessary reflection, still the image size is about 110MB while the jar has 30MB, I found regarding timbre that adds 50MB to the image size but not sure is happening What you suggest to decrease the image size @borkdude?

borkdude 2021-01-30T16:32:45.078800Z

@ericdallo > while with JVM it takes 90s with graalvm takes 130s, not sure there is anything to improve that though. This is why I think the JVM makes most sense for projects like this, I keep recommending this for my own LSP server for clj-kondo

borkdude 2021-01-30T16:33:04.079Z

As for the image size, you can look at the analysis callgraph or use the dashboard options.

ericdallo 2021-01-30T16:36:58.079200Z

I see ๐Ÿ˜” Besides that, everything is pretty much fast, like diagnostics, code actions, refacotings which improve user xp IMO I still need to make the memory test, then we can think in the tradeoff.

ericdallo 2021-01-30T16:37:12.079400Z

How can I check the analysis callgraph?

borkdude 2021-01-30T16:37:50.079600Z


borkdude 2021-01-30T16:38:01.079800Z

this will dump some information about the pulled in classes, methods, etc

ericdallo 2021-01-30T16:38:17.080Z

Nice, I'll try

borkdude 2021-01-30T16:38:23.080200Z

and there is also a dashboard option:

borkdude 2021-01-30T16:38:41.080400Z

I haven't used the dashboard that much, but it seems useful. You can drill down into classes

lread 2021-01-30T18:50:09.082300Z

Just upgraded rewrite-clj v1 branch tests to use Graal v21.0.0. One more datapoint, @borkdude, for the not being needed!

lread 2021-01-30T18:52:46.084300Z

For my Graal sci test under JDK11, I did still need to include borkdude/sci.impl.reflector. Did not dig in at all, did you expect this still to be needed @borkdude?

borkdude 2021-01-30T19:04:22.084700Z

yes, this lib is needed, but already pulled in by sci itself?

ericdallo 2021-01-30T19:20:08.085200Z

probably yes, since I removed from clojure-lsp and still works

lread 2021-01-30T19:45:52.085900Z

Oh sorry, I meant the 0.0.1-java11 override version of borkdude/sci.impl.reflector for JDK11.

borkdude 2021-01-30T19:48:38.086300Z

Right, that one isn't needed anymore, since it has the same MethodHandle code

lread 2021-01-30T19:53:29.086800Z

Cool, thanks, Iโ€™ll retry without it.

ericdallo 2021-01-30T20:11:30.086900Z

@borkdude for some reason the dashboard flag just gives me OoO even with a -JXmx of 6GB :man-shrugging: I did some hacking to run Graalvm 21.0.0 on My NixOS (docker doesn't work) instead of 20.2.0 and the startup time fall from 130s -> 75s... it's faster than JVM classpath scan ๐Ÿ˜ฎ (~87-90s), I will make more tests, special the memomry one (not sure how to measure that yet)

ericdallo 2021-01-30T20:11:59.087100Z

Also, I added the new -H:+InlineBeforeAnalysis flag, not sure this could cause that performance improvement though

borkdude 2021-01-30T20:13:14.087300Z

@ericdallo Be sure to use the --no-fallback option so you won't get a fallback image, this is effectively just a JVM

borkdude 2021-01-30T20:13:41.087500Z

And yeah, the dashboard data may require moar memory

borkdude 2021-01-30T20:13:45.087700Z

it's a lot of data

ericdallo 2021-01-30T20:15:08.087900Z

yeah, I'm using the --no-fallback, I don't want that kind of lie anymore ๐Ÿ˜‚

borkdude 2021-01-30T20:15:43.088100Z

You could also post an issue about a performance regression on the graalvm issue tracker

ericdallo 2021-01-30T20:18:35.089Z

@borkdude sorry, I probably was not clear, I was using Graalvm 20.2 with all my tests because it's the only available for NixOS while this , also the docker image doesn't work with NixOS for some reason related with shared libs, but I managed to manually install the graalvm 21.0.0 in the nix way

ericdallo 2021-01-30T20:19:12.089700Z

so it's not a regression since I went from 20.2.0 -> 21.0 ๐Ÿ˜„

borkdude 2021-01-30T20:19:22.090Z

so 21 is faster right?

borkdude 2021-01-30T20:19:44.090200Z

ok, then it's good!

ericdallo 2021-01-30T20:20:23.090400Z

yep, at least for the only thing I noticed that it was slower than JVM, now it's not anymore ๐Ÿ˜„

lread 2021-01-30T22:18:33.092800Z

If I retry my rewrite-clj sci tests without overriding borkdude/sci.impl.reflector to 0.0.1-java11, native-image compile works fine, but when I interpret my tests I get java.lang.NoSuchMethodError: java.lang.reflect.AccessibleObject.canAccess exceptions. Iโ€™ll create a todo for myself to look at this sometime later.

borkdude 2021-01-30T22:19:18.093300Z

@lee I have encountered this error as well. Adding this method to the reflection config did the trick for me

borkdude 2021-01-30T22:19:48.093600Z

See comment here:

lread 2021-01-30T22:21:02.094600Z

Huh, I thought I had tried that! Worth another shot, thanks!

borkdude 2021-01-30T22:21:09.094800Z

@lee I think this is because the method is looked up using reflection:

borkdude 2021-01-30T22:21:22.095100Z

(or something)

borkdude 2021-01-30T22:21:42.095400Z

so basically it's missed by the analysis and you have to help it a little bit

borkdude 2021-01-30T22:22:52.095800Z

This might become a very common config for projects that hit this path, so probably worth documenting

borkdude 2021-01-30T22:23:31.096300Z

[{"name": "java.lang.reflect.AccessibleObject",
  "methods" : [{"name":"canAccess"}]},

lread 2021-01-30T22:24:05.096600Z

Ya, Iโ€™ll add a note to clj-graal-docs.

borkdude 2021-01-30T22:29:52.097200Z

I think I'll copy your note to the reflector graal-fix repo then as well, since if you ran into that one, you'll probably now need this

lread 2021-01-30T22:32:53.097400Z

ah, good idea!

lread 2021-01-30T22:35:20.099800Z

Note that I did not need it when 1) compiling rewrite-clj and its tests, only when 2) compiling rewrite-clj for sci tests. But I donโ€™t use a reflection.json for 1, so maybe thatโ€™s the diff.

borkdude 2021-01-30T22:36:00.100400Z

yeah, sci uses this Reflector class to do interop

borkdude 2021-01-30T22:36:06.100700Z

so that makes sense

lread 2021-01-30T22:36:14.100900Z

Ah, right.

lread 2021-01-30T22:37:20.102100Z

reflection.json fix you shared above works for rewrite-clj sci tests, thanks!

borkdude 2021-01-30T22:38:17.102300Z


lread 2021-01-30T22:39:53.103100Z

oh poo, I guess Iโ€™ll need to generate a different reflection.json for jdk8 than I do for jdk11.

borkdude 2021-01-30T22:40:28.103600Z

in babashka I generate these using a script

borkdude 2021-01-30T22:40:46.104200Z

different feature flags also need different reflection configs there

lread 2021-01-30T22:40:48.104300Z

yeah, I stole that at one point, maybe I just need to grab an update!

lread 2021-01-30T22:41:39.105Z

ok, good stuff, should not tax my wee brain too much! :simple_smile:

borkdude 2021-01-30T22:43:22.105500Z

there is only so much manual JSON or YAML my brain can handle before I switch to a clojure script