graalvm

Discuss GraalVM related topics. Use clojure 1.10.2 or newer for all new projects. Contribute to https://github.com/clj-easy/graal-docs and https://github.com/BrunoBonacci/graalvm-clojure. GraalVM slack: https://www.graalvm.org/slack-invitation/.
alekcz 2020-03-31T15:27:31.013900Z

Hello. Does anyone have any tips on how to speed up my native-image build time

borkdude 2020-03-31T15:48:05.014800Z

@alekcz360 Can you tell more about your current project? What kind of code / dependencies, what arguments do you pass to native-image? Etc?

alekcz 2020-03-31T15:53:24.015800Z

:dependencies [ [org.clojure/clojure "1.9.0"]
                  [org.clojure/tools.cli "1.0.194"]
                  [org.martinklepsch/clj-http-lite "0.4.3"]
                  [compojure "1.6.1"]
                  [borkdude/sci "0.0.13-alpha.12"]
                  [http-kit "2.3.0"]
                  ;scgi
                  [com.climate/claypoole "1.1.4"]
                  ;includes for hosted environemnt
                  [cheshire "5.9.0"]
                  [de.ubercode.clostache/clostache "1.4.0"]
                  [hiccup "1.0.5"]
                  [seancorfield/next.jdbc "1.0.409"]
                  [org.postgresql/postgresql "42.2.11"]
                  [honeysql "0.9.10"]
                  [org.martinklepsch/clj-http-lite "0.4.3"]
                ]

:native-image {:name     "pcp"
                 :jvm-opts ["-Dclojure.compiler.direct-linking=true"]
                 :opts     ["--enable-url-protocols=http,https"
                            "--report-unsupported-elements-at-runtime"
                            "--no-fallback"
                            "--initialize-at-build-time"
                            "--allow-incomplete-classpath"
                            "--initialize-at-run-time=org.postgresql.sspi.SSPIClient"
                            "--enable-all-security-services"
                            "--no-server"
                            "-H:ReflectionConfigurationFiles=reflect-config.json"]}

alekcz 2020-03-31T15:54:04.016800Z

I'm aiming to have an alternative to PHP-FPM that run sci scripts

borkdude 2020-03-31T15:54:11.017Z

Sometimes GraalVM takes a lot longer when it has just barely enough memory. Do you use an -Xmx setting?

alekcz 2020-03-31T15:54:53.017800Z

I gave up on Netty and handwrote my TCP connections. Netty made the build very long.

alekcz 2020-03-31T15:55:16.018300Z

Yeah locally -Xmx is acceptable. It's set to 12g ๐Ÿ™ˆ

alekcz 2020-03-31T15:55:37.018900Z

That doesn't work on github actions. Haha. The runners have 7GB ram

borkdude 2020-03-31T15:56:11.019300Z

Yeah, I'm also hitting the limit on CircleCI a lot lately (4gb, now migrated to Github actions...)

borkdude 2020-03-31T15:57:26.020300Z

Avoiding reflection and dependencies is usually also easier on GraalVM. This is a project which also uses postgres with Graal native: https://github.com/leafclick/pgmig

borkdude 2020-03-31T15:57:56.021100Z

I don't have any specific recommendations beyond this. Maybe try bisecting a bit, to see what causes ๐Ÿ“ˆ

alekcz 2020-03-31T15:58:53.021600Z

Thanks @borkdude I'll report back if I make any progress

borkdude 2020-03-31T15:59:09.021900Z

Sounds like an exciting project btw. I think @jeroenvandijk also had a similar idea with sci

2020-03-31T16:01:23.022900Z

Thanks @borkdude. @alekcz360 Iโ€™ve created a poc for a php like interpreter with Sci here https://gist.github.com/jeroenvandijk/fb4cafb5022cead538cde1c7c6e39c4d Not sure if I left it in a finished state. Iโ€™m still working on this idea

1๐Ÿ‘
alekcz 2020-03-31T16:03:48.024600Z

@borkdude @jeroenvandijk my repo is here https://github.com/alekcz/pcp. It's still a mess Seems to work well. I just need to get the pipeline working so I can test in an actual ubuntu server.

1๐Ÿ‘1โค๏ธ1๐Ÿ’ช
alekcz 2020-03-31T16:05:01.025900Z

@jeroenvandijk ooh that looks really cool. It goes a level deeper than mine. I'll keep an eye on that gist.

2020-03-31T17:53:55.030700Z

@alekcz360 I only wrote the php parsing, it doesnโ€™t serve anything yet (as you can see I think). I was thinking it could make Clojure as accessible as php. Your work is already more useful as it actually serves something ๐Ÿ™‚ What I like about php is partially itโ€™s accessibility (just edit an html-like file), but mostly the upload and refresh feature. Sci makes this possible for Clojure. This is something iโ€™m exploring further

borkdude 2020-03-31T17:55:32.031500Z

to make it more light weight maybe you can use nginx which fires off the executable and outputs the result, so you can keep the webserver out of the executable

borkdude 2020-03-31T17:56:49.032400Z

but since you're going to need a server anyway, why not use the JVM?

2020-03-31T18:11:34.034300Z

For production usage the JVM might give better performance. For local development having a server that quickly starts has some benefits in the user experience I think (especially for non-clojure devs). But then it should not depend on nginx maybe

alekcz 2020-03-31T21:12:48.036Z

@borkdude yip. The webserver is out for now. The aim is to let people use the same tool to test locally.

alekcz 2020-03-31T21:13:44.036800Z

Turns out the culprit was hiccup. Once I stripped it out. It was basically christmas.

borkdude 2020-03-31T21:14:59.037500Z

huh... I didn't expect hiccup to be the problem. I wonder how bootleg deals with hiccup: it's also a graalvm project: https://github.com/retrogradeorbit/bootleg

borkdude 2020-03-31T21:18:24.038200Z

I never tested the hiccup library myself with graalvm, it also doesn't appear in this list: https://github.com/lread/clj-graal-docs/blob/master/doc/external-resources.md#libraries-compatible-with-graalvm

borkdude 2020-03-31T21:20:33.038800Z

hmm, hiccup uses eval: https://github.com/weavejester/hiccup/blob/8021043ae0eb64d1a31d87c0fa6071f44bc1c7f3/src/hiccup/compiler.clj#L197 that doesn't work with graalvm

borkdude 2020-03-31T21:21:30.039600Z

this 15 line hiccup function worked for a small project: https://github.com/borkdude/babashka/blob/270b0acd71173eed1bc1f5f4c445bb50ebc4fffc/examples/notes.clj#L22

borkdude 2020-03-31T21:22:06.040500Z

maybe @retrogradeorbit can chip in tomorrow (he's probably asleep now) on how he did hiccup for bootleg

borkdude 2020-03-31T21:23:19.040800Z

hmm, this article does mention hiccup with graalvm: https://www.innoq.com/en/blog/native-clojure-and-graalvm/

borkdude 2020-03-31T21:24:04.041100Z

this one also has it: https://github.com/yogthos/graal-web-app-example/blob/master/project.clj