Clojurians Log v2

Hello. Does anyone have any tips on how to speed up my native-image build time

@alekcz360 Can you tell more about your current project? What kind of code / dependencies, what arguments do you pass to native-image? Etc?

:dependencies [ [org.clojure/clojure "1.9.0"]
                  [org.clojure/tools.cli "1.0.194"]
                  [org.martinklepsch/clj-http-lite "0.4.3"]
                  [compojure "1.6.1"]
                  [borkdude/sci "0.0.13-alpha.12"]
                  [http-kit "2.3.0"]
                  ;scgi
                  [com.climate/claypoole "1.1.4"]
                  ;includes for hosted environemnt
                  [cheshire "5.9.0"]
                  [de.ubercode.clostache/clostache "1.4.0"]
                  [hiccup "1.0.5"]
                  [seancorfield/next.jdbc "1.0.409"]
                  [org.postgresql/postgresql "42.2.11"]
                  [honeysql "0.9.10"]
                  [org.martinklepsch/clj-http-lite "0.4.3"]
                ]

:native-image {:name     "pcp"
                 :jvm-opts ["-Dclojure.compiler.direct-linking=true"]
                 :opts     ["--enable-url-protocols=http,https"
                            "--report-unsupported-elements-at-runtime"
                            "--no-fallback"
                            "--initialize-at-build-time"
                            "--allow-incomplete-classpath"
                            "--initialize-at-run-time=org.postgresql.sspi.SSPIClient"
                            "--enable-all-security-services"
                            "--no-server"
                            "-H:ReflectionConfigurationFiles=reflect-config.json"]}

I'm aiming to have an alternative to PHP-FPM that run sci scripts

Sometimes GraalVM takes a lot longer when it has just barely enough memory. Do you use an -Xmx setting?

I gave up on Netty and handwrote my TCP connections. Netty made the build very long.

Yeah locally -Xmx is acceptable. It's set to 12g 🙈

That doesn't work on github actions. Haha. The runners have 7GB ram

Yeah, I'm also hitting the limit on CircleCI a lot lately (4gb, now migrated to Github actions...)

Avoiding reflection and dependencies is usually also easier on GraalVM. This is a project which also uses postgres with Graal native: https://github.com/leafclick/pgmig

I don't have any specific recommendations beyond this. Maybe try bisecting a bit, to see what causes 📈

Thanks @borkdude I'll report back if I make any progress

Sounds like an exciting project btw. I think @jeroenvandijk also had a similar idea with sci

Thanks @borkdude. @alekcz360 I’ve created a poc for a php like interpreter with Sci here https://gist.github.com/jeroenvandijk/fb4cafb5022cead538cde1c7c6e39c4d Not sure if I left it in a finished state. I’m still working on this idea

@borkdude @jeroenvandijk my repo is here https://github.com/alekcz/pcp. It's still a mess Seems to work well. I just need to get the pipeline working so I can test in an actual ubuntu server.

@jeroenvandijk ooh that looks really cool. It goes a level deeper than mine. I'll keep an eye on that gist.

@alekcz360 I only wrote the php parsing, it doesn’t serve anything yet (as you can see I think). I was thinking it could make Clojure as accessible as php. Your work is already more useful as it actually serves something 🙂 What I like about php is partially it’s accessibility (just edit an html-like file), but mostly the upload and refresh feature. Sci makes this possible for Clojure. This is something i’m exploring further

to make it more light weight maybe you can use nginx which fires off the executable and outputs the result, so you can keep the webserver out of the executable

but since you're going to need a server anyway, why not use the JVM?

For production usage the JVM might give better performance. For local development having a server that quickly starts has some benefits in the user experience I think (especially for non-clojure devs). But then it should not depend on nginx maybe

@borkdude yip. The webserver is out for now. The aim is to let people use the same tool to test locally.

Turns out the culprit was hiccup. Once I stripped it out. It was basically christmas.

huh... I didn't expect hiccup to be the problem. I wonder how bootleg deals with hiccup: it's also a graalvm project: https://github.com/retrogradeorbit/bootleg

I never tested the hiccup library myself with graalvm, it also doesn't appear in this list: https://github.com/lread/clj-graal-docs/blob/master/doc/external-resources.md#libraries-compatible-with-graalvm

hmm, hiccup uses eval: https://github.com/weavejester/hiccup/blob/8021043ae0eb64d1a31d87c0fa6071f44bc1c7f3/src/hiccup/compiler.clj#L197 that doesn't work with graalvm

this 15 line hiccup function worked for a small project: https://github.com/borkdude/babashka/blob/270b0acd71173eed1bc1f5f4c445bb50ebc4fffc/examples/notes.clj#L22

maybe @retrogradeorbit can chip in tomorrow (he's probably asleep now) on how he did hiccup for bootleg

hmm, this article does mention hiccup with graalvm: https://www.innoq.com/en/blog/native-clojure-and-graalvm/

this one also has it: https://github.com/yogthos/graal-web-app-example/blob/master/project.clj