graalvm

Discuss GraalVM related topics. Use clojure 1.10.2 or newer for all new projects. Contribute to https://github.com/clj-easy/graal-docs and https://github.com/BrunoBonacci/graalvm-clojure. GraalVM slack: https://www.graalvm.org/slack-invitation/.
borkdude 2020-10-01T07:38:45.031500Z

It should have benefit on binary size and runtime speed at least, but in some cases also memory consumption during compilation

sergey 2020-10-01T17:33:54.031700Z

Gotcha. I double checked with a (medium-sized) cli app at my company and didn't notice a significant difference except for maybe a 1-2% runtime speed improvement. I also tried recompiling babashka: the compilation w/out flags used a bit less RAM, and the non-flag binary was 1.5MB smaller (69.MB vs 70.8MB). Not sure if there's a performance difference between the two babashka builds

sergey 2020-10-01T17:35:57.031900Z

Maybe there's a difference in very high-performance scenarios?

sergey 2020-10-01T17:39:44.034100Z

Hey y'all - a while back I posted in here asking about using the cognitect AWS api from native-image. I wanted to share that I figured out how to do it and https://github.com/latacora/native-cogaws that shows how to get s3 read and write calls working

2👍
sergey 2020-10-01T17:44:22.034300Z

The key insights: 1. you need to provide configuration files for Reflection, resource loads, etc. 2. the cognitect aws client should not be instantiated in a global def (put it in a function or wrap it in a delay) 3. you need to require two internal https://github.com/latacora/native-cogaws/blob/master/src/latacora/native_cogaws.clj#L4 in the file that uses the client

sergey 2020-10-01T17:47:21.034600Z

#3 was something that was suggested in this channel, so thanks for the tip (I think it was from @borkdude or @jeroenvandijk?)

alexmiller 2020-10-01T17:49:54.034900Z

cool!

borkdude 2020-10-01T17:56:36.036500Z

GraalVM can't create load classes at runtime, so that might be the issue

ghadi 2020-10-01T17:56:47.036900Z

the cognitect aws-api dynamically loads namespaces based on the target service's protocol

ghadi 2020-10-01T17:56:51.037100Z

s3's protocol is rest-xml

ghadi 2020-10-01T17:57:02.037500Z

sqs's protocol is query

borkdude 2020-10-01T17:57:07.037600Z

How big is the binary that is resulting from the reflection based approach?

borkdude 2020-10-01T17:57:17.037800Z

I figure it would be quite big quite fast?

borkdude 2020-10-01T18:18:58.038700Z

Side note: this is a way to call AWS at native (startup) speeds using babashka: https://github.com/tzzh/pod-tzzh-aws It's a pod leveraging the Go API but the Go code is generated using babashka itself (could also be done using Clojure, detail)

borkdude 2020-10-01T18:21:12.039900Z

I guess it's a similar approach as cognitect AWS API except the code generation as all done beforehand, no reflection stuff

borkdude 2020-10-01T18:22:17.040400Z

(he also published this today: https://github.com/tzzh/pod-tzzh-mail - similar approach leveraging a Go mail library)

borkdude 2020-10-01T18:23:24.040900Z

Babashka pods can also be called from the JVM btw

borkdude 2020-10-01T18:25:05.041900Z

And what memory usage is reported during compilation?

sergey 2020-10-01T18:55:13.042400Z

I've seen the epiccastle post, but wasn't aware of that issue

borkdude 2020-10-01T18:57:06.042600Z

it's not graalvm related per se, just performance related in general

borkdude 2020-10-01T18:57:11.042800Z

the sci issue

sergey 2020-10-01T19:02:00.043Z

compilation memory usage goes up to 7.2GB; the binary size is around 82MB (for native-cogaws)

borkdude 2020-10-01T19:04:50.043300Z

yeah, that's what I was afraid of

borkdude 2020-10-01T19:05:02.043500Z

how long does compilation take?

sergey 2020-10-01T19:05:11.043700Z

200s

borkdude 2020-10-01T19:05:23.043900Z

on Github actions? or ?

sergey 2020-10-01T19:05:35.044100Z

just locally, I haven't tried via GH actions

borkdude 2020-10-01T19:05:52.044300Z

What CPU do you have?

sergey 2020-10-01T19:07:47.044500Z

Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz (from /proc/cpuinfo)

borkdude 2020-10-01T19:08:37.044700Z

ok thanks. This is similar what I got with hato, memory spikes. I think this can be heavily optimized for native.

borkdude 2020-10-01T19:09:02.044900Z

Usually runtime require/find-var/resolve etc spike memory + binary size

sergey 2020-10-01T19:11:28.045100Z

How would you go about optimizing? All I could think of was removing entries from config files, but that seems pretty time consuming/prone to bugs

borkdude 2020-10-01T19:12:40.045300Z

I don't know the cognitect library very well, but I think the approach taken by the pod-tzzh-aws library is more promising: instead of a runtime approach a compile time approach could work better, where all code paths are generated before hand and no reflection / dynamic requires are done at runtime.

sergey 2020-10-01T19:16:46.045500Z

> where all code paths are generated before hand and no reflection / dynamic requires are done at runtime Is this different from what happens when you feed native-image the reflection config file?

borkdude 2020-10-01T19:17:52.045700Z

Not sure, but for what it does (a couple of calls to s3) I think 7-8GB and 80mb is way too much

borkdude 2020-10-01T19:18:39.045900Z

I think @jeroenvandijk also looked into this once, maybe he can tell you more

sergey 2020-10-01T19:20:36.046100Z

yea, seems pretty excessive. We had another cli app (just s3 calls) that went up from 60MB to 100MB after I added the reflection config. I haven't tried on more involved apps, so can't say if more services will mean more CPU/binary size

borkdude 2020-10-01T19:22:14.046300Z

@sergey923 Here's some data for pprint: https://clojure.atlassian.net/browse/CLJ-2582

borkdude 2020-10-01T19:22:46.046500Z

If I was a heavy user of cognitect aws or AWS in general, which I am not at the moment, I would probably rewrite that thing for GraalVM specifically

ghadi 2020-10-01T19:50:43.047200Z

@borkdude aws-api does not do reflection

ghadi 2020-10-01T19:51:11.047700Z

it reads edn descriptor files that describe the service endpoints

borkdude 2020-10-01T19:58:28.049200Z

Ah, thanks. There may be other issues why it's heavy on GraalVM native-image. One issue at a glance: $ ls src/cognitect/aws, I spot that it has a dynaload namespace. If that's similar to what spec does, it already explains something. I made a variant of dynaload with specific GraalVM settings: https://github.com/borkdude/dynaload

ghadi 2020-10-01T19:59:09.050100Z

for AOT, you can preload all the dynaloaded things, like the repo does above

ghadi 2020-10-01T19:59:39.051Z

the set of dynaloaded namespaces is bounded

borkdude 2020-10-01T20:00:19.051500Z

yes, but even then, having code around with find-var, require on the non-top-level (function bodies) in it can still make the image more bloated than necessary. It will work, but it will also be more bloated than necessary. See e.g. https://clojure.atlassian.net/browse/CLJ-2582

borkdude 2020-10-01T20:01:25.052600Z

What I would probably do is fork that lib, get rid of these references and optimize

borkdude 2020-10-01T20:02:04.053300Z

If it doesn't do reflection, it is weird why such a huge reflection config is necessary in the AOT-ed lib to make it work

ghadi 2020-10-01T20:04:50.054800Z

I don't know, I haven't analyzed it. Speaking with my maintainer hat on, aws-api does not explicitly do anything "reflective". It just delays the loading of a few namespaces.

ghadi 2020-10-01T20:05:02.055100Z

needs core.async, xml, json libraries, + jetty

borkdude 2020-10-01T20:05:40.055800Z

Maybe it would already help if this part was avoided:

src/cognitect/aws/dynaload.clj
14:    (or (resolve s)

ghadi 2020-10-01T20:05:44.055900Z

reflect-config.json seems to include cheshire, which we don't use

borkdude 2020-10-01T20:06:14.056200Z

yeah, I saw that, seems weird

ghadi 2020-10-01T20:07:11.057200Z

the only truly dynamic thing we plan on doing is having a pluggable http client

ghadi 2020-10-01T20:07:46.057500Z

right now it's hardcoded to jetty

borkdude 2020-10-01T20:08:04.057800Z

is it?

(defn resolve-http-client
  [http-client-or-sym]
  (let [c (or (when (symbol? http-client-or-sym)
                (let [ctor @(dynaload/load-var http-client-or-sym)]
                  (ctor)))
              http-client-or-sym
              (let [ctor @(dynaload/load-var (configured-client))]
                (ctor)))]
    (when-not (client? c)
      (throw (ex-info "not an http client" {:provided http-client-or-sym
                                            :resolved c})))
    c))

ghadi 2020-10-01T20:08:46.058800Z

it is -- we don't expose this functionality yet

borkdude 2020-10-01T20:09:08.059400Z

anyway, these are the kinds of spots that need attention when dealing with GraalVM native-image probably. With pprint there was only one or two lines that needed changing and boom, 20mb less binary size

borkdude 2020-10-01T20:09:36.059800Z

alter-var-root can help patching these things without even touching the code

borkdude 2020-10-01T20:10:22.060100Z

(my alter-var-root patch for pprint: https://github.com/borkdude/babashka/blob/master/src/babashka/impl/pprint.clj#L8-L49)

ghadi 2020-10-01T20:14:42.062200Z

I am more interested in figuring out how to structure truly dynamic code to be graal-sympathetic

ghadi 2020-10-01T20:15:03.062700Z

like how could you compile aws-api with the jetty client vs with the something-else-http client

sergey 2020-10-01T20:16:07.063500Z

that might have been me experimenting with some stuff

ghadi 2020-10-01T20:20:02.064700Z

not dynamic in the produced image, just a higher level front-end API to image generation

ghadi 2020-10-01T20:20:55.065700Z

e.g. here is my program, it includes some multimethods, I want you to preload these namespaces which extend the multimethods, this is my entrypoint

ghadi 2020-10-01T20:22:17.066700Z

like what is the data that is in @sergey923’s repo?

ghadi 2020-10-01T20:23:16.068Z

{:preload-nses [cognitect.aws.protocols.rest-xml 
               cognitect.aws.protocols.query 
               cognitect.aws.protocols.rest-json]
 :entrypoint latacora.foo/main}

ghadi 2020-10-01T20:24:08.068500Z

same stuff with jlink in the jvm

ghadi 2020-10-01T20:24:39.069200Z

you tell it the set of modules, but also any dynamic Services that need to be present

borkdude 2020-10-01T20:38:38.070100Z

@ghadi I haven't looked into it myself but Quarkus is a JVM framework which has loads of modules/extensions that work with GraalVM. It might have some clues as to how to approach what you're interested in

borkdude 2020-10-01T20:42:27.070900Z

@ghadi fwiw, my dynaload variant has a setting for GraalVM: https://github.com/borkdude/dynaload it behaves fully dynamic on the JVM, but less dynamic in CLJS and static in GraalVM native-image (where you're supposed to require the namespaces in a certain order, sure that could be configured using some .edn file)

2020-10-01T20:51:44.071400Z

@sergey923 You might have seen my fork of cognitect's library https://github.com/AdGoji/aws-api It solves the case where you need to do something specific and you want to take that extra effort to have a standalone binary (no python or other dep). For quick ad hoc AWS calls it's not suited, but it takes the signing of requests (hardest part). I'm guessing it would be possible to run the whole library with graalvm if you get rid of some of the lazy loading (in the http client and credential lookup mostly I think). There is also some XML parsing that might bloat the binary more