Discuss GraalVM related topics. Use clojure 1.10.2 or newer for all new projects. Contribute to and GraalVM slack:
Nico 2020-02-21T13:51:06.077700Z

I'm trying to build my project ( with graalvm and it just runs for a long time (like, half an hour) then runs out of memory

borkdude 2020-02-21T13:52:10.078500Z

@nihilazo Instead of starting with a big project, it's often better to start with smaller parts and then add things bit by bit to see what's causing this

borkdude 2020-02-21T13:52:43.079200Z

for reference, all my own graal projects compile within 2 minutes on CircleCI. if it takes longer, I find that suspicious

Nico 2020-02-21T14:22:48.079500Z

hmm, I'll try compiling a hello world project and see if that works

Nico 2020-02-21T14:22:59.079900Z

should I be using the jdk8 or jdk11 version of graalvm for clojure?

borkdude 2020-02-21T14:24:29.080500Z

jvm11 works, but I've ran into some issues with it (unlikely that you will encounter them with hello world)

borkdude 2020-02-21T14:24:38.080800Z

I'm still using jdk8

borkdude 2020-02-21T14:25:46.081300Z

@nihilazo Are you trying to graalvm compile datahike? That's slightly more advanced than hello world I would say?

borkdude 2020-02-21T14:25:56.081600Z

last time I tried that, it didn't work for me

borkdude 2020-02-21T14:27:44.082Z

oh sorry, I misread. yeah, hello world should work

Nico 2020-02-21T14:28:14.082800Z

yeah, I was trying to graalvm compile datahike

Nico 2020-02-21T14:28:34.083200Z

which is probably a bad idea

Nico 2020-02-21T14:28:39.083500Z

I'll see if the hello world works

borkdude 2020-02-21T14:29:02.084Z

I tried that before, but that didn't work for me. that's why I ended up using transit instead of datahike in clj-kondo (I wasn't even able to make a performance benchmark because I never got that far)

borkdude 2020-02-21T14:29:53.084700Z

if you want to have a fast CLI solution with a database you could consider babashka + the sqlite-cli:

borkdude 2020-02-21T14:30:16.085400Z

I wasn't able to compile sqlite itself with graalvm either

Nico 2020-02-21T14:31:24.086600Z

I'm not sure I even really need a database? The queries are super simple (literally just checking if something has a tag, and checking if it doesn't) but there is potentially a reasonable amount of things being checked

Nico 2020-02-21T14:31:40.087100Z

actually, that said, I wonder if datahike is actually worse performance-wise than doing something like that manually

borkdude 2020-02-21T14:32:10.087700Z

@nihilazo What I did for clj-kondo is split the data up into partitions so I can choose to only read from disk what I need. you can tailor this to your problem to make it performant

borkdude 2020-02-21T14:32:40.088200Z

transit is compact and fast to read/write

borkdude 2020-02-21T14:32:48.088500Z

and works with graalvm

Nico 2020-02-21T14:33:13.089100Z

ah, for this in every case I want to search all the data, and it's stored in a text format on disk already, I might try doing it without any DB at all and see how it goes

Nico 2020-02-21T14:33:29.089500Z

might even be able to do everything I was doing with datahike using a couple of filter s

Nico 2020-02-21T14:33:47.090200Z

but using a database seemed like the more "proper" way

borkdude 2020-02-21T14:34:02.090500Z

@nihilazo JUXT published a post today where they plowed through some text files with babashka:

borkdude 2020-02-21T14:34:45.091Z

if you can make the search faster by pre-organizing it into an indexed way first, that can help

Nico 2020-02-21T15:31:34.092600Z

compiling the new datahike-less version results in some unsupported things, but that's progress!

Nico 2020-02-21T15:31:39.092800Z

also, the hello world works perfectly

Nico 2020-02-21T15:36:31.093400Z

I'm getting the errors from clojure.spec.gen.alpha, is it workaround time?

borkdude 2020-02-21T15:37:04.093800Z

@nihilazo Are you loading clojure.spec yourself from anywhere?

borkdude 2020-02-21T15:37:35.094500Z

is it the CLJ-1472 error?

Nico 2020-02-21T15:38:11.095400Z

maybe, I'm not pulling in spec myself but I am pulling in tools.cli

Nico 2020-02-21T15:38:21.095900Z

actually, I don't need tools.cli, I can just remove that

borkdude 2020-02-21T15:38:26.096100Z

tools.cli should work without this error, I've used it in a couple of projects

Nico 2020-02-21T15:38:48.096600Z

the only thing I'm using outside of my own code is and clojure.string

borkdude 2020-02-21T15:39:00.096900Z

your own code could be triggering it. are you using things like resolve?

borkdude 2020-02-21T15:39:32.097300Z

also, it could be that you have to set the macro spec checking off

borkdude 2020-02-21T15:40:00.097700Z

check out the options that I'm using here:

Nico 2020-02-21T15:41:05.098600Z

ah, maybe I need the -J-Dclojure.spec.skip-macros=true line

borkdude 2020-02-21T15:41:49.098800Z

could very well be

Nico 2020-02-21T15:43:04.099200Z

hmm, with that I'm getting errors due to spec as well, despite not using spec at all

borkdude 2020-02-21T15:43:41.099500Z

btw, using without a require is bad practice:

Nico 2020-02-21T15:47:28.100300Z

there's probably bad practice all over this thing tbh, I'm fairly new to clojure

borkdude 2020-02-21T15:48:34.101Z

I don't mean it to be offending, the reason it's "bad" is that you should not trust other tools to load for you. If you use a linter like clj-kondo, it will warn you about such practices.

Nico 2020-02-21T15:52:44.101800Z

no offence taken, just warning you that most of it is probably bad

Nico 2020-02-21T15:52:53.102100Z

and thanks for trying to help me figure this out

Nico 2020-02-21T16:12:14.102500Z

might be worth building the patched spec anyway?

borkdude 2020-02-21T16:13:41.103Z

I'm not sure. your project doesn't seem to be doing anything weird. does it work with clojure 1.9?

Nico 2020-02-21T16:21:09.103300Z

seems like it

Nico 2020-02-21T16:21:17.103600Z

just tested with 1.9 and it works fine

Nico 2020-02-21T16:24:09.104Z

when I try a graal build with 1.9, I get a different error, Error: No instances of <|>.FilePermission are allowed in the image heap as this class should be initialized at image runtime.

borkdude 2020-02-21T16:28:34.104500Z

oh are you by any chance using graal v20?

borkdude 2020-02-21T16:28:57.105Z

try 19.3.1 and report this error on the github graal project. @sfyire also got it

Nico 2020-02-21T16:30:48.105500Z

I'm using whatever the graalvm in AUR is, I'll try 19.3.1

borkdude 2020-02-21T16:31:35.106Z

yeah, 20 just got out and has a couple of issues. But reporting those issues definitely helps and they will be fixed.

borkdude 2020-02-21T16:31:52.106400Z

19.3.1 is the LTS release

Nico 2020-02-21T16:39:15.106900Z


Nico 2020-02-21T16:39:28.107200Z

gotta love arch linux for having the latest everything, even when the latest doesn't work

Nico 2020-02-21T16:45:42.108400Z

can graalvm not access environment variables? It builds perfectly with the older graalvm and older clojure but I use System/getenv and that doesn't seem to be working, seems to fail

Nico 2020-02-21T16:45:54.108600Z

won't read in the variable

borkdude 2020-02-21T17:01:27.109400Z

@nihilazo It depends if you're doing that at compile time or run time. When you execute that call at compile time, the call is made too early

borkdude 2020-02-21T17:02:38.110200Z

even bb can do this:

$ bb '(System/getenv "PWD")'
so surely your custom graalvm clojure app can.

borkdude 2020-02-21T17:03:39.110700Z

yeah, so the problem here is that you're doing it top-level:

borkdude 2020-02-21T17:03:58.111200Z

that won't work, since that captures the value of that environment variable when you compile it, not when you run it

Nico 2020-02-21T17:04:38.111600Z

that doesn't seem like the case when I run it though

Nico 2020-02-21T17:04:48.111900Z

when I run it from the jar, it reacts to whatever the variable is at that point

Nico 2020-02-21T17:05:03.112300Z

I can change it between runs and it changes

borkdude 2020-02-21T17:05:14.112600Z

just make it into a variable inside -main

borkdude 2020-02-21T17:06:21.113400Z

I wonder, is your jar AOT-ed?

Nico 2020-02-21T17:06:55.113700Z

everything is AOT-ed

borkdude 2020-02-21T17:07:10.114Z

then I would even expect that not to work on the JVM

borkdude 2020-02-21T17:09:18.114800Z

anyway, this is a problem I've had before and the solution is to make reading the environment variable a call at runtime, i.e. in the execution of -main, not top-level/before it.

Nico 2020-02-21T17:09:36.115300Z

ok, thanks

Nico 2020-02-21T17:14:35.116100Z

everything is now working on graalVM! Thank you for helping me out and answering my stupid questions

borkdude 2020-02-21T17:14:45.116300Z

they're not stupid and you're welcome!

borkdude 2020-02-21T17:17:32.116700Z

if you can make a tiny repro of the: > No instances of are allowed in the image heap as this class should be initialized at image runtime that would be helpful for fixing it in v20

borkdude 2020-02-21T17:19:08.117200Z

usually posting your AOT-ed uberjar + native-image build instructions is sufficient for the graalvm devs to reproduce it

borkdude 2020-02-21T17:19:18.117500Z

so they won't have to bother with compiling clojure

Nico 2020-02-21T17:21:24.118100Z

I'll try that, not sure exactly what the cause was though

borkdude 2020-02-21T17:21:44.118400Z

wasn't it just v20 jdk8 versus v19 jdk8?

Nico 2020-02-21T17:24:08.118800Z

it was v20 jdk11 vs v19 jdk11

Nico 2020-02-21T17:24:25.119300Z

it worked in v19 and not v20

Nico 2020-02-21T17:24:37.119600Z

but I'm not sure what the thing in the code was that caused it so I can do a tiny repro

borkdude 2020-02-21T17:26:02.120600Z

well, that's up for them to find out I think. just saying: this works with graalvm 19.1.3 jdk 11 on macOS/linux/Windows but not on v20 .... is sufficient, provided you give them the uberjar + build arguments of native-image

borkdude 2020-02-21T17:26:48.120900Z

the repo is here:

borkdude 2020-02-21T17:30:16.121300Z

@nihilazo Here's an example:

Nico 2020-02-21T17:34:19.121800Z

ah, ok

Nico 2020-02-21T17:34:26.122100Z

that makes things easier

Nico 2020-02-21T17:34:27.122200Z