graalvm

Discuss GraalVM related topics. Use clojure 1.10.2 or newer for all new projects. Contribute to https://github.com/clj-easy/graal-docs and https://github.com/BrunoBonacci/graalvm-clojure. GraalVM slack: https://www.graalvm.org/slack-invitation/.
plexus 2020-08-25T05:55:28.018Z

anyone know of a way to check at runtime if you're running inside a native image vs on a regular JVM?

plexus 2020-08-25T06:18:26.018600Z

perfect! thanks!

plexus 2020-08-25T08:09:12.019100Z

took a lot of trial and error but I managed to get funnel to daemonize/background itself

1πŸŽ‰
plexus 2020-08-25T08:10:06.019800Z

some gnarly JNI shenanigans... still need to clean this up but might do a write-up later

1πŸ‘€
plexus 2020-08-25T08:14:36.022Z

actually the thing that took the most time was trying to somehow statically include my native code in the final executable, because it's just one tiny function, it seemed so silly to have to distribute a separate .so for that... tried all kinds of linker options and read tons of stuff online until finally decided that it can't be done πŸ™‚ then figured out a great hack around it. Include the .so as a resource, copy it to the filesystem when starting up and then load it from there

2020-08-25T08:32:04.023300Z

sounds pretty nifty! so for linux you have .so, macos .dylib, and windows .dll and choose among them at runtime?

borkdude 2020-08-25T08:33:19.023800Z

@plexus If you are going to do a writeup, might want to also put it in clj-graal-docs

2020-08-25T08:33:49.024200Z

oh yes, that would be nice

borkdude 2020-08-25T08:34:29.024500Z

@plexus The resource hack has already been done before as part of spire, https://github.com/borkdude/clojure-rust-graalvm and some others

borkdude 2020-08-25T08:35:04.025100Z

Spire was the first one that did it I think

2020-08-25T08:36:33.026Z

mmm, lacking .dll πŸ˜›

borkdude 2020-08-25T08:36:55.026800Z

I leave .dll as an exercise to the reader ;)

2020-08-25T08:37:07.027200Z

he he πŸ™‚

borkdude 2020-08-25T08:37:09.027300Z

This is only a POC

2020-08-25T08:37:27.027600Z

a very nice one at that 🎩

borkdude 2020-08-25T08:38:06.028200Z

The background / daemonizing I would be interested to see

borkdude 2020-08-25T08:39:29.028600Z

Here are already some docs about interacting with native libraries: https://github.com/lread/clj-graal-docs#interfacing-with-native-libraries

plexus 2020-08-25T08:39:51.029300Z

yeah haven't gotten to the macos part yet, since I don't have access to a mac.

borkdude 2020-08-25T08:40:21.029800Z

@plexus CircleCI and Github Actions have so that should not be too hard to get around

plexus 2020-08-25T08:41:30.030100Z

I know, but for debugging that's not very helpful

borkdude 2020-08-25T08:42:48.030300Z

CircleCI offers ssh access

borkdude 2020-08-25T08:43:50.030800Z

Not too long ago I didn't have access to Windows so I debugged everything on appveyor. A bit tedious but it works. Having SSH access makes all the difference for getting fast feedback though.

borkdude 2020-08-25T08:49:23.031700Z

You might want to poke @marc-omorain for macOS access if it's not enabled for your open source project yet, if you go with CircleCI

Marc O'Morain 2020-08-25T08:49:50.032Z

πŸ‘‹

plexus 2020-08-25T08:55:28.032800Z

this is the one project where we're using GH actions actually, although generally my intention is to migrate towards GH actions for the other LI projects as well

borkdude 2020-08-25T10:39:32.033400Z

Impressive, so you're using C to daemonize the Graal process

borkdude 2020-08-25T10:40:22.033600Z

what's this for?

if ((chdir("/")) < 0) {
      exit(EXIT_FAILURE);
    }

plexus 2020-08-25T10:42:34.035Z

some code I cargo culted from this example http://jnicookbook.owsiak.org/recipe-no-029/ as far as I understand it changes the working directory of the process to "/", which I'm guessing is so that it becomes independent of the location where it was started, for instance if the directory it was started from gets deleted

plexus 2020-08-25T10:43:10.035600Z

yeah it's really just using JNI so I can do fork + setsid

borkdude 2020-08-25T10:43:19.035800Z

what if the process doesn't have permission to chdir to /?

plexus 2020-08-25T10:43:40.036100Z

then I suppose it fails. When would that be the case though?

borkdude 2020-08-25T10:43:51.036300Z

don't know :)

borkdude 2020-08-25T10:44:05.036600Z

does daemonize/chdir/etc also work via JNI in the JVM?

borkdude 2020-08-25T10:44:27.037200Z

I've been considering some function in babashka to do this, but the JVM itself doesn't support this

plexus 2020-08-25T10:45:42.037600Z

ls -ld /
drwxr-xr-x 21 root root 4096 Jun  1 12:43 /

plexus 2020-08-25T10:45:56.038Z

read+execute permissions globally, I'm guessing that's pretty standard

borkdude 2020-08-25T10:46:37.039300Z

does chdir("/") also work on Windows with the forward slash?

plexus 2020-08-25T10:46:53.039700Z

you could in theory do the same thing on the JVM, but I believe it's not considered safe. It may also not be safe in this case, time will tell

plexus 2020-08-25T10:47:11.040100Z

re. windows, probably not. I'm not doing windows builds at this point.

plexus 2020-08-25T10:48:19.040800Z

https://blog-old.headius.com/2009/05/fork-and-exec-on-jvm-jruby-to-rescue.html > Update: The biggest problem with using fork+exec in this way is that you can't guarantee nothing happens between the fork call and the exec call. If, for example, the JVM decides to GC or move memory around, you can have a fatal crash at the JVM process level. Because of that, I don't recommend using fork + exec via FFI in JRuby, even though it's pretty cool.

borkdude 2020-08-25T10:48:53.041100Z

hmmm

borkdude 2020-08-25T10:49:50.042100Z

I guess you can also just run funnel &?

plexus 2020-08-25T10:50:30.042600Z

yes, but then it's not detached from the parent shell process. If you close the shell it will kill the child

plexus 2020-08-25T10:50:40.042900Z

I think you could do nohup funnel &

borkdude 2020-08-25T10:51:41.043300Z

I guess you can install funnel as a service, systemd kind of thing?

plexus 2020-08-25T10:52:03.043500Z

yes, totally

plexus 2020-08-25T10:53:17.044400Z

the direction I'm going now is that tools like kaocha-cljs2 that need it will try starting it in the background. If there's already one running then it's a no-op.

plexus 2020-08-25T10:55:42.045Z

although at this point I'd like users to install and run it themselves, so they know it exists, since it's a good place to look when troubleshooting

borkdude 2020-08-25T10:59:34.045600Z

@plexus I guess you can also use java.lang.ProcessBuilder to run one in the background, without daemonizing

borkdude 2020-08-25T10:59:59.045900Z

each test run would then have to run its own instance

borkdude 2020-08-25T11:00:12.046200Z

or rely on some pid file or something to check if there's already one

plexus 2020-08-25T11:06:21.046700Z

but then it would only live as long as the test runner process, that defeats the purpose

plexus 2020-08-25T11:06:50.046900Z

> Funnel provides persistence and discoverability. It keeps connections to long lived processes (like a browser), so that short-lived processes (like a test runner) can discover and interact with them . This is why we recommend running Funnel as a separate long-lived process, rather than for instance embedding it into the tool that uses it.

jumar 2020-08-25T11:50:00.049500Z

If you start a subprocess from a Java process the child isn’t terminated when parent exits - that’s a typical behavior in Linux