babashka

https://github.com/babashka/babashka. Also see #sci, #nbb and #babashka-circleci-builds .
kokada 2021-05-21T02:24:03.084200Z

Well, it is really strange:

diff --git a/src-java/babashka/impl/Graal.java b/src-java/babashka/impl/Graal.java
index 9754185..604b5f6 100644
--- a/src-java/babashka/impl/Graal.java
+++ b/src-java/babashka/impl/Graal.java
@@ -18,23 +18,22 @@ public class Graal {
 
         @Override
         public List<String> getHeaderFiles() {
-            return Collections.singletonList("<stdlib.h>");
+            return Collections.singletonList("<stdio.h>");
         }
     }
 
     @CFunction
-    private static native int setenv(CCharPointer name, CCharPointer value, int overwrite);
+    private static native int puts(CCharPointer name);
 
     // API
     public static int setEnv(String name, String value) {
-        int ret = 0;
+        int ret = -1;
         System.out.println("setenv" + " " + name + " " + value);
         try (CCharPointerHolder nameHolder = CTypeConversion.toCString(name);
              CCharPointerHolder valueHolder = CTypeConversion.toCString(value)) {
-            ret = setenv(nameHolder.get(), valueHolder.get(), 1);
-            System.out.println(System.getenv(name));
+            ret = puts(nameHolder.get());
+            System.out.println(Integer.toString(ret));
         }
-        System.out.println(System.getenv(name));
         return ret;
     }
user=> (babashka.utils/set-env "this is the text that should be printed" "")
:setting "this is the text that should be printed" ""
setenv this is the text that should be printed
this is the text that should be printed
40
40
nil
Definitively seems to be working, but the setenv does seems to do its job but it doesn't actually set the variable :thinking_face:

borkdude 2021-05-21T09:19:05.084500Z

@thiagokokada

borkdude 2021-05-21T10:06:59.084900Z

In the latest commit on the set-env branch this now works:

$ ./bb -e "(require '[babashka.os :as os])" -e '(prn (os/set-env "FOO" "BAR")) (prn (os/get-env "FOO"))'
true
"BAR"

borkdude 2021-05-21T10:09:46.085600Z

So, changing environment variables in Java isn't supported. But this now works in an experimental branch (called set-env👿

$ ./bb -e "(require '[babashka.os :as os])" -e '(prn (os/set-env "FOO" "BAR")) (prn (os/get-env "FOO"))'
true
"BAR"
Yet, it doesn't compose well with System/getenv, since the Java implementation caches the environment variables on the first call to such methods

2021-05-21T10:31:22.085700Z

Nice! FOO should also be set outside of bb, right? So echo $FOO returns "BAR"?

borkdude 2021-05-21T10:34:41.085900Z

I don't think that is how it works

borkdude 2021-05-21T10:34:54.086100Z

a process cannot set the environment of the parent process I think

2021-05-21T10:48:10.086300Z

Ah ok so it would be useful for processes relying on env vars invoked from within the babashka process, is that right?

borkdude 2021-05-21T10:51:20.086500Z

yeah, although I don't know if processes invoked from babashka using processbuilder will even see the effect, since Java has its own way of handling this

borkdude 2021-05-21T10:52:23.086700Z

ah good, that seems to work:

./bb -e "(require '[babashka.os :as os])" -e '(prn (os/set-env "dude" "1337")) @(babashka.process/process ["env"] {:inherit true})'

borkdude 2021-05-21T10:52:44.086900Z

but you can already set those env vars as an arg to babashka.process so for that use case it doesn't really matter

2021-05-21T11:19:00.087100Z

Maybe for nested process calls that you dont control :thinking_face:

kokada 2021-05-21T13:26:56.087300Z

Well, this cache actually makes sense since there is no way to set environment variables in JDK

kokada 2021-05-21T13:27:01.087500Z

Good to know that it works

borkdude 2021-05-21T13:27:25.087700Z

I discovered also a Windows edge case. They don't support setenv, only _putenv ;)

borkdude 2021-05-21T13:27:41.087900Z

which looks like _putenv("FOO=bar");

borkdude 2021-05-21T13:27:50.088100Z

no problem, can work around it ;)

kokada 2021-05-21T13:27:58.088300Z

@borkdude Could we also have unsetenv wrapper?

borkdude 2021-05-21T13:28:01.088500Z

but all in all it may not be such a good idea to include this, although it works

borkdude 2021-05-21T13:28:42.088700Z

@thiagokokada sure, we can support it if it works with GraalVM. But why would we do so?

kokada 2021-05-21T13:31:20.088900Z

For my usecase, I would like to set environment variables for the current process since I use them for some things While I could use another mechanism to have this "state" (like an atom), the subprocess also needs to inherit those environment variables So to use atom+`babashka.process`, I would probably need to always inject this environment variables to every subprocess (not exactly bad, but this gets old quickly)

borkdude 2021-05-21T13:32:09.089100Z

@thiagokokada we could support this, but it can be confusing when people also use System/getenv

borkdude 2021-05-21T13:32:29.089300Z

so as a rule: when you use os/set-env you should also use os/get-env

kokada 2021-05-21T13:32:42.089500Z

We could document this no :thinking_face: ?

borkdude 2021-05-21T13:32:51.089700Z

but if a library already uses System/getenv it doesn't work as expected for them

borkdude 2021-05-21T13:33:29.089900Z

e.g. (os/set-env "JAVA_CMD" "foo/java") followed by (clojure "something") would not work, since clojure uses System/getenv

kokada 2021-05-21T13:33:33.090100Z

Huh... Good point

borkdude 2021-05-21T13:35:15.090400Z

In bb tasks you could accomplish setting an env variable with (shell {:extra-env {"FOO" "BAR"}} "bb some-other-task")

borkdude 2021-05-21T13:35:44.090700Z

just invoke bb another time

kokada 2021-05-21T13:39:45.090900Z

Huh... You gave me a good idea

kokada 2021-05-21T13:39:51.091100Z

To have a task that starts the program

kokada 2021-05-21T13:40:01.091300Z

With the correct environment variables set

kokada 2021-05-21T14:09:01.091500Z

Well, I think it will not work for this case

kokada 2021-05-21T14:09:17.091700Z

Since I need to parse the filename from the current script too

borkdude 2021-05-21T16:13:16.091900Z

@thiagokokada Can you explain what you mean by this?

kokada 2021-05-21T16:15:35.092100Z

Well, it is actually complicated We have a binary (let's call foo) that is symlinked multiple times including the country info So foo-br is "foo for Brazil", foo-mx is "foo for Mexico" So I have to get the current script filename, parse it and set the environment variable (`FOO_COUNTRY=br` for foo-br)

kokada 2021-05-21T16:15:53.092300Z

Nowadays we use the (in)famous #_shell code here hack

kokada 2021-05-21T16:17:12.092600Z

There is some alternatives, I could create a separate file that calls the main one and just set this environment variable

kokada 2021-05-21T16:18:23.092800Z

But after analyzing the code, the actual code works well enough and I don't think it would have any advantage of changing this now

kokada 2021-05-21T16:18:39.093Z

Just reducing the usage of shell, but it is already small enough

kokada 2021-05-21T16:19:26.093200Z

(BTW, the most usage of the shell in this case is actually another shell script that I use to bootstrap babashka if not installed, this script can't be written in Babashka anyway since I would have a classic chicken-egg problem)

borkdude 2021-05-21T16:20:40.093500Z

So is there any change you want or is it fine like it is?

kokada 2021-05-21T16:24:09.093700Z

IMO, I would like to have something like babashka.os/set-env, but only if we could make it work with System/getenv, otherwise I think the drawbacks are worse than the benefits

borkdude 2021-05-21T16:24:44.093900Z

we could patch System/getenv but this would not work with calls that don't go through the interpreter

borkdude 2021-05-21T16:25:09.094200Z

so my initial hunch is not to hack it in, unless we see clear benefits

kokada 2021-05-21T16:26:04.094400Z

I think we can leave as is, if I (or someone else) find some case that it can't be workaround without changing the current env we could add with all the warnings and bells

kokada 2021-05-21T16:27:34.094600Z

(I feel like that I may hit this issue in the future again, but I also feel that this would be a good time to rethink the way do some things)

kokada 2021-05-21T16:43:20.094800Z

@borkdude Sorry about all the inconvenience about this, at least we now know that this is possible (with caveats)

borkdude 2021-05-21T17:26:45.095Z

@thiagokokada Absolutely no problem, I just started investigating to see if it was possible at all. And now I learned a bunch about C interop ;)

1🙌
kokada 2021-05-21T17:27:48.095300Z

Yeah, I found it fantastic that it is relatively easy to do C interop in GraalVM, and also the binary is still static (at least when using C stdlib)

borkdude 2021-05-21T17:28:38.095600Z

yeah, I learned that when you do JNI the binary is no longer static

borkdude 2021-05-21T17:29:06.095800Z

this is a benefit of using the C interop by GraalVM, that musl binaries can still be fully static

1👍
borkdude 2021-05-21T20:02:40.097300Z

The talk I did back in February at the Graal 2021 workshop at CGO is finally available: Babashka: a native Clojure interpreter for scripting https://youtu.be/Yjeh57eE9rg

1