babashka

https://github.com/babashka/babashka. Also see #sci, #nbb and #babashka-circleci-builds .
ghosttoaster 2021-05-20T02:18:29.064700Z

does bb handle aliases? -A:dev and such?

ghosttoaster 2021-05-20T02:34:48.065300Z

I guess the better questions is how do I load fuctions in to the user namespace?

borkdude 2021-05-20T07:32:30.065400Z

@cdimara Can you explain your use case? bb has an :tasks {:init ...} section where you can put startup code when you invoke a task

borkdude 2021-05-20T07:32:50.065600Z

It also has a BABASHKA_PRELOADS environment variable that can contain code that is always executed before anything else

ghosttoaster 2021-05-20T17:28:01.069800Z

I just wanted to preload some helpful functions in the user namespace for this one project.

ghosttoaster 2021-05-20T17:29:12.070Z

Let me try the tasks thing that could be it. I'm assuming BABASHKA_PRELOADS runs on every evocation of bb? And I only need this helper code for this project and only during development.

borkdude 2021-05-20T17:44:07.070200Z

> I'm assuming BABASHKA_PRELOADS runs on every evocation of bb?

borkdude 2021-05-20T17:44:11.070400Z

correct

borkdude 2021-05-20T11:30:08.067400Z

babashka 0.4.2-0.4.3: - Better error handling in parallel tasks - Add more agent functions and fix binding conveyance in send-via - Better transit support for pods (fixes issue with mysql pod and java.time.LocalDateTime results) - Misc. other improvements

πŸ‘ 4
1
5
πŸ‘ 1
littleli 2021-05-20T14:13:11.068100Z

Not many developers realize it, but your little "clojure interpreter" has a support of concurrency such that virtually all popular scripting languages can't compete with at all πŸ‘

7
richiardiandrea 2021-05-20T16:25:08.069300Z

FWIW I tried the static 0.4.1 release and it works like a charm πŸ˜‰

borkdude 2021-05-20T16:33:58.069600Z

A hack that might be coming to bb some day as well ;)

πŸ˜„ 2
😎 1
borkdude 2021-05-21T09:19:05.084500Z

@thiagokokada

borkdude 2021-05-21T10:06:59.084900Z

In the latest commit on the set-env branch this now works:

$ ./bb -e "(require '[babashka.os :as os])" -e '(prn (os/set-env "FOO" "BAR")) (prn (os/get-env "FOO"))'
true
"BAR"

2021-05-21T10:31:22.085700Z

Nice! FOO should also be set outside of bb, right? So echo $FOO returns "BAR"?

borkdude 2021-05-21T10:34:41.085900Z

I don't think that is how it works

borkdude 2021-05-21T10:34:54.086100Z

a process cannot set the environment of the parent process I think

2021-05-21T10:48:10.086300Z

Ah ok so it would be useful for processes relying on env vars invoked from within the babashka process, is that right?

borkdude 2021-05-21T10:51:20.086500Z

yeah, although I don't know if processes invoked from babashka using processbuilder will even see the effect, since Java has its own way of handling this

borkdude 2021-05-21T10:52:23.086700Z

ah good, that seems to work:

./bb -e "(require '[babashka.os :as os])" -e '(prn (os/set-env "dude" "1337")) @(babashka.process/process ["env"] {:inherit true})'

borkdude 2021-05-21T10:52:44.086900Z

but you can already set those env vars as an arg to babashka.process so for that use case it doesn't really matter

2021-05-21T11:19:00.087100Z

Maybe for nested process calls that you dont control :thinking_face:

kokada 2021-05-21T13:26:56.087300Z

Well, this cache actually makes sense since there is no way to set environment variables in JDK

kokada 2021-05-21T13:27:01.087500Z

Good to know that it works

borkdude 2021-05-21T13:27:25.087700Z

I discovered also a Windows edge case. They don't support setenv, only _putenv ;)

borkdude 2021-05-21T13:27:41.087900Z

which looks like _putenv("FOO=bar");

borkdude 2021-05-21T13:27:50.088100Z

no problem, can work around it ;)

kokada 2021-05-21T13:27:58.088300Z

@borkdude Could we also have unsetenv wrapper?

borkdude 2021-05-21T13:28:01.088500Z

but all in all it may not be such a good idea to include this, although it works

borkdude 2021-05-21T13:28:42.088700Z

@thiagokokada sure, we can support it if it works with GraalVM. But why would we do so?

kokada 2021-05-21T13:31:20.088900Z

For my usecase, I would like to set environment variables for the current process since I use them for some things While I could use another mechanism to have this "state" (like an atom), the subprocess also needs to inherit those environment variables So to use atom+`babashka.process`, I would probably need to always inject this environment variables to every subprocess (not exactly bad, but this gets old quickly)

borkdude 2021-05-21T13:32:09.089100Z

@thiagokokada we could support this, but it can be confusing when people also use System/getenv

borkdude 2021-05-21T13:32:29.089300Z

so as a rule: when you use os/set-env you should also use os/get-env

kokada 2021-05-21T13:32:42.089500Z

We could document this no :thinking_face: ?

borkdude 2021-05-21T13:32:51.089700Z

but if a library already uses System/getenv it doesn't work as expected for them

borkdude 2021-05-21T13:33:29.089900Z

e.g. (os/set-env "JAVA_CMD" "foo/java") followed by (clojure "something") would not work, since clojure uses System/getenv

kokada 2021-05-21T13:33:33.090100Z

Huh... Good point

borkdude 2021-05-21T13:35:15.090400Z

In bb tasks you could accomplish setting an env variable with (shell {:extra-env {"FOO" "BAR"}} "bb some-other-task")

borkdude 2021-05-21T13:35:44.090700Z

just invoke bb another time

kokada 2021-05-21T13:39:45.090900Z

Huh... You gave me a good idea

kokada 2021-05-21T13:39:51.091100Z

To have a task that starts the program

kokada 2021-05-21T13:40:01.091300Z

With the correct environment variables set

kokada 2021-05-21T14:09:01.091500Z

Well, I think it will not work for this case

kokada 2021-05-21T14:09:17.091700Z

Since I need to parse the filename from the current script too

borkdude 2021-05-21T16:13:16.091900Z

@thiagokokada Can you explain what you mean by this?

kokada 2021-05-21T16:15:35.092100Z

Well, it is actually complicated We have a binary (let's call foo) that is symlinked multiple times including the country info So foo-br is "foo for Brazil", foo-mx is "foo for Mexico" So I have to get the current script filename, parse it and set the environment variable (`FOO_COUNTRY=br` for foo-br)

kokada 2021-05-21T16:15:53.092300Z

Nowadays we use the (in)famous #_shell code here hack

kokada 2021-05-21T16:17:12.092600Z

There is some alternatives, I could create a separate file that calls the main one and just set this environment variable

kokada 2021-05-21T16:18:23.092800Z

But after analyzing the code, the actual code works well enough and I don't think it would have any advantage of changing this now

kokada 2021-05-21T16:18:39.093Z

Just reducing the usage of shell, but it is already small enough

kokada 2021-05-21T16:19:26.093200Z

(BTW, the most usage of the shell in this case is actually another shell script that I use to bootstrap babashka if not installed, this script can't be written in Babashka anyway since I would have a classic chicken-egg problem)

borkdude 2021-05-21T16:20:40.093500Z

So is there any change you want or is it fine like it is?

kokada 2021-05-21T16:24:09.093700Z

IMO, I would like to have something like babashka.os/set-env, but only if we could make it work with System/getenv, otherwise I think the drawbacks are worse than the benefits

borkdude 2021-05-21T16:24:44.093900Z

we could patch System/getenv but this would not work with calls that don't go through the interpreter

borkdude 2021-05-21T16:25:09.094200Z

so my initial hunch is not to hack it in, unless we see clear benefits

kokada 2021-05-21T16:26:04.094400Z

I think we can leave as is, if I (or someone else) find some case that it can't be workaround without changing the current env we could add with all the warnings and bells

kokada 2021-05-21T16:27:34.094600Z

(I feel like that I may hit this issue in the future again, but I also feel that this would be a good time to rethink the way do some things)

kokada 2021-05-21T16:43:20.094800Z

@borkdude Sorry about all the inconvenience about this, at least we now know that this is possible (with caveats)

borkdude 2021-05-21T17:26:45.095Z

@thiagokokada Absolutely no problem, I just started investigating to see if it was possible at all. And now I learned a bunch about C interop ;)

πŸ™Œ 1
kokada 2021-05-21T17:27:48.095300Z

Yeah, I found it fantastic that it is relatively easy to do C interop in GraalVM, and also the binary is still static (at least when using C stdlib)

borkdude 2021-05-21T17:28:38.095600Z

yeah, I learned that when you do JNI the binary is no longer static

borkdude 2021-05-21T17:29:06.095800Z

this is a benefit of using the C interop by GraalVM, that musl binaries can still be fully static

πŸ‘ 1
kokada 2021-05-20T17:53:36.070800Z

This would be really useful to have, something that I miss from Python in my Babashka scripts

kokada 2021-05-20T17:57:09.071100Z

BTW, could this also be turned in a JVM library (like babashka.fs is)? Asking because one of the nice things of Babashka is that my codebase is 100% compatible with Clojure proper too

kokada 2021-05-20T17:57:28.071300Z

(But if isn't, no problem too, I can probably workaround just this case. The important part is just that I can call the function to set the env, not that it actually works in Clojure)

borkdude 2021-05-20T17:59:44.071600Z

Yes, I want to release this is a library which can be used within babashka, but the hardest part is the name. Perhaps babashka.core? ;)

borkdude 2021-05-20T17:59:56.071800Z

Perhaps some util lib that can be used across more graalvm projects

borkdude 2021-05-20T18:00:02.072Z

but again, the name

borkdude 2021-05-20T18:10:40.072200Z

I have another function that could be in it: "exec"-ing the current process into another one

borkdude 2021-05-20T18:13:41.072400Z

Basically these three things: https://github.com/babashka/babashka/labels/native

borkdude 2021-05-20T18:14:01.072600Z

Perhaps I should call the lib babashka.native

borkdude 2021-05-20T18:14:14.072800Z

or babashka.os

2021-05-20T18:43:47.073Z

Babashka.x πŸ˜…

borkdude 2021-05-20T18:44:31.073200Z

What does the x stand for?

2021-05-20T18:45:13.073500Z

Exec + a var

borkdude 2021-05-20T18:45:49.073700Z

perhaps babashka.svm or babashka.graal or so

2021-05-20T18:45:56.073900Z

There are probably better names indeed

2021-05-20T18:46:02.074100Z

Than X I mean

borkdude 2021-05-20T18:46:21.074300Z

those calls only work in a native image so the name must probably reflect that

2021-05-20T18:47:03.074500Z

So it would be a library specific to native image or even a smaller subset?

borkdude 2021-05-20T18:48:12.074700Z

yes, set-env and exec sould only work in a native image. but those functions could be useful to all graalvm native projects

borkdude 2021-05-20T18:48:34.074900Z

but I could also make a babashka.aux library with all sorts of useful functions

borkdude 2021-05-20T18:48:43.075100Z

e.g. get-env could also be in there

2021-05-20T19:10:02.075300Z

My 2 cents; I don’t recognize the name svm, but maybe other graal people do. aux could be anything I guess. So graal and native sound most descriptive to me.

borkdude 2021-05-20T19:11:05.075500Z

ok thanks

kokada 2021-05-20T21:21:32.075700Z

I would suggest to follow the Python naming suggestions here, so babashka.os seems ok to me

kokada 2021-05-20T21:22:24.075900Z

babashka.graalvm.os doesn't seem bad too, or some variation of it

kokada 2021-05-20T21:22:53.076100Z

(But not sure about native, this may be confusing depending of the context)

borkdude 2021-05-20T21:23:19.076300Z

yeah, I like babaska.os as well, but we also have babashka.process

kokada 2021-05-20T21:25:29.076500Z

For me this split makes sense, for the same reason Python has subprocess and os

borkdude 2021-05-20T21:25:29.076700Z

but it would be confusing maybe to have get-env in babashka.process since process is about spawning new processes and os is about the current process perhaps?

kokada 2021-05-20T21:27:11.076900Z

IMO, os for me is for calling OS specific calls, so Python's os exposes things like os.chdir and os.getenv (but there are implementations for them on Windows too, even when they're no-ops)

kokada 2021-05-20T21:27:19.077100Z

While process is related to subprocess

borkdude 2021-05-20T21:27:50.077400Z

yeah, I like it

borkdude 2021-05-20T21:28:28.077600Z

Right now I'm testing set-env in babashka itself. There it doesn't work, weirdly enough.

1
borkdude 2021-05-20T21:28:36.077800Z

While in the isolated case it works.

kokada 2021-05-20T21:28:46.078Z

BTW, any reason this library needs to be GraalVM specific? For example, couldn't we use JNI to implement a Java version compatible :thinking_face: ?

kokada 2021-05-20T21:29:46.078300Z

(Not saying that you need to implement it, just asking if it wouldn't make sense to make it more "generic")

borkdude 2021-05-20T21:30:11.078500Z

Yeah, if that works, I would be ok with it. Not sure if that works though :)

borkdude 2021-05-20T21:32:33.078700Z

$ ./bb -e '(babashka.utils/set-env "FOO" "BAR") (prn (System/getenv "FOO"))'
:setting "FOO" "BAR"
setenv FOO BAR
null
null
nil

borkdude 2021-05-20T21:32:43.078900Z

(branch set-env :-s)

kokada 2021-05-20T21:33:32.079100Z

Well, let me try this on Linux

borkdude 2021-05-20T21:34:08.079300Z

Note that the standalone case (in original post) does work

kokada 2021-05-20T21:34:57.079700Z

Thanks, this saves me the working of trying to build it πŸ™‚

borkdude 2021-05-20T21:35:12.079900Z

These binaries are posted to #babashka-circleci-builds

kokada 2021-05-20T21:37:02.080100Z

> These binaries are posted to #babashka-circleci-builds I always forget about this πŸ˜…

kokada 2021-05-20T21:37:20.080300Z

Tried the same example as above, yep, not working

borkdude 2021-05-20T21:38:25.080500Z

And this one? https://gist.github.com/borkdude/335c9911cabf4db6d47cc772b4c69d4d

kokada 2021-05-20T21:40:26.080700Z

Well, let me setup a GraalVM environment since I generally build those things with Docker πŸ˜…

borkdude 2021-05-20T21:41:43.080900Z

sure

kokada 2021-05-20T21:49:32.081100Z

./setenv foo bar
?

borkdude 2021-05-20T21:49:37.081300Z

yes

borkdude 2021-05-20T21:49:48.081500Z

it should print bar

kokada 2021-05-20T21:50:21.081700Z

./setenv foo bar
bar
Yep, looks ok

borkdude 2021-05-20T21:50:34.081900Z

ok, I copied that code over to babashka

borkdude 2021-05-20T21:50:41.082100Z

and there it doesn't work, weird right

kokada 2021-05-20T21:51:26.082300Z

Yeah, weird

borkdude 2021-05-20T22:19:46.082500Z

I also printed the return value now. It's 0.

$ ./bb -e '(babashka.utils/set-env "FOO" "bar") (prn (System/getenv "FOO"))'
:setting "FOO" "bar"
setenv FOO bar
null
null
0
nil

borkdude 2021-05-20T22:19:55.082700Z

which suggests it should work right

kokada 2021-05-20T22:47:09.082900Z

Maybe it is not propagating to the Babashka process :thinking_face: ?

kokada 2021-05-20T22:47:50.083100Z

Can you print the result of System/getenv right after setting it (still inside the Java code)?

kokada 2021-05-20T22:48:59.083300Z

Ah sorry, looking at the code this is exactly what you do

kokada 2021-05-20T22:49:10.083500Z

And right there it is already returning null

kokada 2021-05-20T22:58:12.083700Z

BTW, if this ever work correctly maybe it makes sense to also implement unsetenv

kokada 2021-05-20T22:58:16.083900Z

https://man7.org/linux/man-pages/man3/setenv.3.html