Clojurians Log v2

Is it possible to create tab-completion with a graalvm compiled project? Like the video below? If so, can someone point me at the right direction?

Or is that more of an OS think that you need to package?

It’s os/shell like https://iridakos.com/programming/2018/03/01/bash-programmable-completion-tutorial

ah perfect, no idea why that was so hard for me to find haha

Thanks

@kevin.van.rooijen https://github.com/l3nz/cli-matic/issues/39 might be of interest to you as well. If you want to go further into the direction of generating the completion from an abstract option declaration, I can recommend https://github.com/spf13/cobra/blob/master/bash_completions.go as reference.

Added clj-http to my project and getting this error:

Error: No instances of javax.net.ssl.SSLContext are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked.

I can use clj-http-lite but I'm just being stubborn at this point.

Thanks, this is really helpful!

Seems like the graalvm-clojure project struck out of luck as well https://github.com/BrunoBonacci/graalvm-clojure/tree/master/http-kit#notes-on-http-kit-client

Assuming you're trying to use the client?

@naomarik isnt't this just these flags?

"-H:EnableURLProtocols=http,https"
       "--enable-all-security-services"

apparently not: https://github.com/oracle/graal/issues/1074

yeah was about to post that https://github.com/oracle/graal/issues/1074

I'm playing around with attaching a debugger but I have no idea what I'm doing... seeing if I can get a breakpoint somewhere

if you can figure that one out, it would be great. clj-http-lite has been the escape hatch for it for a while

which works quite well also btw

erm. sorry. this is http-kit, not clj-http

yeah it works for me, but I tend to get stubborn with issues and it's hard to not dig myself in a hole until I get things working the way I want

Not sure how to do this, in clojure top level require seems to initialize classes which cascades all the way down to the java SSL stuff and trying to wrap http client in a function to dynamically require gets it compiled out of graalvm.

@naomarik you can't dynamically require something in a graalvm image, at runtime. it has to required at compile time

yeah, don't have enough knowledge to deal with this

it boils down to this: eval does not work in graalvm native-image

I hacked clj-http wrapping the problematic code in a delay and it works, passes all tests too.

cool!

The classes were in a def so always being executed.

do the tests pass with the native binary?

I can compile it into my graalvm and it works, is that what you mean?

lein test passes and my standalone graalvm doesn't complain about anything

I mean, can you do everything that's done in the units tests, with your graalvm binary? that remains to be seen right?

not sure how to run all the unit tests in a graalvm binary

@naomarik one way of testing this is to bind the clj-http functions into babashka and then run the tests using babashka

I also test other libraries like this such as clj-commons/clj-yaml

Not sure how to do that ;/ you have an example?

I can throw my project up on a fork if you want to try it

sure: 1) create one of those and hook it up in babashka/main.clj: https://github.com/borkdude/babashka/blob/master/feature-yaml/babashka/impl/yaml.clj 2) copy the library tests to https://github.com/borkdude/babashka/tree/master/test-resources/lib_tests/clj_yaml 3) run all tests within the bb binary: https://github.com/borkdude/babashka/blob/master/script/lib_tests/clj_yaml_test

Here's a patch: https://pastebin.com/raw/d5CCRK9z

Will try that

@naomarik let me know if you get stuck. this is very interesting

here are some hints how to get started. most often people don't clone recursively: https://github.com/borkdude/babashka/blob/master/doc/dev.md

Hey @borkdude, I copied https://github.com/dakrone/clj-http/blob/3.x/test/clj_http/test/core_test.clj into test-resources/lib_tests like like clj-yaml.core-test , I edited babashka.impl.clj-http and main.clj get rid of each error as it came up and got around here:

➜  lib_tests git:(master) ✗ ./clj_http_test
java.lang.Exception: Could not require ring.adapter.jetty. [at /home/naomarik/babashka/test-resources/lib_tests/clj_http/core_test.clj, line 1, column 1]
Error encountered performing task 'run' with profile(s): 'test'
Suppressed exit
➜  lib_tests git:(master) ✗ ./clj_http_test
java.lang.Exception: Unable to resolve classname: java.net.SocketTimeoutException [at /home/naomarik/babashka/test-resources/lib_tests/clj_http/core_test.clj, line 1, column 1]
Error encountered performing task 'run' with profile(s): 'test'
Suppressed exit

@naomarik I see: the clj-http tests run a server to test clj-http. this requires a bit of a different approach. the server needs to be started in a JVM, and then the clj-http test code needs to run in babashka itself

yeah seems like a long story 🙂 can't spend time on it anymore unfortunately, but running around babashka source, man you've done a lot of work!

@naomarik you can try to do a couple of request just outside the tests though: bb -e "(require '[clj-http ...] (client/get ...)", etc

yup! actually i did that and it works

@naomarik do you have a link to your fork?

might be worth doing an issue + PR against the original repo

@naomarik I also tried it locally myself now with your fork.

$ ./bb "(require '[clj-http.client :as client]) (:status (client/get \"<https://www.google.com>\"))"
java.lang.ClassNotFoundException: org.apache.commons.logging.impl.LogFactoryImpl [at line 1, column 50]

I would categorically seek to avoid adding clj-http.client to a native image

it's not that good of an http library

though many use it

and it's certainly not designed for native-image (many many deps)

40MB is a lot

yep

saw a similar thing with hato. clj-http-lite might be a nice middle ground for native-image CLIs

or if shelling out to curl is fine, then https://github.com/borkdude/babashka.curl is probably the lightest solution

it's still cool that naomarik got it to compile though, kudos

oh strange, for my project clj-http compiles to 66MB, clj-http-lite is 57MB

Without either it's 56MB, so clj-http adds 10MB for me.

Using GRAALVM_VERSION="20.1.0" , openjdk version "11.0.7" 2020-04-14 under WSL2 ubuntu

that's weird. I'm on mac here

These are binaries built on CI:

[8:02 PM] [linux-static - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19]: <https://10052-201467090-gh.circle-artifacts.com/0/release/babashka-0.1.3-SNAPSHOT-linux-static-amd64.zip>
[8:02 PM] [linux-static - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19] binary size: 105772312
[8:03 PM] [linux - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19]: <https://10049-201467090-gh.circle-artifacts.com/0/release/babashka-0.1.3-SNAPSHOT-linux-amd64.zip>
[8:03 PM] [linux - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19] binary size: 110654720
[8:05 PM] [macos - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19]: <https://10050-201467090-gh.circle-artifacts.com/0/release/babashka-0.1.3-SNAPSHOT-macos-amd64.zip>
[8:05 PM] [macos - dakrone-clj-http@bfffbd9400c04e3c345f2251b4ab8a4fa68a7d19] binary size: 103977664

all over 100MB

without clj-http ~60MB

maybe you did some other things apart from the patch @naomarik?

no nothing, are you using the latest graalvm?

i believe the graalvm I'm using is java 11 as well

not sure if that makes a difference

http-kit works btw on latest dev version

I'm using the same version you are using. Maybe you already had a lot of overlapping deps with clj-http and my project didn't