Clojurians Log v2

@sogaiu probably need a nimble install zip for the "cannot open file zip/zipfiles"

the install does indeed have tdeps-120 fix

the docker compilation is zero config required, so that would be the prefered way for testing compilation

thanks for the tips -- regarding docker: do you need to install something to use docker? i haven't really used docker.

@sogaiu that's just an apt-get away if i recall correctly, I never developed on linux and managed it, so i'd say it's pretty easy =)

it makes the build reproducible without any configuration effort

they have very newbie friendly installation/gettign started pages

I think that's out of our hands, i tried it (though graalvm isn't yet installed here), and of course get an error, but that's ajava error, so that means we're already started... unless clj.native_image calls clojure.exe again, the problem will be internal to graalvm ?

unfortunately, sh script/docker_build gave the following:

mkdir: cannot create directory ‘dockerout/’: File exists
**** Building the docker image 
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post <http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%7D&amp;cachefrom=%5B%5D&amp;cgroupparent=&amp;cpuperiod=0&amp;cpuquota=0&amp;cpusetcpus=&amp;cpusetmems=&amp;cpushares=0&amp;dockerfile=Dockerfile&amp;labels=%7B%7D&amp;memory=0&amp;memswap=0&amp;networkmode=default&amp;rm=1&amp;session=ek963fgyyx0zigxuid1xvqp45&amp;shmsize=0&amp;t=clojureclibuild&amp;target=&amp;ulimits=null&amp;version=1>: dial unix /var/run/docker.sock: connect: permission denied
!!!!!!!!!!!!! ERROR : Error building docker image !!!!!!!!!!!!!

does this mean i need root privs to build?

i'm not sure i'm comfortable with that sort of requirement just for building

no need for root

after removing the dockerout directory and rerunning, i got a similar error:

**** Building the docker image 
ERRO[0000] failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/run/docker.sock: connect: permission denied 
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post <http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%7D&amp;cachefrom=%5B%5D&amp;cgroupparent=&amp;cpuperiod=0&amp;cpuquota=0&amp;cpusetcpus=&amp;cpusetmems=&amp;cpushares=0&amp;dockerfile=Dockerfile&amp;labels=%7B%7D&amp;memory=0&amp;memswap=0&amp;networkmode=default&amp;rm=1&amp;session=ba26ydwtuk58x7iwdmodlp0k8&amp;shmsize=0&amp;t=clojureclibuild&amp;target=&amp;ulimits=null&amp;version=1>: dial unix /var/run/docker.sock: connect: permission denied
!!!!!!!!!!!!! ERROR : Error building docker image !!!!!!!!!!!!!

did you try the hello-world docker image ?

no i didn't

that's a docker issue

oh you need to be in the docker group

as a user

ok, i will try that

docker run hello-world is the first thing you should try after installing it

well, i'm not keen on docker 🙂 but at any rate, your advice seemed to allow things to proceed further, so thanks for that!

my goal there was to make it so no installation woudl be required outside of docker, which is well documented

no worries -- you're just dealing with an opinionated ex-system administrator who has preferences in other directions 🙂

i'm seeing it from my linux newbie point of view, make it easy =)

why am i reminded of a rich hickey talk? 🙂

well, that's not idle easiness though

build reproductability is at stakes

i agree about importance of build reproducibility -- that's a good thing

it's just a matter of means

maybe my ignorance of the linux world, but i don't see how to script the installation of required dependencies across all distributions, and mac too

i don't know of any magical method to handle this stuff well. as i understand it, some folks use continuous integration systems such as circleci and such for this sort of thing.

yes that's another solution, but that's still docker under the hood!

i didn't get the feeling that all such systems used docker under the hood -- is that really the case?

also i don't know that the user/builder has that working

sogaiu: well i assumed that, there must be some kind of virtualization going on

i'm pretty sure nixos doesn't for example

isn't docker kind of a pseudo virtualization thing?

ah, the build seems to have complete successfully 🙂

weee !

nice!

so i downloaded graalvm to test with adorn

but chrome is trying to verify the download

it has been doing so for hours now

ha ha ha

sorry it's giving you trouble

may be it's not worth your time

well that's chrome's time, i don't care =)

lol -- btw, i copied the docker-built installer to the windows 10 pro machine and ran it with some success, and the installed clj seems to be working on the few things i tried it on 🙂

very nice !

good job!

there is a new -Scp-jar switch to force using a classpath jar, for testing

will take a look

not sure if i'm using it right, but i get: Error: Could not find or load main class clojure.main for the input: clj -Scp-jar in a clojure project directory

oh god ><

i'll chekc it out

that is a bug

works here though

i'll rebuild and try again

i can test with a different installer

i tried with the installer.exe from the releases page for 0.0.6 and got the same results

that-s very strange it works here

you're on windows, right ?

yes obviously

try adding a -Srepro switch

and maybe giving me your deps.edn

java is very fiddly with classpaths in a jar, might be that it cannot find a file or a directory and just bails out

i tried with 2 projects -- also tried with the -Srepro switch, but no change in output -- one of the projects is adorn, so you have the deps.edn for that i think

should i look for another project to try with?

let me try adorne again

ah go it there

same output

ah, with depstar it doesn't give that output

didn't need -Srepro for depstar for it to work -- clj -Scp-jar worked as-is

looks like trouble with git deps

i'll have to think on that one, i may be missing abit of data to be able to do it without support from tools.deps

going for coffee

🙂

https://github.com/cark/clojure-cli-portable Now with git deps in classpath jar bug fixed

@carkh it's looking a bit better -- of the 2 projects i had problems with before, 1 of them now works. perhaps you saw already, but adorn gives the following error when invoked with clj -Scp-jar -A:native-image:

clj -Scp-jar -A:native-image
Exception in thread "main" java.nio.file.InvalidPathException: Illegal char <:> at index 2: \C:\Users\user\.m2\repository\com\cognitect\transit-java\0.8.332\transit-java-0.8.332.jar
        at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
        at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
        at java.nio.file.Paths.get(Paths.java:84)
        at com.oracle.svm.core.util.ClasspathUtils.stringToClasspath(ClasspathUtils.java:49)
        at com.oracle.svm.driver.NativeImage.handleClassPathAttribute(NativeImage.java:917)
        at com.oracle.svm.driver.NativeImage.processManifestMainAttributes(NativeImage.java:886)
        at com.oracle.svm.driver.NativeImage.addImageClasspathEntry(NativeImage.java:1330)
        at com.oracle.svm.driver.NativeImage.addCustomImageClasspath(NativeImage.java:1300)
        at com.oracle.svm.driver.DefaultOptionHandler.consume(DefaultOptionHandler.java:102)
        at com.oracle.svm.driver.NativeImage$NativeImageArgsProcessor.apply(NativeImage.java:1249)
        at com.oracle.svm.driver.NativeImage.processNativeImageArgs(NativeImage.java:1449)
        at com.oracle.svm.driver.NativeImage.completeImageBuild(NativeImage.java:929)
        at com.oracle.svm.driver.NativeImage.build(NativeImage.java:1175)
        at com.oracle.svm.driver.NativeImage.performBuild(NativeImage.java:1145)
        at com.oracle.svm.driver.NativeImage.main(NativeImage.java:1104)

i don't get that

the regular error

that's an error from nativeimage you have there

so the process indeed started correctly

that error you mentioned i get when i don't have GRAALVM_HOME set appropriately

I have no graalvm at all ...chrome still checking it =)

well, then i guess it's not surprising 🙂

but you did get in, my guess is that nativeimage is rebuilding a classpath

and wants unix classpaths

but i don't know how to specify that...We're on windows so there is a colon

the trouble is with them i think

strangely enough, there is some discussion of something similar: https://stackoverflow.com/questions/43972777/exception-in-thread-main-java-nio-file-invalidpathexception-illegal-char

not sure if it's relevant but the comments at: https://stackoverflow.com/a/43972833 has something

can't remove the leading /

or it will just not work with a classpath jar

ah well

about -Scp-jar -- borkdude's clj-kondo builds an uberjar to hand to native-image -- iiuc, that piece more or less works under windows. is that a similar approach? fwiw, the relevant script is here: https://github.com/borkdude/clj-kondo/blob/master/script/compile.bat

would examining an uberjar built in this manner provide any hints / help?

hum but do you have more success without -Scp-jar ?

native-image has a -jar option that allows one to pass in a jar -- if i build an uberjar and hand it to native-image that way, the build completes

how do you build your uberjar ?

the whole windows sdk7.1 + remove redistributables is annoying to say the least ... i'm not sure this project is a good test bed for the cli tools

@carkh uberjar can be built by using leiningen: e.g. https://github.com/borkdude/clj-kondo/blob/master/script/compile.bat#L16 -- so that means a project.clj becomes necessary. it's unfortunate, but apparently until the clj / clojure thing becomes usable, it is at least a work-around that works. regarding the windows sdk 7.1 thing -- yes highly unfortunate. it might be that the windows sdk thing will work if installed from chocolatey, but i didn't find that to work (installation was successful, but building with it was not). yes, perhaps it's not worth your time. please forget the adorn thing for cli tools testing. perhaps i should find some other good projects to test with -- at least depstar seems to be helping in this regard 🙂

maybe the uberjar may be built with a deps.edn project

you're starting the clj.native-image scritp directly from the main-opts

so i guess it's doing the work to actually build the image right there

for building uberjars, i'm not sure whether aot is necessary for this type of situation, but if it is, the number of options available via deps.edn shrinks a bit (e.g. depstar doesn't do aot) -- there are other interesting seeming options (e.g. badigeon and pack), but i haven't really learned how to use them just yet. actually, i think i'm starting to remember that aot is necessary. adorn's deps.edn uses taylorwood's clj.native-image project (https://github.com/taylorwood/clj.native-image) for building the native image binary -- that's what is invoked via the alias. so iiuc it's actually clj.native-image that calls graalvm's native-image binary. fwiw, in the deps.edn alias, "script" is just the name of a namespace.

looking at https://clojure.org/reference/compilation#_gen_class_examples, it sure was easier with lein =)

we're back at doing it manually

it was certainly easy to do aot with lein. perhaps this isn't so bad? https://github.com/juxt/pack.alpha#uberjar

on second thought, it's not clear about whether there is aot support there...

When I was messing with the TDEPS-120 stuff earlier, I found the format of the MANIFEST.MF file to be exceedingly finicky (one reason I wound up falling back on the built-in JDK methods to generate it)

Eg, the second block of code is how they approached it in Gradle here: https://discuss.gradle.org/t/filename-too-long-in-windows/9222/15

the following is what you mean by built-in JDK methods? https://github.com/timgilbert/lein-classpath-jar/blob/master/src/lein_classpath_jar/plugin.clj#L51-L54

Yeah, right. Doing it externally you need to add in like a header and then wrap every line at exactly 70 characters or something

It is a truly annoying file format

what fun... https://docs.oracle.com/javase/1.5.0/docs/guide/jar/jar.html#Notes%20on%20Manifest%20and%20Signature%20Files

@timgilbert yes also it will silently fail if a directory isn't there

so i have to check if these exist

and each line is limited to a very small value, like 128characters IIRC

it's 72bytes actually ><

Yeah

having fun with unicode as bytes, no breaking runes with a \n ...weee

i'm pretty sure the jvm doesn't enforce this limit, because it was working without truncating the lines, and many file names are longer than 72bytes on my config

anyways we're complying now : https://github.com/cark/clojure-cli-portable

Hmm, getting an install error trying to compile this on windows:

CC: libzip_all
Error: invocation of external compiler program failed. The system cannot find the file specified.
Additional info: "Requested command not found: \'gcc.exe -c  -w -mno-ms-bitfields -DWIN32_LEAN_AND_MEAN -Os  -IC:\\Users\\tim\\scoop\\apps\\nim\\current\\lib -IC:\\Users\\tim\\src\\clojure-cli-portable\\src -o C:\\Users\\tim\\nimcache\\clojure_r\\libzip_all.c.o C:\\Users\\tim\\.nimble\\pkgs\\zip-0.2.1\\zip\\private\\libzip_all.c\'. OS error:" 2
!!! COMPILE ERROR !!!

I think it's just a problem with my local setup, I installed nim via scoop install nim

But I guess I don't have a gcc.exe on my path anywhere. Anyways, I will mess with it

i can help

what's your nim --version ?

Actually, seems like I hit a weird sourceforge bug installing gcc from scoop, but I got past that bit

oh

Needed to scoop install upx also

ahyes i should update the readme with that bit

though it's not required

it's more of a finishing touch =)

Ok, got it compiled. Will take me a bit of time to test it as I need to try to eradicate the various other incarnations from my %PATH%

=)