Clojurians Log v2

is there a way to package bb + the clojure script for windows users?

currently using cat config.edn | ./app.clj with #!/usr/bin/env bb -I

@danieltanfh95 I think only macOS takes args in env. For standalone scripts it might be better to just use edn/read

oh

lemme try with my windows machine later

btw did anyone manage to integrate spec with babashka?

@danieltanfh95 For now there is: https://github.com/borkdude/spartan.spec There are some reservations about including spec into bb, although it can work. The hesitation is because spec is alpha.

nice

You can post some feedback on that here: https://github.com/borkdude/babashka/issues/558

im using edn instead of command line args, it’s easier to do complex arguments with it, especially if spec is available

I've also done that for carve: https://github.com/borkdude/carve

spec will be at one point part of babashka, but only when it's out of alpha probably

until then spartan.spec will be a drop-in replacement

Out of curiosity, I’m looking at the code of https://github.com/borkdude/cpm and wondering how it works. It says it’s piggiebacking on tools.deps, but in the code I don’t see this connection. Is this already there?

@jeroenvandijk My thinking was that dependencies between packages could be done via Git deps instead of implementing dep resolution ourselves. Also it uses the classpath to discover packages. But maybe it will just be simpler if cpm will be a standalone binary with a packages repo similar to brew

I now added a --global option: https://github.com/borkdude/cpm/#global

Cool, yeah I haven’t thought about it enough. I believe a package manager would definitely be useful to grow babashka. I’m guessing this is coming from a need to organize pods as well?

Anyway, I also read it is work in progress. So don’t mind me, was just curious 🙂

That was the initial inspiration yes

Nice

You can now also install a SNAPSHOT babashka version using CPM: https://github.com/borkdude/cpm/#babashka

which is kind of meta, since you need a SNAPSHOT version to run CPM with bb :)

I guess I could also add a SNAPSHOT version of clj-kondo. Hmm, this could be nice for people who want to be on the latest bleeding edge ;)

To package things up nicely, use babashka's --uberjar option:

$ bb -cp src:packages -m cpm.main --uberjar cpm.jar

$ bb -jar cpm.jar --install clj-kondo/clj-kondo --global --verbose

I just noticed 🙂 Nice though!

➜  ~ bb -jar cpm.jar --install clj-kondo/clj-kondo --global --verbose
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  Unable to resolve classname: java.util.zip.ZipInputStream
Location: /Users/jeroen/.gitlibs/libs/borkdude.cpm/borkdude.cpm/bf8b8e284382ee7c9fa180d6db15595b54f25bbe/src/cpm/impl/impl.clj:34:13
Phase:    analysis

And I did this

export BABASHKA_CLASSPATH="$(clojure -Sdeps '{:deps {borkdude.cpm {:git/url "<https://github.com/borkdude/cpm>" :sha "bf8b8e284382ee7c9fa180d6db15595b54f25bbe"}}}' -Spath)"

Yes, this is why you need the snapshot version :)

which you can install with clojure

Btw, do keep the feedback coming and do use it (maybe only for dev now), this is the only way it will improve over time

Hm, I guess cpm can also be used to install individual babashka (or clojure / whatever) scripts on the path

works!

clj -Sdeps '{:deps {borkdude.cpm {:git/url "<https://github.com/borkdude/cpm>" :sha "bf8b8e284382ee7c9fa180d6db15595b54f25bbe"}}}' -m cpm.main --install clj-kondo/clj-kondo --global --verbose

How would it work? Something like #/usr/bin/env bb + additions to babashka stdlib that can add stuff to the classpath?

Well, e.g. deps.clj is a standalone script with a bb shebang. If I would package that in up a zip, then it would be no different from installing a binary

It does depend on bb so maybe that would have to be added as a dependency as well

bb already has a stdlib thing to add stuff to the classpath, but I don't see how that is relevant here? for some specific scripts yes

So then the script would maybe add :package/depends [org.babashka/babashka] and that package will be installed as well

https://github.com/borkdude/cpm/issues/7

cool, sounds great 🙂

I guess a bit like how you can add binaries via ruby gems? (maybe also in python via pip, not sure)

clojure -M -m cpm.main --install borkdude/deps.clj --global --verbose

clojure -M -m cpm.main --install borkdude/deps.clj org.babashka/babashka --global --verbose

if deps.clj depends on babashka

so then you will have bb on your global path as well

ah cool, so do i understand correctly that cpm would wrap deps + babashka? So calling cpm would allow fetching all deps and then run a main ?

no. it doesn't wrap anything. it can install deps and babashka, that's all

after that you can run deps.clj, the script.

cpm doesn't run programs, it's just a package manager that can put binaries (or scripts) on your path

just like brew basically

but cross platform

and possibly using clojure for scripting if you need to move things around after fetching the artifacts

ah i see. So you meant the executable could be a babashka script and because of it’s defined deps those will also be available in that shell session. I think i get it ☺️

yes

so if you don't have bb installed, but you install deps.clj (as a bb script, not the binary deps), then it will also make sure bb will be installed, so you can run the script

very cool

I see now that’s more ~/.cpm/path gives binary paths. I think it’s clear now 🙂

nice idea

yes, and you can add:

# cpm
export PATH="`cat $HOME/.cpm/path 2>/dev/null`:$PATH"

would that also pick up new binaries just after installing it via cpm? I’m guessing brew adds symlinks to /usr/local/bin for that reason

yes, if you use --global it will overwrite that path file

you will need to reload your shell though

yeah i meant that part. Might not be an issue

I'm not sure about working with symlinks, since I also want to support Windows

True, probably better to wait for some feedback

it's a minor issue as I see it now

maybe a tiny bash function could be added and that can be called post-install

I'm sure there's a way to fix it

I agree

alias cpm_path='export PATH="`cat $HOME/.cpm/path 2>/dev/null`:$PATH"'
cpm_path

don't typical shells have a rehashing command usually so you don't have to reload? something like this: https://superuser.com/questions/490983/how-to-rehash-executables-in-path-with-bash

it's not always called rehash iiuc

(can be improved by first removing the old cpm_path from the path)

could be but the way cpm is set up now is that the path is added via a file

there are no global symlinks, etc

See https://clojurians.slack.com/archives/CLX41ASCS/p1601292201085800

We could also make main return a bash command (using a --bash) flag so you can use source (and --powershell, etc), not sure if that's useful. Will wait with that.

Cool 🙂

If I understand correctly the current distribution method is via maven? Are you considering adding git repos? Maybe useful for being able to add (babashka) gists?

can you be more specific?

yeah i was thinking of creating a gist and being able to call (+ install) it as an executable? From the install instructions I’m thinking it’s not possible (yet). It seems the script is currently made to download a zipfile?

- cpm.edn files are currently read from the classpath. I am considering also supporting .cpm/packages/your-packages so you can git clone your package repository there. - yes, it currently only fetches zip files, I want to add tar.gz, possibly other things as well

maybe it's already possible to download a gist as a zip file from github

yep, there is a "Download zip" button

e.g. https://gist.github.com/tito/4096543/archive/fe2f62e11337a65ea501923c0f4dbcfd4ed75fc9.zip

ah cool, so the current way would be to add it manually or add it via in repository dir of the cpm repo. And you are consider having personal packages

I’m thinking out loud 😊

the current way is to have a jeroenvandijk/my-cool-gist.cpm.edn file on your classpath (either bb classpath or clojure classpath)

cpm/packages is also on the classpath because packages is in :paths in deps.edn

It sounds like the sky is the limit 😎

😎

Maybe for installing things like GraalVM we need to add some more stuff, e.g. when unzipping it, you will get a directory structure in which the binaries are not on the top level, so we need to expose a couple of paths instead of just the top level

And maybe having aliases for binaries would also be nice in case of conflicts

Hey, I am trying to pass a function from a pod to another client side function from a pod but that’s causing issue as it can’t be serialized

(require '[pod.tzzh.a :as a])

(a/do-something-client-side a/some-pod-fn)

correct, you can't pass functions through pod borders

ah ok I see thanks :thumbsup: I can work around it but I would have thought that client-side functions didn’t need to serialize their args / speak to the pod

this example (a/do-something-client-side a/some-pod-fn) should work purely client side, if a/do-something-client-side was for example defined as (def do-something-client-side identity) but it appears like you are passing a function object back to the pod or vice versa

everything that goes back to the pod or comes from the pod has to be serializable

you could work around it using symbols or macros

'a/some-pod-fn or (defmacro client-side [fn-sym])

OK cool I will take another look thanks

but I wouldn't know why you would pass back a pod function itself to the pod?

more concrete example will likely help

afk for an hour, will check back then

OK yeah basically it’s for the aws pod I was trying to improve the pagination and lazily load pages so I have now a new namespace that exposes this client code:

(defn paginator-fn
      "Returns a fn that lazily fetches the pages for a given aws fn"
      [page-fn]
      (fn get-pages
        [input]
        (lazy-seq
          (let [page (page-fn input)]
            (if-let [next-continuation-token (:NextContinuationToken page)]
              (cons page (get-pages (assoc input :ContinuationToken next-continuation-token)))
              [page])))))

(require '[babashka.pods])
(babashka.pods/load-pod ["./pod-tzzh-aws"])
(require '[pod.tzzh.paginator :as p])
(require '[pod.tzzh.s3 :as s3])

(let [paginator (p/paginator-fn s3/list-objects-v2)]
    (paginator {:Bucket "some-bucket"
                :Delimiter "/"
                :Prefix "some-prefix"
                :MaxKeys 50}))

ah yes, then simply make that (p/paginator-fn 's3/list-objects-v2) or write p/paginator-fn as a macro so you can pass it as a symbol and expand to (p/paginator-fn* 's3/list-objects-v2)

hmm, I'm not sure if I understand still

is (p/paginator-fn s3/list-objects-v2) running entirely client side?

yeah, that should work I think

running only (p/paginator-fn s3/list-objects-v2) does that work without error?

no that’s the one that’s throwing an error

I need to see your pod branch for more info

back in an hour

so if you could post me the link by then, that'll be great

@thomas.ormezzano A test case: define a client side pod fn as (defn foo [f x] (f x)) and call it from bb as (foo inc 1). This should not communicate with the pod at all, and should return 2

OK I’ll try that, I have also pushed my branch https://github.com/tzzh/pod-tzzh-aws/pull/2

clojure.lang.ExceptionInfo: Cannot JSON encode object of class: class clojure.core$inc: clojure.core$inc@112957440

do I still need name when using code when describing the namespace ?

@thomas.ormezzano which bb version are you using?

v0.2.0

that should be ok

can you push your branch with the foo function added?

:thumbsup: done

@thomas.ormezzano I added a test for this at babashka pods and it works as expected: https://github.com/babashka/babashka.pods

Maybe it's a capitalization issue?

bb pods expect the json to be "code"

OK thanks for checking I must have messed up somewhere I’ll try to have a deeper look when I have a bit more time

or wait, maybe it's an issue in JSON decoding in bb pods... I haven't tested that case

neh, it's in bencode, should be the same

Can you print the JSON that is generated by the go code?

bencode / json

hmm, you could try to run your pod using the babashka.pods library on the JVM and insert some debugging

to see what is coming through

ok it’s definetely the serialisation not working, basically if I have Code string “code” then it works but the other Vars get an empty string

aha

ok fixed it now 🎉 thanks a lot for the help

🎉

ok so for anybody that might be using the pod I have open https://github.com/tzzh/pod-tzzh-aws/pull/2/files which would allow to have better pagination as the pages would be loaded lazily, I also want to remove the current .*-pages fn, e.g s3/list-objects-v2-pages as the pod is still very new and they wouldn’t be useful anymore with the new pagination. If anybody is using the pod and is not happy with this let me know.

I think considering the age of this project, some breakage would be ok. As a safe step you could undocument the things you want to remove and then remove it later

Also keeping a CHANGELOG.md with breaking changes would help users

Hi there i am a babashka newbie...I wonder what I can do to make a clojure lib working...for instance this one? http://clojuremongodb.info/ I tried to hook it up in the classpath but got an error back

@richiardiandrea bb is only able to interpret a selected set of Java classes (e.g. if joda time is used in that lib, bb can't handle it) and not all Clojure constructs (notably deftype is currently missing) So currently there's no way you can run that library from source in bb.

oh i see, so the Java class needs to be in bb for it to work...basically any wrapper is a bit out of luck?

unless you write a pod and compile that library with GraalVM to something native, that could work, but that's work, not out the box

I see

@richiardiandrea A full list of currently supported classes: https://github.com/borkdude/babashka/blob/master/src/babashka/impl/classes.clj

I will inspect the pod documentation, thank you Borkdude for the prompt answers

@richiardiandrea Sure! Shelling out to another CLI is also an often utilized technique: https://github.com/DarinDouglass/clj-psql

But I do have a postgres sql pod as well: https://github.com/babashka/babashka-sql-pods/

I'm not getting quite the output that I expect --- check this out:

$ echo "hello there lets repunctuate"|bb -io '(spit "foo" (str/replace *input* " " ":"))'
$ cat foo                                                                                           
("hello:there:lets:repunctuate")

missing output flag maybe?

hm, the presence or absence of the o flag has no effect on this

$ echo "hello there lets repunctuate"|bb -i '(type (str/replace *input* " " ":"))'             
java.lang.String

:thinking_face:

this is a good idea, in my case I would shell to either mongo or rabbitadmin 😉

@mathpunk

$ echo "hello there lets repunctuate" | bb -io '(type *input*)'
clojure.lang.Cons

$ echo "hello there lets repunctuate" | bb -io '(str/replace (first *input*) " " ":")'
hello:there:lets:repunctuate

huh! I guess I expected str/replace to throw an error, rather than returning something that looked a lot like it had been executed on a string

yeah, I guess that function is pretty forgiving:

user=> (clojure.string/replace 1 "1" "2")
"2"

just a very chill, accepting function

thanks for the help!

ran into an uberscript error. I assume this is the same issue of only certain Java classes work right now?

$ bb -cp $(clj -Spath):src -m templatus.core --uberscript templatus.clj
rlwrap: warning: Your terminal 'dumb' is not fully functional, expect some problems.

warnings can be silenced by the --no-warnings (-n) option
----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  Unable to resolve classname: java.util.regex.Matcher
Location: clostache/parser.clj:1:1

----- Context ------------------------------------------------------------------
1: (ns clostache.parser
   ^--- Unable to resolve classname: java.util.regex.Matcher
2:   "A parser for mustache templates."
3:   (:use [clojure.string :only (split)]
4:         [clojure.core.incubator :only (seqable?)])
5:   (:require [<http://clojure.java.io|clojure.java.io> :as io]
6:             [clojure.string  :as str])

@goomba which bb version is this

good question

$ bb --version
babashka v0.2.0

Are you trying to make an uberscript for non-bb usage?

@goomba?

The next error you will hit using this project is:

Message:  Could not resolve symbol: clojure.lang.Reflector/invokeConstructor
Location: clojure/core/incubator.clj:73:4

73:   (clojure.lang.Reflector/invokeConstructor
       ^--- Could not resolve symbol: clojure.lang.Reflector/invokeConstructor
74:    (clojure.lang.RT/classForName class-name)

@goomba if you're looking for a templating library, this one works with bb: https://github.com/weavejester/comb

if your goal was to simply create an uberscript that you can run with clojure on the JVM: I can likely improve the uberscript feature by just ignoring classes in namespace forms

@borkdude thanks for the feedback. The Babashka runtime is the goal or I could just run it with clj. I l'll look into comb, thank you for the feedback!