Clojurians Log v2

Probably a super niche usage, but maybe useful for someone: https://gist.github.com/thiagokokada/854e6be42fe80afb946324c62d299ce8 An equivalent of Python's os.isatty() in Babashka, to check if the stdin/`stdout`/`stderr` is connected to a TTY or not (useful to check if the script is being redirected) This uses test (the same as if [ ... ] used in shell) to check it. I don't know if it is portable, but I think it should be considering that test is POSIX.

BTW, process is really great compared to other subprocess libraries that I used in Clojure in the past. Without it, this kind of feature would be impossible (since AFAIK, even Java doesn't offer anything to do it)

Included :enter, :leave and current-task in a branch. It works like this:

$ cat bb.edn
{:tasks {:enter (println ">>" (:name (current-task)))
         :leave (println "<<" (:name (current-task)))
         a (+ 1 2 3)
         b {:depends [a]
            :task (+ 2 3 a)}
         c {:enter (println "My beautiful message")
            :leave nil
            :depends [b]
            :task b}}}
$ bb run --prn c
>> a
<< a
>> b
<< b
My beautiful message
11

I wonder if :enter should be run before the dependencies of the task or like it is now (it was easier to implement like it is now).

current-task gives more info, e.g. when I do :leave (clojure.pprint/pprint (current-task)) in c, I get:

{:enter (println "My beautiful message"),
 :leave (clojure.pprint/pprint (current-task)),
 :name c,
 :before [a b],
 :depends [b],
 :task b}

:before are the tasks that were scheduled before the current task

you can stick any data in the task and it will also be available in current-task

Binaries in #babashka-circleci-builds if you want to play around with this. Perhaps @karol.wojcik this is useful for logging somehow.

for what it's worth, I consider enter/leave wrapping the specific task, so I agree it should work as it does now; i.e. [enter-dep dep leave-dep enter-task task leave-task] :thumbsup:

If I put :enter (clojure.pprint/pprint (current-task)) in b you will also get:

:dependees #{c}

Cool, but is it a set to future-proof with parallel tasks, etc? ie. is it smart enough to parse a graph and understand all potential dependees? Or is it basically "who called me"?

In the --parallel setting, if c depends on the current task, c will always wait before the current task is finished.

Same is true for the non-parallel setting

oh, you mean, why is it a set?

because there is not really an ordering of who depends on "me" , I guess

you will also get :after [c] which is the exact ordering (scheduling) of tasks that are run after you, even if they don't depend on you

the algorithm is deterministic (but may change over versions, not likely)

Makes sense and it's nice to have this metadata available during runtime; I also like the move to the function current-task and that it returns a map with all this context!

I think there is some unwritten rule that any data-driven Clojure library slowly grows into an ad hoc, informally specified DAG (and basically just differs with the amount of work that is put into the implementation of graph traversals and inspection tooling). 😅

hehe. maybe I should base tasks on EQL / Pathom or something? ;)

Btw, maybe it's better to make :after, :before and :dependees functions of the current-task rather than putting it in the map, since there might be some work needed to calculate this (although it's pretty fast right now)

:after and :before are basically free because I'm adding them as I go through a loop, they are readily available, but :dependees is calculated once at the start by basically reversing :depends of all the relevant tasks

How exactly would that work? you would still need to pass all the info needed to generate that context info, right? I can't imagine a lot of extra work is going into actually generating the sets and maps. (And it's not like we're spending a lot of memory, either)

true. https://github.com/babashka/babashka/blob/2cb361ffc3fdab84c49bb9de292a2e404c735d6d/src/babashka/impl/tasks.clj#L260

used here: https://github.com/babashka/babashka/blob/2cb361ffc3fdab84c49bb9de292a2e404c735d6d/src/babashka/impl/tasks.clj#L302

yeah, so you build an index once and use it; I suppose you could wrap it in a function/delay so it won't run if nobody ever needs it, but ¯\(ツ)/¯

and so then :dependees can be a function so it can be lazy-calculated

yeah

(dependees (current-task))

(dependees (current-task) {:transitive true})

but anyway, yeah, it's pretty cheap to add it in a field right now.

even with 100 tasks it's below 1ms:

user=> (def test-case (zipmap (range 0 100) (repeatedly #(do {:depends (take (rand-int 5) (repeatedly (fn [] (rand-int 100))))}))))
#'user/test-case
user=> (time (tasks->dependees (range 100) test-case))
"Elapsed time: 0.769574 msecs"

heck, I guess it can even be lazily calculated when people call current-task. I guess that's the nice thing about exposing a function rather than a dynamic var

I'm totally satisfied with current printing 🙂 Here are some screenshots:

You mean, you don't need any hooks for this, you do it manually, right?

I'm using

(def TASK_NAME (or (resolve 'babashka.tasks/*-task-name*)
                   (resolve 'babashka.tasks/*task-name*)))

(alter-var-root #'babashka.tasks/-log-info
                (fn [f]
                  (fn [& strs]
                    (hpr (str "Command " (red "<") (accent @TASK_NAME) (red ">"))))))

Anything else I'm doing manually

This API will slightly change to (:name (current-task))

also -log-info won't be there anymore

😞

(everything starting with -foo is private, you should not be using it, only for experimentation)

Ok got it. Next release of babashka and I will have to use hooks

How did you get the "required" commands, by parsing the EDN manually?

I just know what holy-lambda needs. I'm trying to check whether tool exists via which. If it does not exists then I'm returning a warning.

ah, I thought it was about the required tasks. in the next version one could print:

:enter (println "Already completed tasks" (:depends (current-task)))

BTW if someone depends on me they are my dependent, so perhaps :dependees should be :dependents

Ah, is that how you call it

I will rename it

For the auto-completion stuff, I was also playing around with the idea in my head to just generate stub scripts:

bb tasks --gen-stubs run

run/task-a
run/task-a.bat
run/task-b
run/task-b.bat

Interestingly this library (which has been suggested to me to use for bb itself) supports generating an auto-completion script: https://picocli.info/autocomplete.html

Yeah i think this is a valid approach as well :)

Regarding subcommands, that’s not a thing in bb.edn yet, right?

arg parsing is the job of a task, bb doesn't really get in the way but doesn't help with it either

it just provides the raw args

yeah exactly, I’ve used this property during experimentation. It’s useful

Did you find examples of generated scripts? I’m curious what they will look like

no, I guess you could try it

Or am i misunderstanding and it’s not generating a bash script

yeah i’ll try it eventually 🙂

I think this is an example (from the tests) https://github.com/remkop/picocli/blob/5ee80a3ff1bc1d3ac0e326a6be8a1b8c06b9f50b/src/test/resources/picocompletion-demo_completion.bash

So the magic happens in bash… I think we can do better 🙂

But good for inspiration

port to bb ;)

Here is a version of generating stubs:

(require '[clojure.edn :as edn])

(def tasks (->> (slurp "bb.edn")
                (edn/read-string)
                :tasks
                keys
                (filter simple-symbol?)))

(require '[babashka.fs :as fs])

(fs/create-dirs "run")

(doseq [t tasks
        :let [f (fs/file "run" (str t))]]
  (spit f (str "#!/usr/bin/env bash\nbb run " t))
  (.setExecutable f true))

And then run/a, run/b

Maybe we can use our Dutch charm here. I believe the creator of PicoCli is Dutch

I mean for our usage, not for picocli, I don't think picocli will make users require bb ;)

Nice 🙂

haha maybe not

I’m not sure it is common practise, but I’m used to use script/ (I guess I copied this from Rails). Is using run/ as directory also common practise?

No, I did this because bb run <task> is the command, but script would also work. This should be configurable.

I personally like script

Cool. Yeah it seems many people use it

Many old Rails users maybe I don’t know

A script that dumps scripts in script so you can invoke a task using auto-completion based on filename:

#!/usr/bin/env bb

(require '[clojure.edn :as edn]
         '[clojure.string :as str])

(def tasks (->> (slurp "bb.edn")
                (edn/read-string)
                :tasks
                keys
                (filter simple-symbol?)))

(require '[babashka.fs :as fs])

(def script-dir "script")
(fs/create-dirs script-dir)

(doseq [t tasks
        :let [f (fs/file script-dir (str t))]
        :when (not (str/starts-with? t "-"))]
  (spit f (str "#!/usr/bin/env bash\n# generated by stubs.clj\nbb run " t))
  (.setExecutable f true))

I understand this is a safe path, but having completion over bb ... would be much cooler IMO, I love that now with bb my scripts directory is gone, and I hope for good 🙂 I guess would be nice to have both approaches, so people/teams that need more compatible solutions could use that, and others could avoid the extra files

Kind of an happy accident that forward slashes don't work in task names ;)

I dont know anything about how shells do auto-complete, is the issue that different shells (bash, zsh…) have different ways to do it?

yeah, I think it can be supported using these bash/zsh scripts

Hi! A friend of mine showed me using docopt with Babashka, loaded using deps/add-deps:

(deps/add-deps
 '{:deps {nubank/docopt {:git/url "<https://github.com/nubank/docopt.clj>"
                         :sha "12b997548381b607ddb246e4f4c54c01906e70aa"}}})

(require '[docopt.core :as docopt])

(deps/add-deps
 '{:deps {
          ;; A local dependency:
          eigenhombre/example {:local/root "."}}})

(deps/add-deps
 '{:deps {eigenhombre/namejen {:mvn/version "0.1.14"})

Hopefully I’m answering this right, I’m happy to be corrected 🙂

1) Code is loaded from the classpath by shelling out to the clj command line tool

2) Babashka has many dependencies built in because a) some cannot be dynamically loaded b) are big and would add extra latency because of extra parsing. So yes, you could strip down babashka if that’s important to you (see the different versions of Babashka based on feature flags)

3) See 2

An alternative to libraries are Pods. These are binaries themselves, often Clojure programmes compiled by Graalvm itself (not a requirement), and connect with Babashka via a socket

I’m trying to understand https://github.com/remkop/picocli/blob/master/src/test/resources/picocompletion-demo-help_completion.bash as we speak 😅 on a second look it doesn’t look that complicated. I’ll let you know

A few additions. 1) You can set the BABASHKA_CLASSPATH or --classpath manually using your favorite build tool. This became tedious so now babashka also supports a bb.edn similar to deps.edn (but with less features, no aliases yet for example). 2) When bb must calculate the classpath using bb.edn or babashka.deps/add-deps it will shell out to java and uses a tools.deps uberjar to download deps and calc the classpath, This is effectively what the clojure bash script also does. 3) So this happens similar to how the clojure CLI does it, but it does not use or need the installed clojure CLI, because this is done using https://github.com/borkdude/deps.clj as a built-in library 4) Interpreted libs are slower than built-in ones and not all of the code is compatible with babashka, so not all libs can be run from source. Having some things built-in, even if they are source-compatible with bb, sometimes makes sense, like clojure.tools.cli. 5) Babashka has feature flags that can be used to trim the binary down, if you don't need all of the built-in libs. It also has feature flags to enable more built-in libs. 6) Most notably libs relying on non-standard Java classes, deftype or definterface are not source-compatible with babashka 7) Some non-compatible libs can be made compatible by introducing :bb reader conditionals

^ @eigenhombre

This semi works. Needs proper unit tests for all the weird edge cases https://gist.github.com/jeroenvandijk/42a570415a45c68989f4f506977050c4

how must one use it?

this should do it https://gist.github.com/jeroenvandijk/42a570415a45c68989f4f506977050c4#file-autocomplete-clj-L6-L19 although it can be a bit funky. There seems to be some state involved

so in this case bb-complete is not a command but it will be autocompleted. If you type bb-complete and press TAB it should do something

very vague sorry. I need to test it more thoroughly and several times to make sure it works in any state

I’ll look into it later again, but i’m just posting it here in case someone wants to try it

I have some other questions: 3) This still needs java installed right? If I want a fully "static" (something that only depends on Babashka) script I need either to use uberscript or uberjar right? 4) If I add the built-in libraries to babashka in my bb.edn, does it load the built-in ones or the ones defined in bb.edn. I am asking this because I include some builtin libraries in my bb.edn because my Babashka's project also have a project.clj with lein-tools-deps to make development easier (e.g.: at least for now, CIDER doesn't support Babashka REPL very well), but I wouldn't want to inccur an additional load for this. If this is an issue I may move all bb.edn dependencies to project.clj. 7) Does Babashka's interprets :clj reader macro? If yes, what it does in case of having both :clj and :bb?

3) correct, or take care that the deps have been already downloaded and provide the classpath manually 4) it loads the built-in ones, unless you use (:require [foo.bar] :reload) 7) Yes, it does, for compatibility with existing libraries. If you have both, then put :bb as the first branch

> If this is an issue I may move all `bb.edn` dependencies to `project.clj`. Actually, this seems to work fine, and seems to be a better idea regardless so I did it Anyway, I am still curious if Babashka will load the dependencies from bb.edn or the builtin ones

Thank you @borkdude. So, when Babashka pulls in a dependency from, say, Maven, and runs it, it is literally interpreting the source code it gets out of the downloaded jar file?

yes

Wow, cool. It is so great to see Babashka continuing to evolve in new and useful ways.

> it is literally interpreting the source code it gets out of the downloaded jar file? And this only works for Clojure jars right? Since sci can't interpret Java files

True. It only works for Clojure source

It is really amazing how well eveything works, thanks for the explanation @borkdude

@pithyless and others. I have one other naming bikeshed:

:after [update-project-clj -uberjar build-server -jar lsp-jar]
:before [java1.8]

I could also just leave this information out completely, and only add :dependents #{-jar update-project-clj -uberjar}. I guess that info is useful in the sense of answering the question "why am I being invoked", whereas the exact order of tasks isn't very useful information and can even be confusing (with respect to parallel).

:past :future or :previous :next?

I think this kind of information maybe interesting to debugging purposes, but not if it is showed always