Clojurians Log v2

What is the idiomatic way to know what was the original task that bb was called with?

@pithyless this is why I wanted to add these :before and :after things, but I decided not to add those yet because it wasn't clear what the added value was. Right now I believe there is no way to know this. What is your use case?

Probably some self-inflicted pain. ;] I used the :enter hooks to log some info about each running task, but I was thinking in CI I want to spew a lot more info about each task (since it's harder to recover the context), but e.g. on my dev machine I may just want to log that the primary task is running (without having all the dependencies logging their info).

I thought I could maybe just use :init for this, but not sure if there is a reference to a task I can grab at that point?

not really no

Maybe you can make some DEV environment variable which turns off the logging?

also FYI, I was surprised this morning about the way :continue works in shell. I expected it to ignore the error if passed true and otherwise call the proc if one was passed as :continue Turns out if you pass true it works as expected, but if you pass a proc then you deal with all exit-codes (including zero). It makes sense, when you actually read the code for handle-non-zero but it was surprising nonetheless. Maybe that part of the docs could be revised? I thought of :continue as "how to handle errors", but it seems the name comes more from "override the default handling logic, ala continuation-passing style code"

the :continue fn should return either true or false (or some other truthy value), what is surprising there?

> Maybe you can make some `DEV` environment variable which turns off the logging? Actually, I'm passing around a :task/log-level in my context so no issue there; what I wanted to solve was can I have an :enter that logs stuff dependent on the log-level AND if we're now in the primary original task

🤦 re-reading the code now; OK, so I totally misread that part of the codebase this morning

I guess we can add a :primary flag to current-task but what if you have tasks like:

{a :task-a
 b {:depends [a] :task :task-b}
 b:clean (do (clean) (run 'b)}

so actually, what I need to do was override what the error-handling code does (the part that throws the ex-info); and I wanted to avoid rewriting a lot of the exit-checking code, but handle-non-zero is an internal detail of shell right now without much of a way to override it

btw, perhaps the b:clean approach is also a way to deal with your problem: since you can execute something before the task (set some state?) and then run the "primary" task?

should we "rename" / "change" :continue to :non-zero ?

I would consider whatever bb XX was originally called with as the primary task; there just doesn't seem to be a way to hook in and know what bb was originally called with right now, correct?

But not sure how important of a use-case this is for everyone

@pithyless yes, I tried to explain what the problem with this is here: https://clojurians.slack.com/archives/CLX41ASCS/p1620892550258200

(afk, brb)

for me the run 'b doesn't really change anything, since it's no longer what the user initiated from the shell; but I can see how the reverse may also be considered true and obvious; so I guess I don't have a really good case to argue here :]

so, let me take a step back; if :continue is either true or a fn that returns a truthy value, I think the name is fine; it was a misunderstanding on my part

for me the bigger issue, is if :continue is falsy, I can't change the behavior of the error condition (aside from just catching the ex-info exception)

ideally, I'd like to re-use all the logic of shell and handle-non-zero but pass in an alternative operation for the (throw (ex-info ..)) part of handle-non-zero

something like :error-handler (fn [proc opts] ...)

^ a good example of this is replacing the existing exception handler with something like (line-status/die exit-code "...") that will both exit the process and print a nice visible warning

this, BTW, is also what lread seems to want to do here - https://github.com/lread/rewrite-clj/blob/main/script/helper/shell.clj - but since he's just wrapping babashka.process/process he's missing out on the other goodies like automatic tokenize , etc.

Hi, do you know why i am getting “WARNING: this project requires babashka 1.0.0 or newer, but you have: 0.4.0”?

@zikajk This is printed when someone has in bb.edn: :min-bb-version "1.0.0"

but version 0.4.0 is currently the most recent :)

@pithyless gotcha. So maybe we can have an additional :error-handler function then, which will be triggered on a non-zero exit code or if :continue returns false?

@borkdude Yep, that'd be great!

alrighty

Someone's already living the dream of bb 1.0 🙂

I had this in the docs somewhere as an example, perhaps someone copied it ;)

I had to use future version as an example since it didn't work with other versions in the first version that this feature was added ;)

and about the "main" task, what should we do there? we can add back what we had with :dependents perhaps? if :dependent is empty, then you know you are the primary task. although with explicit run this will probably not behave like you think it will?

The joke is one me 😄

{:paths          ["src"]
 :deps           {seancorfield/honeysql                                {:mvn/version "2.0.0-beta2"}
                  douglass/clj-psql                                      {:mvn/version "0.1.2"}
                  clojure-term-colors/clojure-term-colors                {:mvn/version "0.1.0"}
                  borkdude/spartan.spec                                  {:git/url "<https://github.com/borkdude/spartan.spec>"
                                                                          :sha "12947185b4f8b8ff8ee3bc0f19c98dbde54d4c90"}}
 :min-bb-version "1.0.0"}

:)

My original hack was going to be calling tasks->dependees but I noticed it's commented out for now. You're probably right, that it may not always work as expected, so maybe just #hammock it for now? I don't want to be partially responsible for some not well thought-out features that are going be a maintenance burden :D

Babashka adds babashka.file to the System properties when you call bb with bb foo.clj. We could do something similar for run: babashka.task or so

or we could add another function babashka.tasks/plan which returns a map with some information about the execution plan

Both of those ideas sound sensible. The former would be a small overhead to the existing surface area (and would also solve my immediate ask). The latter could potentially have more use-cases, but also more edge-cases. IIRC, the last discussion about an exec plan was halted due to expectations vs non-deterministic ordering in parallel execution.

right

I guess adding another system property doesn't hurt

As for bike-shedding the name for the error handler:

(shell {:error-handler (constantly nil)} "ls foo")
(shell {:on-error (constantly nil)} "ls foo")
(shell {:error (constantly nil)} "ls foo")
(shell {:error-fn (constantly nil)} "ls foo")

perhaps :error is nice and succinct?

You could then write this instead of :continue:

(shell {:error (constantly nil)} "ls foo")

Another question: should the :error function determine the result value of shell or should it just cause side effects (like printing or throwing)

e.g.:

(def x (shell {:error (fn [_] :dude)} "ls foo"))
(= x :dude)

:error-handler
:on-error
:error-fn
:error

if :error is just for side-effects, what would shell be returning?

the process

like it does now

I think it's safe to assume, that :error is the new return value; the caller can easily return proc if that's the intent

it surely is the more flexible solution

ok

so (fn [proc] proc)

also, should it be (fn [proc opts]) or just [proc]?

it's now just one map arg: {:proc .. :task .. :babashka/exit ...}

if you just decide to re-throw this entire map in an ex-info, then it will behave the same as the original error-handler

handle-non-zero takes additional opts from shell; so if we don't pass that on to the handler, it can't use those options

those opts are part of the :proc, e.g. :in and :out

or am I missing something?

you tell me ;)

https://github.com/babashka/babashka/blob/master/src/babashka/impl/tasks.clj#L43

so this could be rewritten as (get-in (deref proc) [:opts :continue]) ?

why would you like to have access to :continue in the :error handler?

so, not necessarily to continue, but if we call (shell {:foo :bar :error (fn []..) } "ls") I guess it doesn't necessarily make sense that we would have access to :foo ; and besides you can always wrap that error handler in a closure if you need additional context

I guess so yeah

Let's go with :error-fn. :continue accepts a naked boolean, whereas :error-fn only accepts a function

$ clojure -M:babashka/dev -e '(babashka.tasks/shell {:error-fn (constantly 1337)} "ls foo")'
ls: foo: No such file or directory
1337

Applied both things on master now. Please test :) Binaries should appear soon in #babashka-circleci-builds

@borkdude is :babashka/exit safe to depend on? or should I use (:exit (:proc opts))? https://github.com/babashka/babashka/blob/master/src/babashka/impl/tasks.clj#L50 https://github.com/babashka/babashka/blob/5014012bd63dd63fecafc94230dfcce10f2e9490/test/babashka/bb_edn_test.clj#L79

:babashka/exit is safe to depend on: you can throw an ex-info anywhere in a script and set this value, then bb will exit with this value, if no other code handles the exception

ah cool, TIL!

this is only supported since 0.4.0

@borkdude confirming that "babashka.task" and "error-fn" works on my machine. Kudos for the frenzied and selfless work, as always! 🤘

babashka 0.4.1 🎉 https://github.com/babashka/babashka/blob/master/CHANGELOG.md#041 Among other tiny updates, babashka linux static is now based on musl and should work on pretty much any amd64 linux version! 🎉 thanks @rahul080327 and @thiagokokada

So has anyone yet started using BB as their installer/dotfile manager?

a bb.edn with dependencies (e.g. install homebrew before installing things using homebrew) sounds like the perfect use

there’s obviously a bootstrapping problem but with staticallly compiled bb and everything having bash/curl/wget that’s not too much of a barrier 🙂

most systems have curl now, except those Alpines

even Windows has it

yeah, I am saying basically: curl babashkainstall &amp;&amp; ./bb <http://init.bb|init.bb> is no problem

great!

@cldwalker what do you do when one of your cloned repos needs updating?

Just manual for now as the diff b/n computers is usually only a repo or two. Could be a useful thing to add down the road

:thumbsup:

hah! nice. I actually didn’t know about brewfile, so that solves that part

Does https://github.com/borkdude/rewrite-edn work with babashka?

it does

How can I call a bb task from outside a folder with bb.edn?

(cd the-folder; bb the-task) ?

(shell {:dir "the-path"} "bb my-task")

heh

yeah, I thought there was a option for that, but the cd one looks good and easy

Hum not sure cd one would work, since the CI uses on clojure JVM

(apply sh (concat command [:dir dir])

["cd" "../my-dir" "&&" "bb" "-m my-project.main"]

not sure that would work with clojure sh

no, that won't work as shelling out from the JVM is not the same as executing bash, but you provide :dir so why not set that to the bb.edn dir?

the dir is dynamic for multiple shell tasks, and this one I want to make using babashka

I can't change that :dir 😕

so shell out another time? (sh "bb" "-m" "my-project.main" :dir "../my-dir")

oh, I see, would be:

["bb" "-m my-project.main" ":dir" "../my-dir"]

that's the option I want so

or (sh "bb" "-m" "my-project.main" :dir (str dir "/../my-dir")) (but then using proper file system stuff)

which sh are you using here?

clojure.java.shell

it's a common file for multiple tasks, I can't change that code, that's why I can only change the command one that is dynamic

it's a bit yucky to execute a bunch of shell commands like this, as you can't really see what's going on, unless you print a giant string blob at the end

does it execute a bash script? 🧵

your "&&" suggests so

yeah, all other places have something like: ["../other-dir/other-script.sh"]

but this one, I want to use bb instead of a bash one

and you don't see any output?

just fire and forget?

yes, I think

well, if it's bash, then you can append (cd the-folder && bb the-task) , the parens start a subshell

hum.. sounds good

I'll try, thanks!

but does clojure.java.shell understand the () ? like (apply sh ["(cd ../my-dir && bb -m main)"])

no. I asked if this was bash

if this is a standalone sh invocation, then why don't you set the :dir yourself

yeah, sorry, it is not, it's a

(apply sh (concat command [:dir dir])

because the dir is set dynamicly for all other commands

what do the other commands look like?

clojure.java.shell/sh can only execute one file at a time

so unless the first thing is "bash" "-c" I have no clue how that setup works

It's a Nubank migration tool, that allows you create multiple migrations and we have something like a migration.edn:

{:migrations [{...
               :command ["../other-dir/migration.sh"]}
              {...
               :command ["../my-dir/migration.sh"]}]}

and I'd like to add a new command, but that uses bb with a bb.edn with custom deps like rewrite-edn

so each :command is executed by a separated sh invocation?

exactly

can you have {:dir ... :command ["..."]} ?

The CI/other service kind of parse the edn and call (apply sh (concat command [:dir dir]))

there dir is the dir with all migrations

so I can't change that easily

interesting, so let's see how clojure.java.shell/merges :dir, maybe you can override it

yeah, if that works would be nice

you know, you can just make a bash script which then cds into the right dir and hook that up to your command thing

> can you have `{:dir ... :command ["..."]}` ? How this works?

yeah I thought about that, but...

calling directly bb would look way better 😛

and if that works, next tasks would follow the same standard

then you should make your migration tool more flexible using an overridable :dir argument, I think

I can't control your migration tools, sorry :)

Yeah, it seems would need that change

haha no problem, is not that easy to change the migration tool

I'll probably end with a .sh to cd and call bb

is just that maybe in the future could make sense bb has a --from-dir or something

if that is possible somehow

to work around the limitations of your migration tool?

hahaha no, maybe someone can need that as well

I'm not even sure if that is possible to implement

yeah me too

the .sh that cd and call bb worked, thanks for the help!

:thumbsup: