Clojurians Log v2

if you mean to add variables when calling shell, you can use:

(shell {:extra-env {"SOME" "VAR"}} "your command")

No, in this case I really need to overwrite the environment variables while the script is running

Like doing a export ENV on shell

For now my workaround is to use the #_(<shell code here>) trick described on Babashka's book (https://book.babashka.org/#_running_a_script) to edit the environment variables, but this is ugly

Specially because I can't define a shell function, since this would cause an error because Clojure will try to interpret anything inside {} as a map (and fail to parse it

BTW, it is also possible to change an environment variable in Python I found it curious that even the Java solutions involved doing some magic since it seems that Java itself doesn't have a method for it

@thiagokokada you can do it like this

(shell (format "env VAR1=%s VAR2=%s %s" "var1" "var2" "ls -la")

@mike1452 the above can be done better by passing a map with :extra-env as the first arg:

(shell {:extra-env {"FOO" "BAR"}} "ls -la")

Cool.

There is also :env to overwrite all env vars. These are the same options as in babashka.process

You can also use :out "foo" :err "log.txt" to write to a file

what is fqdn of shell ? babashka.process ? can't find it there

@mike1452 shell is a variant of babashka.process tweaked more towards usage in bb.edn. It is automatically imported from babashka.tasks

Thanks! Now it works

bb -e '(babashka.tasks/shell  {:extra-env {"FOO" "BAR"}} "env")'

bb -e '(babashka.tasks/shell  {:env {"FOO" "BAR"}} "env")'

I wanted to check env settings from one line script.

Like I said in the above thread, I want to change the current environment variables for the program, not for a subprocess

Maybe this will interest someone, it is a modification of test_runner.clj that automatically discovers test files: https://gist.github.com/thiagokokada/fee513a7b8578c87c05469267c48e612

awesome!

FYI, i’ve been playing with autocompletion on the command line. I’ve created a small POC that could maybe grow into something bigger. It’s inspired by how the aws cli tool is structured and can do autocompletion in the same way. The responsiveness is much better though 😎 https://github.com/jeroenvandijk/matryoshka

@russell.matney I started with your work as inspiration, but i couldn’t figure out how to do nested completions. I’m not sure if the AWS approach is better or not, but it works

(forgot the link)

https://github.com/jeroenvandijk/matryoshka/blob/9f70e63418b7b9ee7a58255d83a392da4b7f35de/matryoshka#L83 This is not a "doesn't work yet" problem, but a problem that aliases are only valid in the next top level s-expression. You should use resolve to resolve a fully qualified symbol, if you are in the same expression.

Clojure 1.10.1
user=> (defn foo [] (require '[clojure.set :as set]) (set/union #{1 2 3} #{4 5 6}))
Syntax error compiling at (REPL:1:47).
No such namespace: set

Yeah indeed. I guess I have to have a better look at what you did in bb tasks. This was a quick “addon” for now (i should have left it out)

bb tasks pushes all the :requires to the top level

@jeroenvandijk would your solution work for both bash / zsh? it would be very nice :)

I’m using zsh, not sure about bash, but I’m guessing if we try we could even make it work for Windows 🙂 It uses the same approach as the AWS tool https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-completion.html#cli-command-completion-linux

🤯

The output is slightly different from what @russell.matney did. Maybe the underlying framework is different

I don’t know if this is the way for bash, but in my (basic) bash shell it doesn’t work:

bash-3.2$ autoload bashcompinit && bashcompinit
bash: autoload: command not found
bash-3.2$ autoload -Uz compinit && compinit
bash: autoload: command not found
bash-3.2$ compinit
bash: compinit: command not found
bash-3.2$ export PATH=".:$PATH"
bash-3.2$

not out of the box at least 😅

@jeroenvandijk really cool!

i'll have to dig into the implementation later today

the other thing that keeps coming to mind is wanting to support all of bb's cli opts, not just tasks

Hmm, crap, if you have :deps in bb.edn but if you also have deps.edn, those deps.edn deps are actually also downloaded and added to the classpath ;)

@borkdude I’ve fixed the :deps and :requires “feature”. This works now https://github.com/jeroenvandijk/matryoshka/blob/main/tasks/custom.edn

nice! is this about completion or also about building an alternative task runner?

I don’t know yet. I am looking for something that I can use or all my projects. The autocompletion makes this actually feasible. You start with a category and the suggestions help you navigate

I was thinking of wrapping these commands https://github.com/practicalli/clojure-deps-edn so that I can use them everywhere without having to do constant lookups

what are you currently missing from bb.edn's task runner?

I’m not sure if I’m missing something other than the global part

how does your task runner implement "global" ness?

good question. Not in some particular way, but the difference with babashka tasks is that all the tasks are isolated and it has a tree structure so it should stay fast

I’m not saying this could not be part of babashka tasks and that this is better (i just started working on it today so it’s nothing really). I don’t know yet where this will end. When I have a structure I like I’ll try to see what the main differences are

fast in what sense?

If I have a global library of tasks this could easily become really big. So code loading would be an issue if you are not careful. If you try the aws cli you feel the difference 🙂

But i think it’s a bit premature to talk about my “benefits”. I’m not actually using it for something

well, it's good to take future use case into account without locking down the design to exclude that possibility right now, that's why I am asking

Especially the autocompletion needs to stay fast for a nice experience

The #include is lazy loading here https://github.com/jeroenvandijk/matryoshka/blob/main/tasks.edn#L1

cool :)

So you don’t get hit by a huge startup penalty when you add many libraries. It’s only about the nesting. So a nesting of 4 would mean having to load 4 of these files

instead of something to the power of 4

I think these things can also be implemented using normal keywords instead of reader tags btw

right now a pretty large bb.edn still loads in 1ms

it's first parsed as EDN only and only the task that is executed is actually re-parsed as code

ah yeah so i’m guessing it’s a bit premature

i was also thinking what if you could add a module

i was thinking you could have an aws module and it would look like this file

that was my fantasy at least 😅 Not sure what parts are actually required for this to work

I’m curious how you would use keywords instead of the reader tags btw

(I stole the #include idea from https://github.com/juxt/aero btw. That one is not lazy though)

{:cmds {aws {:cmds "tasks/aws.edn"}}}

haha hmmm no that would be too simple 😛

fair enough, that would be better

yeah, I know that lib. personally I would like to avoid reader tags a little bit, but maybe my feeling isn't justified on good reasons

no i think you are right, probably better to keep it simple

Note sure if it is related, but I noticed that babashka.deps/add-deps doesn’t complain when I give it rubbish

I've been in a few programming language chats and one popular topic / channel was #naming-things

Maybe we need that here as well? 🙂

ha! what a great idea

what kind of rubbish

it just passes stuff through to tools.deps basically

and tools.deps picks up on the local deps.edn

I didn't add the '{:deps ...}' wrapping and I was wondering why my require wasn't working. So not really rubbish, but it was invalid config

I'm trying to make a babashka project with a structure that mirrors a clojure one, and I want a test task that comes with the test dir added to the classpath. First I tried this, but it looks like there's no :extra-paths like in deps.edn

{:paths ["src"]
 :tasks
 {test {:extra-paths ["test"]
        :requires ([foo.test-runner :refer [run]])
        :task (run)}}}

{:paths ["src"]
 :tasks
 {test {:requires ([babashka.classpath :as cp])
        :task (do (cp/add-classpath "test")
                  (require '[foo.test-runner :refer [run]])
                  (run))}}}

@madstap tasks do not (yet) support :extra-paths. you can take a look at how @thiagokokada did it here: https://clojurians.slack.com/archives/CLX41ASCS/p1619532085157000

you could just add the test dir to the classpath in the script and load the script using (load-file ...) in the task

The test-runner script would have to be on the classpath then, for me to be able to run it as a task, or I'd have to run it as a standalone script

you can run standalone scripts with load-file without adding them to the classpath

Oh, right, sorry, didn't catch that. Yeah, that works

you could first use load-file and then call a function from that script, that would work too

Maybe I'm missing something, but this doesn't work

{:paths ["src"]
 :tasks
 {test {:task (do (load-file "test/foo/test_runner.clj")
                  (require '[foo.test-runner])
                  (foo.test-runner/run))}}}

what exactly doesn't work here?

The file does get loaded, but I'm not able to call the function

you probably don't need the require since you already loaded the file

but it should not hurt

I get a ^--- Could not resolve symbol: foo.test-runner/run

hmm, right. this is the gilardi scenario again.

a bit ugly but ((requiring-resolve 'foo.test-runner/run)) should work

That sounds ominous

or simply resolve

The gilardi scenario is basically this:

user=> (defn foo [] (require '[clojure.set :as set]) (set/union #{1 2 3} #{4 5 6}))
Syntax error compiling at (REPL:1:47).
No such namespace: set

The reader has to know about the alias (or namespace) set already before the requires has happened

requiring-resolve works! thanks

I see, that makes sense

I am considering an :enter step which is a hook before the task gets executed

this is for logging the task name for example

but possibly it would also solve this, not sure...

Maybe introducing aliases is better for this

I guess that depends on whether the enter step is ran before reading the task code

or analyzing, maybe

:aliases {:test {:extra-paths ["test"]}}
:tasks {test {:aliases [:test] :task foo.test-runner/run}}

Yeah, that would solve it nicely

Are aliases preferable to an :extra-paths key in the task?

:tasks {test {:extra-paths ["test"] :task foo.test-runner/run}}

dunno :)

I guess it's not too hard to support

I think one could also do it like this, perhaps:

{:tasks {-add-test-cp (babashka.classpath/add-classpath "test"),
           test {:depends [-add-test-cp]  :task foo.test-runner/run}

Sure 🙂

(indenting in slack is hard)

It works

:-D

Easier on the eyes than the requiring-resolve dance, I think

This doesn't work though.

{:paths ["src"]
 :tasks {-add-test-cp (babashka.classpath/add-classpath "test"),
         test {:depends [-add-test-cp]
               :requires ([foo.test-runner :refer [run]])
               :task (run)}}}

that doesn't work because the :requires are processed on the top level before anything else

it should work when you do it like this: :task (do (require ...) (run)

But maybe this is fixable. I think it is. I'll make an issue for this.

Maybe just adding :extra-paths is nicer though. I wonder how common this is

@madstap I pushed support for :extra-paths and :extra-deps in tasks to master now. Are you willing to take it for a spin?

Maybe we won't ever need aliases if we support it like this

since you can depend on a task that adds extra-deps and extra-paths as a sort of alias

wow, that was fast! 🔥

Hmm, yeah, then tasks kinda serve as aliases as well

I'll check it out

are you using macos or linux?

linux

or windows

ok, here is a binary: https://17876-201467090-gh.circle-artifacts.com/0/release/babashka-0.3.7-SNAPSHOT-linux-amd64.tar.gz fresh from #babashka-circleci-builds

nice

Looks like it works

cool so :tasks {test {:extra-paths ["test"] :task foo.test-runner/run}} it is then