Clojurians Log v2

I'm looking for a task scheduler / cron-like-thing that's happy in babashka, any suggestions?

Probably just use cron?

Hi, I would like to confirm my understanding of how to run the aws cli from a babashka file. I would use “https://book.babashka.org/#_clojure_java_shell” as described in the book? Is this the prefered way?

@steven.katz it depends. if you would like to capture output, or only like to cause a side effect and don't want to use the output, but see the output as you execute it

hmm, not sure, right now I guess I don’t really need to capture the output, just look at it as it executes

In that case babashka.process offers a better alternative:

(require '[babashka.process :refer [process]])
@(process ["aws-cli" "your-argument1" "your-argument2"] {:inherit true})

The shell function from babashka tasks can also be used, it basically the same with a slightly different API, but with defaults for running it as a task:

(require '[babashka.tasks :refer [shell]])
(shell "aws-cli your-arg1 your-arg2")

Great, thanks!

I'm trying to use babashka in my CI environment, currently trying to just install the binary and make it work on a "vanilla" alpine image. here's my trouble:

bash-5.0# wget -O /tmp/bb-install <https://raw.githubusercontent.com/babashka/babashka/master/install>
Connecting to <http://raw.githubusercontent.com|raw.githubusercontent.com> (185.199.111.133:443)
bb-install           100% |***********************************************|  3504  0:00:00 ETA
bash-5.0# bash /tmp/bb-install
Downloading <https://github.com/babashka/babashka/releases/download/v0.4.3/babashka-0.4.3-linux-amd64.tar.gz> to /tmp
Successfully installed bb in /usr/local/bin
bash-5.0# which bb
/usr/local/bin/bb
bash-5.0# bb
bash: /usr/local/bin/bb: No such file or directory

not sure what's going wrong tbh

@skuro for Alpine you need the --static argument of the installer script

oh I see

perhaps we can switch to this as the default on linux at one point, @rahul080327 @thiagokokada

mhhh

bash-5.0# /tmp/bb-install --static
Downloading <https://github.com/babashka/babashka/releases/download/v0.4.3/babashka-0.4.3-linux-amd64-static.tar.gz> to /tmp
mkdir: unrecognized option: static

hmm, a bug in the script?

install_dir seems to become --static

ah here's the bug:

if [[ $# -eq 1 ]]; then
   install_dir=${1:-}
fi

yep

we can remove that part I guess, just use explicit --dir ...

lemme do that

fixed, but github caches it for a while, so try again in some minutes

At this point I'm much more confident in making it static everywhere should be safe

Maybe we can do an A/B test somehow 😛

@skuro the issue should be fixed now I think

Thanks I'll try again later

@skuro there is also a bb alpine image btw:

babashka/babashka:0.4.3-alpine

hehe rand-int

Oh ok, missed that completely, i blindly tried borkdude/babashka:alpine

Maybe, but something to keep in mind is that maybe musl may have some difficults doing requests in some configurations (thanks to the fact that it embeds its own DNS resolver instead of using NSS)

The cases where (and if) this will cause issues should be rare, so IMO it is safe to switch, but I think it is important for people to understand the implications of static binaries

ok, we'll just leave it for now

I deleted borkdude/babashka and borkdude/clj-kondo on Dockerhub now. Let's see how many questions I will get and how many CIs will break... tl;dr: use babashka/babashka and cljkondo/clj-kondo

Let me check nixpkgs. I think I migrated both, but not sure

Hello! I would like to load a babashka file, which only contains "helper-functions" for what I would like to script. In the end, I'm loading this file on a closh shell, and running a pipe. 1. First problem I have, when I run bb -f partitioning.clj It returns, as defined, only the last-defined function in the file, #'devibrary.partitioning/create-populate-dir The integral code can be found in, https://github.com/BuddhiLW/CloshBashika/blob/main/devibrary/src/devibrary/partitioning.clj Q: How could I load all the functions properly?

What you see is just the printed result.

Babashka prints the last return value

so the file is loaded.

Maybe I should have been more careful? On the other hand, it's been 6 months now

Well, at least for nixpkgs seems ok

> Maybe I should have been more careful? On the other hand, it's been 6 months now If you really want people to fix their CIs breaking it is the only way 🤷 Those group renamings on GitHub are really bad on UX side (since it only shows when pulling/pushing)

Seems like I'm unable to use the functions inside the namespace

Also, if I manually load the function on the closh environment, it works fine.

I thank you for your time, @borkdude. I'm not really in a rush. If anyone knows what is happening would be great. But, for now, loading things in the shell has been working fine, for my scope/work.

Ah, just saw that you're referring to the Dockerhub (I missed it and thought it was the GitHub repo 🤦 )

no

@pedrogbranquinho I’m guessing the problem is that you are not calling a “main” function. Like @borkdude said “Babashka prints the last return value”. So the last value is a function

If you put (your-function) and the end of the file it should run

A common pattern is

#?(:bb (apply your-main-function *command-line-args*))

Maybe this is also an option https://book.babashka.org/#main-function

@pedrogbranquinho I’m rereading the thread and I’m not sure if I actually understand your problem. Is it your goal to invoke this function from the command line? https://github.com/BuddhiLW/CloshBashika/blob/main/devibrary/src/devibrary/partitioning.clj#L68

Is there some way in Babashka that I can set the directory where a script/project is running, without needing to change the directory :thinking_face: ? Context in 🧵 .

I have a Babashka project (has a bb.edn and src, test directories) that needs to be accessed from anywhere (so I added a bin directory from this project to my PATH)

Maybe this helps: https://github.com/bob-cd/bob/blob/main/bb.edn#L30 the way im using tasks in this mono-repo?

Nowadays what I do to load the bb.edn (that includes this project deps, classpath, etc) is to have a hybrid shell script/babashaka script that cd into the script directory and runs the program

@rahul080327 I don't think this will work because of the next reason

The approach above worked fine, but today I discovered that I can't change to the project directory for one reason: I will lose the directory where the user is running the project

So I have another command that tries to download some things on the current directory, but now it is always downloading it to the directory that my project is hosted (since I changed the directory before running the project)

I can't change this command because it is not called directly by my project (it is called indirectly in a fallback flow)

The major issue that I am facing now is that while bb -f /path/to/script.clj works, it "ignores" the bb.edn file

So if we could have something like bb -f /path/to/script.clj --bb-file "/path/to/bb.edn, I think this would solve the problem

(BTW, this works in Python, since a venv can be called anywhere if you, just by calling your script with path/to/venv/bin/python)

sorry if this is a common qustion, i'm new to this is there an existing time expiring cache that is happy in babashka?

(def devices (cache/ttl-cache-factory #{} :ttl 5000))

Yes, actually, I wish to use it as a command in a pipe. In the repository I prepared a demonstration. Let's say you have shifted-ordered files. So, text5.txt, text6.txt, ..., text10.txt. You can rearrange them removing this shif-value of 5. And, you can change their regex "text", as well as the extension ".txt". All this with this one-liner which is a defcmd-macro (from closh). Then, let's say you want to partition these files in sub-folders with determined regex base-name. You can then use another simple pipe, twoliners. (to reproduce the results, you can go to ./sh-scripting/, run sh lupo-setup.sh , then, cd ../test/ , and run the following:

$rename-test "test" ".txt" 5 "demo" ".org"

$ls |> (map #(file-ids % "demo" ".org")) |> #(populate-key-map (create-range [0 3 7 10]) % 1) |> #(create-populate-dir % "dir-demo" "demo" ".org")

> So if we could have something like bb -f /path/to/script.clj --bb-file "/path/to/bb.edn, I think this would solve the problem Actually, this would need to be a --context /path/to/project, since the :path (and probably other things) on the bb.edn file are relative

maybe launch bb with the --classpath set to the place with the src etc and launch it from the place bb.edn is? Im not sure if the bb.edn path could be specified :thinking_face:

I think this could work, however I would need to load my dependencies manually somehow

i think the bb.edn is also picked up from the --classpath afaik

Huh...

Let me try

could be wrong 😅

Well, I can set my --classpath to path/to/project/src, but this will not load the dependencies

Using --classpath path/to/project will not find my project namespaces

Passing both (`--classpath path/to/project:path/to/project/src`) also doesn't work

how are you invoking bb? providing a particular ns with the classpath?

As it is, I need to load all the functions in closh repl, to use these one-liners - which defeats the purpose a bit. I could make some tweaks and write it in "pure babashka" syntax, and leave closh aside. Currently, I'm maintaining both versions in parallel. I guess I could, then, write the commands to execute inside the script - as I noticed that was the purpose, as you spoke. But, going further in this problematic, do you see an easy way I could load these namespaces and use in a pipe | bb -e (ns-function) fashion?

I actually call the script itself, that loads a namespace from the project, that them loads the -main function 😅

Because I need to do some setup (and I try to avoid as much bash as possible)

But well, I think parsing the :deps from bb.edn should work

It’s hard to understand the exact problem you are trying to solve. Can you try to isolate the issue? Is this specific to closh or is this also happening in the bash shell?

yeah i could imagine shoving the entry point into bb.edn and calling the code from that instead of the scripts?

For piping process it’s good to have a look here https://book.babashka.org/#_input_and_output_flags

invoke bb with a classpath and a task name maybe

I don't think this would work anyway in my case since to read bb.edn, AFAIK you need to be in the same directory as the bb.edn is created

right, im not sure of this, @borkdude should help better 🙏:skin-tone-3:

I eval a babashka file. I want to call, now, the functions which were loaded in this eval. I can't. That's basically the isolated issue there.

(irrespective of what shell I'm in)

ok so first thing is that bb loses it’s state between 2 calls. So the first one is unrelated to the second one

Let me see if I can give you a working version

I appreciate it @jeroenvandijk

You are missing a bb.edn file in the devibrary dir

Here is some background https://book.babashka.org/#project-setup

This works 😄

#_(
   export PROJECT_DIR="$(cd "$(dirname "$(dirname "$0")")"; pwd -P)"
  
   exec bb -f "$0" -- "$@"
  )

;; Add debugging helpers and dependencies
(require '[babashka.fs :as fs]
         '[babashka.deps :as deps]
         '[babashka.classpath :as cp]
         '[clojure.edn :as edn])

(defn- file-from-project-dir [filename]
  (str (fs/path (System/getenv "PROJECT_DIR") filename)))

(deps/add-deps (edn/read-string (slurp (file-from-project-dir "bb.edn"))))
(cp/add-classpath (file-from-project-dir "src"))

bb -m devibrary.core/foo 2
2 Hello, World!

That should work when you add a bb.edn file like this one

{:paths ["src"]}

Kinda hacky? Maybe. But not bad (better than bash)

The other function can be called like bb -m devibrary.partitioning/create-populate-dir 1 2 3 4

I have it working here

➜  CloshBashika git:(main) ✗ git status
On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	devibrary/bb.edn

nothing added to commit but untracked files present (use "git add" to track)

@pedrogbranquinho Hope this is what you needed

Yes, it is! I will dive more deeply in the documentation, also. Because, certainty I will be using it in daily use. Thank you @jeroenvandijk and @borkdude

np @pedrogbranquinho. Hope you are having fun 🙂 Also cool to see you are using Closh

yeah i guess having the ability to specify the bb.edn path should be the best option

@thiagokokada I don't know if this solves your problem, since I didn't read the entire thread, but: > bb -f /path/to/script.clj --bb-file "/path/to/bb.edn you can do this:

BABASHKA_EDN=/path/to/bb.edn bb -f path/to/script.clj

ahh

but note that BABASHKA_EDN is not documented and only used for testing right now

we were missing the --bb-edn flag and missed it in the help too

there is no --bb-edn flag

I was citing kokada

ah sorry i misread

yeah i see the env var 😅

would it be nice to add a cli opt for the edn file? most tools have it for a non standard path?

I think there is a reason that deps.edn doesn't support this. Since you create ambiguity about :paths. Are they relative to the bb.edn file or the local dir?

So I'd like to keep it under the radar for now maybe

Huh... It doesn't seem to work for me It does seems to load the :deps, but not the :paths

exactly :P

we can model it like docker? it uses -f for a non standard path to the Dockerfile and takes a context path

Well, I am quite happy with my solution if this is not possible

I'm not even sure what the problem is, so right now: no :)

😄

> we can model it like docker? it uses -f for a non standard path to the Dockerfile and takes a context path Yeah, this would be ideal for me

But if it isn't possible it seems fine too

Perhaps you can lay out the issue in clear words in a Github discussion

I am kind of overflown with Slack and Twitter threads right now

At least it is possible to manipulate the classpath on those cases

and I will take a look at this later

I can start it, maybe you can add to it @thiagokokada?

Ok

Thank you @rahul080327 @borkdude

Note: please use discussions, not issues, so we can have some back and forth first

Should i add it to the bb tasks thread?

Please start a new one

got it

https://github.com/babashka/babashka/discussions/869

Can you remove the reference to BABASHKA_EDN? It's an internal detail, I think it's better if nobody finds it there and start using it

So the way I'm reading that issue now. My initial response is: just write a bash wrapper script that cds into the right directory and then calls bb

@arthur currently we don't have core.cache / core.memoize in babashka. not sure what the vanilla Clojure / Java answer to this should be

perhaps just an atom with maps with timestamps

Makes sense. I'm looking at it from the lens of a task runner like Gradle, make etc where we can specify the path to the conf, when not in pwd

ok

There's a tiny error in the docs here https://babashka.org/fs/babashka.fs.html#var-delete. Think that should be (delete f) instead of dir.