Clojurians Log v2

@borkdude Can I join babashka pods paths? I would like to take some default pods from ~/.babashka and some other from /.native-deps

@karol.wojcik I'm not sure what you want to do with the joined path?

I want to pack babashka runtime as an AWS Layer with babashka-aws as a default pod. Default pods will be stored in .babashka folder, but still I want allow the users to upload their own pods. Those extra pods will be stored at .native-deps directory, therefore I want babashka to look either in .babashka or .native-deps directory for pods 🙂.

@karol.wojcik Yes, sure. The way you would have to call those pods must be using the filename then instead of the fully qualified symbol, right?

I don't want the user to specify pods via the filename. From the user perspective it doubles the work to deploy an artifact. 1. Use fully qualified symbols. 2. Download and pack pods (soon via bb pods --download). 3. Rename dependencies to use filenames instead of fully qualified symbols. What I would like from babashka side is to allow extra paths for pods (like classpath).

Maybe via environment variable. Something like: BABASHKA_PODS_LOOKUP

@karol.wojcik A fully qualified symbol implies the pod is downloaded from the registry (or in the expected already downloaded location). The other option is to use the pod's filename and then it will be looked up from the system path. There is no combination of these two.

I don't want to combine those two.

I want to specify an extra "already downloaded location" for fully qualified declared pods.

The change would be in resolve function: https://github.com/babashka/pods/blob/82aa3627106181a0ad58c289cd45129603e4fa24/src/babashka/pods/impl/resolver.clj#L159

We could use "or" in 169 and 170 line for custom babashka pods paths.

I would love to make a PR for it if you would like me to.

I don't understand the use case. Why not use the location that is already supported?

Ok let explain myself a little bit. Here is how babashka runtime looks like. It's packed together with following things: .babashka, .m2 and bb executable

#!/bin/sh

set -e

export _JAVA_OPTIONS=-Duser.home=.
export BABASHKA_DISABLE_SIGNAL_HANDLERS=true
export XDG_CACHE_HOME=.

./bb -cp "$(cat classpath)" -m "$(cat ENTRYPOINT)"

I don't understand this part: > to which items from user lambda release artifact will be merged. Which means that user cannot pack babashka pods in .babashka, because it will remove all dependencies declared! What is it, merge or replace?

Are you not able to add to directories in layers, do they always replace? That seems odd to me

Imagine that you unzip two zips in place which both have the same directory inside.

I think that should merge, but doesn't it?

Hmm.. Actually i did not test it, but my intuition says no :D will check it out!

let's put it to the test

I tested it and it merges (like I would expect)

merging is not the right word: it just creates files into the directory structures, but doesn't remove any existing ones. on overwrite zip will prompt you

Here you have foo1.zip and foo2.zip. Both contain a dir foo with different files. After unpacking both, the foo dir has both files.

I hope that aws lambda does the same.

I will check it out and let you know!

I think docker works the same way, but it would be good to check

If someone wants to have some fun, perhaps one could find out how to get zsh (or bash) completions for a project specific bb.edn?

Hello. I'm converting some scripts to bb task. I want to use shorthand syntax for functions. But getting an exception: "Exception in thread "main" java.lang.RuntimeException: No dispatch macro for: (" any hint?

(run!  #(cp-to % dest) (fs/glob rp "**.{css}"))
;;(run! (fn [f] (cp-to f dest)) (fs/glob rp "**.{css}") )

@fnumatic Since bb.edn is an EDN file you cannot use #(...) unfortunately.

if necessary you can split out functions into a different file and put it on your classpath using :paths

@borkdude ok, thanks.

off topic: a nasty trick to inject code into every namespace (by modifying clojure.core) https://github.com/babashka/babashka/issues/799#issuecomment-825654751

is there a short way to just copy a file to a specific directory? my bash script right now

cp $RP/index.html $DOCS

Does (fs/copy "foo" "bar") work?

(fs/copy (fs/path (System/getenv "RP") "index.html") (System/getenv "DOCS"))

it's not really short, but that should do it

you could also define a variable in :init to make it shorter

{:init (do (def docs (System/getenv "DOCS")))}

nope. System vars are not the problem. I just cannot choose a destination dir. I always have to use explicit destdir + destfilename.

hmm, let me check

this is my current workaround in a let block

cp-to (fn [f dir]  
         (fs/copy f (fs/file dir (fs/file-name f)) 
          {:replace-existing true}))

yeah this is annoying and should be fixed.

good work, by the way

Thanks. I see the behavior is similar to:

(require '[<http://clojure.java.io|clojure.java.io> :as io])

(io/copy (io/file "file") (io/file "foo"))

Maybe we should have a function copy-into or something which copies a file into a dir

so it's not a bug at least, but slightly annoying

@fnumatic btw if your use case it to copy a dir into another dir, then you should consider using fs/copy-tree

https://github.com/babashka/fs/pull/23

@fnumatic please add tests. As I currently see it, the function calls itself infinitely

also some bikeshedding: copy-into or copy-to? any preferences anyone?

yes, omg

would prefer copy-to

yeah, I think so too

well, copy-into is less ambiguous perhaps. copy-to could still mean that you copy file X to file Y

ok copy-into then https://github.com/babashka/fs/pull/23/commits/df0108afc5b74b4d1119c3e31c57973b479704c8

Copy into

Whoops. Whenever I'm not certain about a name, I just toss a coin on Twitter. https://twitter.com/borkdude/status/1385662077600243715

cp

(a new challenger appears)

yeah, I think copy-to is more esthetically pleasing, but copy-into is less ambiguous, so even if the poll is in favor of copy-to I may still choose copy-into based on arguments

cp is ambigious in itself

oh, so there's a difference?

hm

I always liked Ruby's FileUtils module where all methods were named after shell commands

(mkdir, mkdir_p etc)

cp behaves like foo target-dir: copy into the dir, and cp foo target-dir/non-existing-or-file creates or overwrites the file, which over time you just learn

what's mkdir_p?

create directory if doesn't exist, otherwise report success

oh mkdir -p equivalent

or create a whole path structure mkdir -p ...

yep

I sorta feel that the destination should come first, given it's ...into and ...to

i.e., (fs/copy-into "quux/dir" "foo/bar/baz.txt")

copy x into y

yes, I'm aware of the implicit

I think destination first is confusing.

but it reads better and in my mind feels better when the destination comes first

I think the unix world disagrees

I wouldnt expect that to be the behaviour

you look at it and you think "I'm copying into /foo/bar/baz.txt"

the argument names solve that problem, copy arguments usually go from src to dest, left to right

how about (fs/copy-from....)

that is a perfectably reasonable choice too

(fs/copy-from "foo/bar/baz.txt" "quux/dir")

(fs/copy-from "foo/dir" "quux/dir")

The second example is already supported by copy-tree

it was an example

sorry, I don't find it intuitive :)

copy-from has better readability

I'm copying from x to y

it doesn't explain whether you are copying from file X to a new file Y or into a dir Y

that's irrelevant

do you care if you do cp x y

babashka.fs doesn't try to emulate bash, it is based on java.nio.file and there such semantics matter

Everything is a file 😉

e.g. fs/copy is based on java.nio.file.Files/copy

THe problem you are introducing the -into or -to is that the explict reading of it in english sets up a wrong premise

so we already have a "style guide" that we have to stick to

if you read out (fs/copy-into "foo/bar/baz.txt" "quux/dir") out loud, you'll see that it just reads wrongly,

ditto with to

(> 5 6): 5 greater than 6 (copy-into foo bar) copy foo into bar makes sense to me

I'm copying into foo bar

foo copy-into bar

it's lisp

x op y is written as (op x y) in lisp

I'm aware of lisp

but you're trying to shoehorn a LISPy schematic into a file system action

now this is a better argument of putting the destination first: https://twitter.com/danielglauser/status/1385667548620869633 I don't agree. this is how you write functions / predicates in lisp in general.

all I'm saying is that it reads wrong if you read it out, in both choices, where as the -from actually does precisely what the action is doing.

I think often the order of arguments can be counter-intuitive and you can explain it both ways, I'll give you that

(apply copy-foo dir [x y z])

(run #(fs/copy-to-dir % dir) ...)

Maybe copy-to-dir is the most explicit name we can have and, to be compatible with the other copy functions, I think it's least confusing if we put the order the same, src -> dest

or maybe not include it... :)

funny enough, me.raynes.fs didn't have this function

does anyone know how python or Ruby calls this function, if available?

Just copy in python it works for both copying into a dir and creating a new file by name

In fs copy already is based on Files/copy so we can't alter that behavior I think (better stick to the standard here). It's better to create another function, it's only the name that is difficult

ok, let's be open minded: Maybe we could change fs/copy to copy the file "into" a dir if the dest is a dir... and deviate a bit from java.nio.files. What do y'all think?

@borkdude my bad. Layer goes to /opt/var. User packed artifacts go to /opt/task. Seems that I need babashka pods classpath. ;(

That would be even better. My goal is, to get rid of various shell scripts. As of now, just copy file to file is counterintuitive. Everything is a file, as @qmstuart said.

ok, let's do it

@karol.wojcik Can you make an issue in the pods repo to clearly explain the case so I can give it some thought?

Sure

Thanks! I'll give it a night's sleep and will look at it this weekend

ok, fs/copy now accepts dirs as dest: https://babashka.org/fs/babashka.fs.html#var-copy published as 0.0.4 and will be available in a new bb near you soon

I took a shot at this and have a prototype working as documented here https://github.com/russmatney/bb-task-completion

Right now it just stringifies the task definition, so looks like

I'm not sure the best approach for a few things - i listed those in the readme - if you have thoughts on next steps, let me know and i can take a shot at a PR, but feel free to run with this however you see fit

Awesome, I'll take a deeper look tomorrow :)

Note that there is also :doc in tasks which you could use to print

although I haven't used it in the case of antq's bb.edn

gotcha - i'll update to prefer that if it's present

Here it is: https://github.com/babashka/babashka/issues/801 ;)

The stringified task definitions break with some tasks:

{:tasks
 {all {:depends [lint-eastwood]}

  lint-eastwood
  {:task (clojure "-M:dev:eastwood '{:source-paths [\"src/main\" \"src/test\"}]'")}}}

thanks for trying it! good find - we'll probably need to escape those quotes