Clojurians Log v2

Hi, I need some rubberducks I think 😄 I want to use echo within a shell command, but want to pass -en as a Parameter. But it seems as if the shell command does not interpret -en as Parameter

(sh "echo" "-en" "foo\n\n")
=> {:exit 0, :out "-en foo\n\n\n", :err ""}

process, which should then be used for pipelining, provides similar results:

(-> (process '[sh -c "echo -en $FOO"] {:env {:FOO "BAR"}}) :out slurp)
=> "-en BAR\n"

Expected output: Something like this, without -en:

@n2o The issue may be that echo is both a built-in shell command and echo being a binary on your system. Shelling out from Java always means it's calling the binary

What is the purpose of echo -en foo ?

For piping you can find some advice here: https://clojurians.slack.com/archives/CLX41ASCS/p1620595628139900

The complete script looks something like this:

(-> (process '[echo -en $SIGNATURE] {:env {:SIGNATURE "something"}})
    (process '[openssl sha1 -hmac $SECRET -binary] {:env {:SECRET "some other stuff"}})
    (process '[base64])
    :out slurp)

Note that this does work for me:

$ bb -e '@(:out (babashka.process/process ["echo" "-n" "foo\n\n"] {:out :string}))'
"foo\n\n"

Why don't you just feed the env var to :in ?

I tried several different things 😄 also using :in. But I think the first problem is the en parameter

Try:

@(:out (process ["openssl" "sha" "-hmac" <your-secret> "-binary"] {:in "something" :out :string})

you don't have to pipe something via echo to stdin of another process, you can do that directly

yes, i hoped so, but the resulting base64 encoded string differs from the one when I put it directly on the console. And I was figuring out the root cause of this 😬

thanks, I’ll give it a try.

btw, there is also https://github.com/babashka/pod-babashka-buddy if you don't want to shell out

Yep, I think I’ll switch to some of your pods, seems easier.

$ bb -e '@(:out (babashka.process/process ["openssl" "sha1" "-hmac" "dude" "-binary"] {:in "foobar" :err :inherit :out :string}))'
"/��B1SKsi�Ӕ @�\n��"

$ bb -e '(-> (babashka.process/process ["openssl" "sha1" "-hmac" "dude" "-binary"] {:in "foobar" :err :inherit}) (babashka.process/process ["base64"] {:out :string}) deref :out)'
"L6UQkkIxU0tzaROn05QgQKkKg90=\n"

(might want to remove the newline)

oh jeah, there it is… Now it is working! Thanks!

hmm, I see it is different yes

but that’s the base64 encoded string I was looking for (on my machine)

Is there something in the babashka/sci universe that can print a list of the names of the :aliases in the nearest deps.edn file?

perhaps it's because of the newline of openssl

neh

@duncan540 not really built-in, you can just parse the edn

@n2o oooh:

$ bb -e '(-> (babashka.process/process ["openssl" "sha1" "-hmac" "dude" "-binary"] {:in "foobar\n" :err :inherit}) (babashka.process/process ["base64"] {:out :string}) deref :out)'
"89CMnuc40b7mfZpt9domvGCGIm0=\n"

$ echo foobar | openssl sha1 -hmac dude -binary | base64
89CMnuc40b7mfZpt9domvGCGIm0=

you see, that's the same...

yes yes, I’ve got it now working on my machine 🙂

:)

I can also produce the same string. Thank you very much! although I currently do not see the difference from my previous approaches :thinking_face: 🤷

the difference I found was that when you add a newline to the input it's the same as on the command line

probably echo also adds a newline to the input

but I was working in a babashka script and defed the symbol with the newlines

so they were always present in my script

This is without the newline:

$ echo -n foobar | openssl sha1 -hmac dude -binary | base64
L6UQkkIxU0tzaROn05QgQKkKg90=

and that's the same as the first example we had

$ bb -e '(-> (babashka.process/process ["openssl" "sha1" "-hmac" "dude" "-binary"] {:in "foobar" :err :inherit}) (babashka.process/process ["base64"] {:out :string}) deref :out)'
"L6UQkkIxU0tzaROn05QgQKkKg90=\n"

so the confusion comes from the newlines :)

Okay, I understand. Great 🎉 again something learned

I'm doing it from a Python context, so the best I can think of so far is to use the use Python stdlib to find the nearest deps.edn on the filesystem, then use your jet tool to transform that to JSON, then pick out the alias names from that

and now piping works as expected in babashka 🥳 thanks

@duncan540 I have an example here to download all deps from all aliases: https://github.com/babashka/babashka/blob/master/examples/download-aliases.clj

oh Python, hmm, I guess you could just shell out to bb from python as well

or use #jet that's also possible

yeah, I'll be able to cobble something together, thanks for the pointer 🙂

Sublime Text GUI plugin:

Awesome :)

I'm having a problem moving my tasks into a local project root. 🧵

If I have a bb.edn with:

{:deps
  {some.lib {:local/root "lib/some.lib"}}

  :requires ([some.lib.foo :as foo]) ;; foo is requiring lread.status-line

  :tasks {bar (foo/bar)}}

Where lib deps.edn has:

{:deps
  {lread/status-line {:git/url "<https://github.com/lread/status-line.git>"
                      :sha     "35ed39645038e81b42cb15ed6753b8462e60a06d"}}}

Then we get an error when running bb bar:

Type:     java.lang.Exception
Message:  Could not find namespace: lread.status-line.

But if I add that status-line dependency directly to bb.edn's :deps everything works. Am I doing something wrong or is there some bug between the interplay of deps and tasks?

no, this is by design. bb deps should go into bb.edn

this used to work by mistake

but the Clojure deps should not be mixed with bb deps imo

hmm, yeah, so this did used to work and I'm not going crazy; I depended on the behavior :]

I don't understand this design decision - if I would like to publish a task as a reusable piece of code that can be used by multiple bb.edns, how should I go about it?

yeah, it was accidental

then you could make that reusable piece of code a library I guess?

and that lib would be imported via :deps in bb.edn, correct?

oh, you would like to publish a task as a reusable piece of code, I didn't anticipate that use case

I mean, the task name etc?

or just some functions?

no, just some functions that a task can call

yes, you can use any library in :deps, same as deps.edn, but specific for bb

so you can e.g. use :local/root or :paths to get at the shared code or if you want to publish it, use :git/url

yep, that's what I'm trying to do

it's like bb is not resolving the deps inside the :local/root deps.edn when calling tasks

bb isn't resolving deps from deps.edn at all

or do you mean, the deps.edn from the local/root library?

that should work

no, when bb has an entry {:deps {foo :local/root "foo"}} -> it's not resolving the "foo/deps.edn"

it should be {:deps {foo {:local/root "foo}}}

it might be a caching issue, maybe try rm -rf .cpcache; rm -rf ~/.clojure/.cpcache

🤦 that did it

it was the caching issue?

Yep, state. Mutable state is always the problem.

maybe bb needs a -Srepro ? XD

cache is turned off with -Sforce but yeah, we should allow passing args down to the deps tool

anywho, thanks @borkdude - I was starting to doubt my sanity

:) you can debug the classpath with (babashka.classpath/get-classpath)

I used to always add -Sforce -Srepro to all my Makefiles out of habit just to reduce these vectors of confusion

(although that comes with its own set of drawbacks)

if you have some ideas how babashka could pass down args to the deps system...

bb --deps -Sforce --deps -Srepro ... run foo

or directly support the relevant CLI args...

bb -Sforce run foo

some prefix would be good probably to not get conflicts with prior or future bb args

like -J-Dfoo=bar but then something else

-C-Sforce

I couldn't install babashka on ubuntu (pop-os)

r@pop-os:~/dev/babashka/babashka$ script/uberjar && script/compile
openjdk version "1.8.0_292"
OpenJDK Runtime Environment (build 1.8.0_292-8u292-b10-0ubuntu1~20.04-b10)
OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)
Syntax error (ClassNotFoundException) compiling at (babashka/impl/classes.clj:345:20).
java.lang.ProcessHandle

Full report at:
/tmp/clojure-2871116251297672484.edn
Error encountered performing task 'do' with profile(s): 'base,system,user,provided,dev,xml,yaml,core-async,csv,transit,httpkit-client,httpkit-server,core-match,hiccup,test-check,rewrite-clj,selmer,reflection'
Suppressed exit

https://pastecode.io/s/4qfA31CkP3

you need graalvm 21.1.0 CE JDK11

https://github.com/babashka/babashka/blob/master/doc/build.md

btw why are you trying to build it yourself rather than downloading the binary?

yes I was trying to build myself somehow I didn't see the binaries 🙂

thank you

I installed with asdf now

ok, just letting you know that bb is available as binaries, so only if you are developing you have to build it yourself

:thumbsup:

Are there any autocompletion features for the shell to access aliases from the Task Runner? If not, I’d love to have more support in the shell to have it tab-autocompleted 😄

@n2o there have been some experimental approaches, I'd love to see contributions for this so we can support bash and zsh (and others, but I'm using zsh)

@n2o https://github.com/russmatney/bb-task-completion

ah, perfect. I’ll take a look at this, Sadly, I am using fish, but maybe we can work on that / I can help you with that

here's also some other approach: https://github.com/jeroenvandijk/matryoshka

I'm fine with supporting any shell, but I haven't had the time to look into this myself and I think this would be a good place to contribute

Perhaps the completion script can be written in bb itself?

or can bb emit some hints to the shell itself via some subcommand? Not sure how this should work

that’s what I was thinking about 😄

Shells should be able to ask to a binary: bb --complete <substring> and it will give a list of completions, or something like this right

Jep, sounds great, thanks for the pointers. interesting that these repos are recently created

https://twitter.com/avidrucker/status/1392202960256606208