babashka

https://github.com/babashka/babashka. Also see #sci, #nbb and #babashka-circleci-builds .
unbalanced 2020-09-10T01:05:50.149800Z

struggling a bit to get the output of a ProcessBuilder workflow (my Java is weak)

unbalanced 2020-09-10T01:07:13.150800Z

(defn find-files! []
 (let [proc (-> (ProcessBuilder. ["find" "./some-dir"])
                 (.redirectOutput ProcessBuilder$Redirect/INHERIT)
                 (.redirectError ProcessBuilder$Redirect/INHERIT)
                 (.start))
        proc-input (.getOutputStream proc)]
    (into [] ????)))

unbalanced 2020-09-10T01:07:40.151Z

(ripped off from https://github.com/borkdude/babashka/blob/master/examples/process_builder.clj)

borkdude 2020-09-10T06:54:13.151900Z

@goomba Not sure where you're going with that example: you want to feed input to the process?

unbalanced 2020-09-10T13:10:50.157200Z

It's probably safe to assume that I'm totally misunderstanding ProcessBuilder and the output streams. My goal here is to capture the output of a long running process, like a large "git clone", and capture the output into a string or a vector of strings.

unbalanced 2020-09-10T13:11:30.157500Z

In some previous work I tried using shell/sh, but it doesn't seem suitable for long running tasks

unbalanced 2020-09-10T13:12:07.157700Z

I tried looking into the Java API for how to work with the redirects of the ProcessBuilder but I got lost in the sauce

borkdude 2020-09-10T13:17:27.157900Z

yeah, let me try to write that

borkdude 2020-09-10T13:18:23.158100Z

What's not clear in your example: would like also to feed input to that process?

borkdude 2020-09-10T13:18:33.158300Z

And why do you redirect output? This means it will be sent straight to stdout

borkdude 2020-09-10T13:21:29.158500Z

@goomba

(require '[<http://clojure.java.io|clojure.java.io> :as io])

(defn find-files [path]
  (let [proc (-&gt; (ProcessBuilder. ["find" path])
                 (.start))
        output (.getInputStream proc)]
    (future
      (run! prn (line-seq (io/reader output))))))

@(find-files (first *command-line-args*))
$ bb /tmp/foo.clj /tmp/version-clj
"/tmp/version-clj"
"/tmp/version-clj/test"
"/tmp/version-clj/test/cljx"
"/tmp/version-clj/test/cljx/version_clj"
...

borkdude 2020-09-10T13:22:06.158800Z

Confusingly, you get the output of the process using .getInputStream because it returns an inputstream

❀️ 1
borkdude 2020-09-10T13:22:33.159Z

You might also want to read this: https://book.babashka.org/#child_processes

❀️ 1
unbalanced 2020-09-10T13:24:44.159500Z

sweet! thank you

borkdude 2020-09-10T13:25:32.159700Z

This is an in progress library which should make all of this easier: https://github.com/babashka/babashka.process

borkdude 2020-09-10T06:57:26.152100Z

You can also take a look at https://github.com/babashka/babashka.process

borkdude 2020-09-10T08:20:45.153200Z

Any thoughts on this discussion? https://github.com/http-kit/http-kit/issues/442#issuecomment-690040016. If you have, consider posting your thoughts there.

borkdude 2020-09-10T10:05:18.153900Z

I also made a #http-kit channel now

dharrigan 2020-09-10T11:13:38.154800Z

I've added a simple example of using Babashka to rewrite an existing bash script into Clojure to perform spotifyd notifications on Linux: https://github.com/dharrigan/spotifyd-notification

πŸ‘ 2
borkdude 2020-09-10T11:15:06.155Z

Cool! Feel free to add to https://github.com/borkdude/babashka/blob/master/doc/libraries.md which also features projects

dharrigan 2020-09-10T11:15:15.155400Z

kk

dharrigan 2020-09-10T11:21:25.155800Z

done

borkdude 2020-09-10T11:23:24.156Z

thanks!

dharrigan 2020-09-10T11:26:05.156200Z

np

unbalanced 2020-09-10T13:39:20.162100Z

Curious about techniques for Babashka applications-as-scripts. For instance, I created two files, config.clj and core.clj (which requries config.clj with a deps.edn and the traditional project structure. However from a non-root directory when I tried bb -f ../path/to/src/core.clj , I got the cannot find my-project.config.clj error message

unbalanced 2020-09-10T13:39:39.162500Z

If I wanted to have a more complex setup but still use it as a script, would I be looking at making an Uberjar?

borkdude 2020-09-10T13:39:54.162700Z

Either an uberscript or uberjar yes

borkdude 2020-09-10T13:41:37.163100Z

@goomba or put the dir with the scripts on the classpath correctly using -cp

unbalanced 2020-09-10T13:42:04.163300Z

root dir?

unbalanced 2020-09-10T13:42:12.163600Z

that has the deps?

unbalanced 2020-09-10T13:42:41.164Z

I gotta tell you @borkdude this damn project is so much damn fun hahaha

borkdude 2020-09-10T13:44:00.165400Z

Good to hear :) So if your sources are in &lt;project&gt;/src then classpath should be src, but you then go into .. then your classpath should become &lt;project&gt;/src but maybe using absolute paths for that is the least confusing

unbalanced 2020-09-10T13:45:40.167100Z

understood. So regardless of what location I'm in, if I set the -cp to the src of the project (relatively or absolutely, as long as it is correct). Does that mean then that if I have src/project_name/*.clj files, and the namespaces are (ns project-name.something), it should still be src?

borkdude 2020-09-10T13:46:04.167500Z

no, you can put it anywhere you want, and make the classpath anything you want

unbalanced 2020-09-10T13:46:04.167600Z

to be more specific:

$ tree 
.
β”œβ”€β”€ deps.edn
β”œβ”€β”€ Makefile
└── src
    └── react_deployment
        β”œβ”€β”€ config.clj
        └── core.clj

unbalanced 2020-09-10T13:46:39.168200Z

so still point to src with the classpath?

borkdude 2020-09-10T13:46:40.168300Z

you can also make the classpath the current directory and have no src dir for example

borkdude 2020-09-10T13:47:14.169200Z

foo/bar.clj becomes (ns foo.bar) yes.

unbalanced 2020-09-10T13:47:19.169400Z

perfect

unbalanced 2020-09-10T13:48:44.169600Z

yep that did it

unbalanced 2020-09-10T13:48:53.169900Z

I can't wait to try out some crazy socket/async stuff

tzzh 2020-09-10T15:10:35.174200Z

Hey, I have been writing a bit of babashka lately (it’s absolutely amazing btw) I was wondering what was the best way to interact with AWS with babashka. I have been doing some stuff with generating and running commands with clojure.java.shell but I was wondering if there was a way to either use the java sdk or if a library to integrate with the AWS api or something (couldn’t find something here https://github.com/borkdude/babashka/blob/master/doc/libraries.md)

tzzh 2020-09-10T15:13:36.174700Z

(or would that be a good candidate for a pod or something ?)

borkdude 2020-09-10T15:14:40.175800Z

@thomas.ormezzano It's been asked a couple of times before. I think you could maybe wrap the aws using clojure.java.shell or maybe create a pod around the Python AWS SDK.

borkdude 2020-09-10T15:15:11.176200Z

This is an example of a pod in Python: https://github.com/babashka/babashka.pods/blob/master/examples/pod-lispyclouds-sqlite/pod-lispyclouds-sqlite.py

tzzh 2020-09-10T15:15:28.177Z

ok amazing thank you will take a look at this :thumbsup:

nate 2020-09-10T18:17:23.178Z

very interesting idea to write an AWS pod in another language, maybe Go would be a good candidate too

borkdude 2020-09-10T18:17:52.178600Z

Yes, Go would also be a good candidate, since it also compiles to native and should have very good startup

borkdude 2020-09-10T18:18:19.179Z

I'm not using AWS myself that much, so I'm leaving this one up to the babashka community :)

borkdude 2020-09-10T18:19:54.179400Z

And where I can I'll try to help of course

lukasz 2020-09-10T18:24:13.181300Z

This is v. relevant to something I'll be working on soon, no promises but I might tackle this

πŸ‘ 2
lukasz 2020-09-10T18:24:19.181500Z

aws pod

lukasz 2020-09-10T18:24:21.181900Z

or something like that

viesti 2020-09-10T18:24:32.182100Z

I did some hack with lambda, attaching babashka and awscli layers, shelling to aws cli worked nicely, although awscli is a python thing so not that stand-alone, compared to go

borkdude 2020-09-10T18:25:26.183Z

@lukaszkorecki The pod could live under the babashka org when it's more or less done, if you wanted to. Also ok to keep it under your company org, etc.

lukasz 2020-09-10T18:26:22.184Z

@borkdude I'm still at the gathering requirements stage, so it's not been fully decided if we're going to go with Clojure (jvm) or BB. But if we go with BB, it's def going to be open source

borkdude 2020-09-14T12:31:16.219300Z

Could be both. If you're going with the JVM library and make it GraalVM compatible, it could also be wrapped in a pod

lukasz 2020-09-14T13:37:47.219500Z

True! I'll keep you updated and bug you with questions ;-)

borkdude 2020-09-14T13:44:53.219700Z

:thumbsup:

Michael W 2020-09-10T23:35:40.184700Z

Have you guys seen this? I am using it for some minor stuff but it's got the entire set of services for AWS. https://github.com/cognitect-labs/aws-api