babashka

https://github.com/babashka/babashka. Also see #sci, #nbb and #babashka-circleci-builds .
walterl 2020-11-17T00:53:10.343600Z

What's the babashka.process way to do echo -e "foo\nbar\nbaz" | fzf, such that fzf uses the piped input, and can still read input from the keyboard? If :in is :inherit-ed, input can't come from somewhere else; if :in is not :inherit-ed, fzf can't read from the keyboard

walterl 2020-11-17T01:04:21.348200Z

It dawned on me that I don't really know how the above command's stdin is managed anyway. I hypothesized that stdin is read from the pipe (the echo command), the pipe is then closed, and then (re-?)connected to the tty. That appears to be wrong, because /proc/<pid>/fd/0 was clearly pointing at a pipe, after the command was accepting input from the keyboard. So now I'm thinking that stdin is read from the pipe only, and keyboard input is then read directly from /dev/tty. But if that's the case, why can't fzf read from /dev/tty when stdin isn't inherited from bb?

borkdude 2020-11-17T09:49:47.349500Z

Nice! I changed the dotimes part with (println (slurp *in*)) so you can pipe a file to this script

borkdude 2020-11-17T09:59:38.349700Z

@clojurians-slack100 Using babashka.process:

(require '[babashka.process :as p])

(defn fzf [s]
  (let [proc (p/process ["fzf" "-m"]
                        {:in s :err :inherit
                         :out :string})]
    (:out @proc)))

(fzf (slurp *in*))

walterl 2020-11-17T14:18:49.351400Z

Thanks @russell.matney, @borkdude! It looks like it's the :err :inherit that makes the difference. It kinda makes sense that it should be inherited, but I'm still not all that clear on why it has this effect. Do you have more insight?

walterl 2020-11-17T14:19:07.351600Z

Either way, I'm very glad that I can now use fzf from bb! \o/

borkdude 2020-11-17T14:29:21.351800Z

@clojurians-slack100 :err :inherit only influences the output, so the rendering in the screen that fzf does

borkdude 2020-11-17T14:29:58.352Z

From Java you can only send one inputstream. But I think fzf will consume it until EOF and then maybe it will switch to reading the keyboard. This is my guess.

walterl 2020-11-17T14:31:36.352300Z

It seems like there's more than that going on, though: if you don't include :err :inherit, fzf understandably won't output anything, but it will also not respond to any input.

borkdude 2020-11-17T14:33:15.352500Z

@clojurians-slack100 it does, at least on my machine. just press a couple of up arrows and then hit enter

borkdude 2020-11-17T14:33:17.352700Z

$ cat src/babashka/main.clj | bb examples/fzf.clj " (:require\n"

borkdude 2020-11-17T14:33:27.352900Z

^ I commented out :err :inherit there

walterl 2020-11-17T14:34:03.353100Z

Interesting! On my Ubuntu 18.04 it definitely doesn't respond

walterl 2020-11-17T14:34:18.353300Z

Not even ^c

walterl 2020-11-17T14:34:27.353500Z

I have to kill bb

borkdude 2020-11-17T14:34:49.353700Z

Maybe that's an old version of fzf?

borkdude 2020-11-17T14:35:00.353900Z

$ fzf --version
0.24.3 (brew)

walterl 2020-11-17T14:35:23.354100Z

fzf --version
0.24.1 (d4c9db0)

walterl 2020-11-17T14:39:04.354300Z

Does the same with 0.24.3 o_O

walterl 2020-11-17T14:39:27.354500Z

I'm testing with this:

(require '[babashka.process :as p])

(defn fzf [s]
  (let [proc (p/check (p/process ["fzf" #_"-m"]
                                 {:in s #_#_:err :inherit
                                  :out :string}))]
    (:out proc)))

; (fzf (slurp *in*))
(fzf "foo\nbar\nbaz")

borkdude 2020-11-17T14:39:40.354700Z

let me try without -m

borkdude 2020-11-17T14:40:00.354900Z

still works for me

borkdude 2020-11-17T14:40:12.355100Z

ctrl-c also works

borkdude 2020-11-17T14:40:32.355300Z

which bb version is this? 0.2.3?

walterl 2020-11-17T14:40:44.355500Z

Yip

borkdude 2020-11-17T14:41:18.355700Z

maybe a subtle difference between linux and macos then

walterl 2020-11-17T14:41:39.355900Z

Yeah. That's what's so interesting πŸ™‚

borkdude 2020-11-17T14:41:57.356100Z

could be an implementation difference inside fzf

borkdude 2020-11-17T14:42:08.356300Z

maybe in linux it blocks on writing to stderr first

borkdude 2020-11-17T14:42:28.356500Z

what do you get for :err :string ?

borkdude 2020-11-17T14:42:50.356700Z

so:

(defn fzf [s]
  (let [proc (p/process ["fzf" #_"-m"]
                        {:in s :err :string ;; inherit
                         :out :string})]
    (:out @proc)))

walterl 2020-11-17T14:43:13.356900Z

I know the library fzf uses that provides is_tty() has different low level (termios) implementations for the different OSs. Maybe it stems from there...?

walterl 2020-11-17T14:43:57.357100Z

For :err :string I get no output but input works!

borkdude 2020-11-17T14:44:09.357300Z

right

borkdude 2020-11-17T14:44:19.357500Z

then I think my guess was right

borkdude 2020-11-17T14:44:35.357700Z

that or maybe the buffer for stderr is bigger in macos

🀷 1
walterl 2020-11-17T14:45:47.358Z

Thanks for helping flesh this out, though. And many thanks for bb πŸ™‚:babashka:

πŸ‘ 1
borkdude 2020-11-17T14:51:55.358300Z

It's quite a cool tool, this fzf

1
borkdude 2020-11-17T14:53:27.358500Z

walterl 2020-11-17T15:05:51.358900Z

It's a game changer, yes. I resisted at first, but it quickly convinced me. As a vim user, it has also become indispensable to navigation https://github.com/junegunn/fzf.vim#commands

russmatney 2020-11-17T16:54:14.362600Z

thanks @borkdude - i took at shot at this last night, but couldn't wrap my head around the input/output - thanks for the clean example!

russmatney 2020-11-17T16:54:39.363200Z

@clojurians-slack100 if you use rofi, the integration there is nice too - something like

(require '[babashka.process :refer [process]])

(defn rofi [s]
  (let [proc (process
               ["rofi" "-i" "-dmenu" "-mesg" "Select" "-sync" "-p" "*"]
               {:in  s :err :inherit
                :out :string})]
    (:out @proc)))

(rofi (slurp *in*))

walterl 2020-11-17T16:57:30.363500Z

TIL: rofi πŸ‘

russmatney 2020-11-17T02:37:32.348900Z

I just found https://junegunn.kr/2016/02/using-fzf-in-your-program, and the clojure example worked for me when run with bb - could probably be ported to a more terse bb/process function

russmatney 2020-11-17T02:37:55.349200Z

(require '[<http://clojure.java.io|clojure.java.io> :as io])
(import 'java.lang.ProcessBuilder$Redirect)

(defmacro with-filter
  [command &amp; forms]
  `(let [sh#  (or (System/getenv "SHELL") "sh")
         pb#  (doto (ProcessBuilder. [sh# "-c" ~command])
                (.redirectError
                  (ProcessBuilder$Redirect/to (io/file "/dev/tty"))))
         p#   (.start pb#)
         in#  (io/reader (.getInputStream p#))
         out# (io/writer (.getOutputStream p#))]
     (binding [*out* out#]
       (try ~@forms (.close out#) (catch Exception e#)))
     (take-while identity (repeatedly #(.readLine in#)))))

(with-filter "fzf -m"
  (dotimes [n 50]
    (println n)
    (Thread/sleep 5)))

borkdude 2020-11-17T10:24:12.351Z

https://twitter.com/borkdude/status/1328642754025558018

πŸ‘€ 1
borkdude 2020-11-17T15:08:15.359400Z

file walker with babashka.fs:

user=&gt; (fs/walk-file-tree {:pre-visit-dir (fn [dir attrs] (prn '-&gt; (str (fs/relativize dir))) (if (fs/hidden? dir) :terminate :continue))})
-&gt; ""
-&gt; "test"
-&gt; "test/babashka"
-&gt; ".cpcache"

πŸš€ 7
borkdude 2020-11-17T15:09:08.359800Z

this stops at the first hidden directory, pretty useless, just a demo ;)

borkdude 2020-11-17T15:45:37.360400Z

a few docstrings: https://babashka.org/fs/babashka.fs.html

borkdude 2020-11-17T16:25:51.361100Z

of course, the multi-arg counter part to io/file:

user=&gt; (fs/path "foo/bar" "baz")
#object[sun.nio.fs.UnixPath 0x21baa903 "foo/bar/baz"]

πŸ™Œ 1
2020-11-17T16:52:05.362200Z

Thread/sleep does not do anything on babashka?

borkdude 2020-11-17T16:52:50.362500Z

@nha why do you think that? it should work:

$ time bb -e '(Thread/sleep 5000)'
bb -e '(Thread/sleep 5000)'   0.01s  user 0.01s system 0% cpu 5.028 total

2020-11-17T16:54:38.363100Z

gah you’re right I forgot it was ms not seconds πŸ˜…

πŸ‘ 1
nate 2020-11-17T22:25:19.365200Z

https://twitter.com/ndj/status/1328826419086909440

5
nate 2020-11-18T14:10:15.373900Z

Thank you! I'm quite fond of them. They're key to one-liners being so easy to write.

borkdude 2020-11-18T14:16:22.374100Z

Some people love them, others find them confusing. I'm contemplating an alternative here: https://github.com/borkdude/babashka/issues/613 Just an idea so far.

borkdude 2020-11-18T14:16:47.374500Z

One downsize of using flags is that these things don't work well with full blown scripts

borkdude 2020-11-18T14:16:55.374700Z

So it would be nice if we could have something that worked in both cases

borkdude 2020-11-18T14:17:49.374900Z

I made a separate doc page for the io flags now, so people know how to port it to script usage: https://github.com/borkdude/babashka/blob/master/doc/io-flags.md

borkdude 2020-11-19T20:59:34.381200Z

I now finished watching your presentation. Really good. Posted it to Reddit too :)

nate 2020-11-19T21:23:28.381400Z

awesome, thank you!

nate 2020-11-19T21:23:32.381600Z

glad you liked it

borkdude 2020-11-17T22:27:12.365500Z

Thanks for posting! I'll add it to the bb readme

nate 2020-11-17T22:34:29.365700Z

oh cool, thank you!

borkdude 2020-11-17T23:19:24.365900Z

Excellent explanation of the i/o flags :) Going to watch the rest tomorrow.