Clojurians Log v2

Mindbending demo of building an AWS Lambda front-end with babashka and scittle that interacts with other AWS features like Polly: https://twitter.com/FieryCodDev/status/1401843357555511301

In my REPL I get:

user> (require '[babashka.process :as p :refer [process]])
;; => nil
user> (process ["ls"])
Error printing return value (IllegalArgumentException) at clojure.lang.MultiFn/findAndCacheBestMethod (MultiFn.java:179).
Multiple methods in multimethod 'simple-dispatch' match dispatch value: class babashka.process.Process -> interface clojure.lang.IDeref and interface clojure.lang.IPersistentMap, and neither is preferred

I guess you aren't meant to be able to print processes like that...

Is this a bb repl?

No, CIDER 🙂

It seems cider is using pprint in the REPL then: https://github.com/babashka/process#clojurepprint

Do you also know why CIDER might be eating my stderr? The below gives stderr output in a regular REPL, but not in CIDER:

(require '[babashka.process :as p :refer [process]]
         '[<http://clojure.java.io|clojure.java.io> :as io])
(def bwa-mem (process ["bash" "-c" "bwa mem -R '@RG\\tID:A\\tSM:A'  /Users/endrebakkenstovner/everclear/snakemake-flow/data/genome.fa /Users/endrebakkenstovner/everclear/snakemake-flow/data/samples/A.fastq | samtools view -Sb - &gt; /Users/endrebakkenstovner/everclear/snakemake-flow/bwa-map/genome/hg19/sample/A/bwa-map.bam"] {:err :inherit
                           :shutdown p/destroy}))
(with-open [rdr (io/reader (:out bwa-mem))]
  (binding [*in* rdr]
    (loop []
      (let [line (read-line)]
        (println :line line)
      (when (not (nil? line))
        (recur))))))

Just curious. Otherwise I'll ask in CIDER.

The above command would only show stderr output.

the stderr of a process is written to System/err, if you use :inherit, not to *err*

Unless you configure it with {:err *err*}

that is probably the issue

Yes, it was! I had been playing around with it for so long XD, thanks!

Anyways, can I use :shutdown to issue a callback after process has finished? Or is some other way recommended? I'd love to do (process bwa-map {:shutdown #(do (notify-others %) (p/shutdown %))

currently there is no callback that you can add when the process exits, but it is possible to add this

But can't I cheat in the way I showed above?

I think you can. But the :shutdown hook is only executed when the JVM or babashka shuts down, not directly when the process itself ends

but in Java 11 there is an onExit hook

which babashka.process could also support

it depends on your use case, so far I didn't need it

https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Process.html#onExit()

That would be neat. My program dispatches plenty of jobs (think make) and I would love to be notified when they finish (ideally with all the info found in (process ...)) so that I could process the results easily.

I'll look into it!

you could use this as of today, but using Java interop

the process is available under :proc

Mind if I ask you questions if I can't get it working? XD I'll start looking into it now 🙂

sure. is your target with this usage the JVM or bb?

It is the JVM 🙂

I don't think bb allows interop with CompleteableFuture yet

ok

I expected this to work from a bb script, since I saw a usage of http://java.net classes in an example in the book:

(import [<http://java.net|java.net> DatagramSocket
         DatagramPacket
         InetSocketAddress])

i've done no setup other than the shebang so if I have to get into pods or a bb.edn, I haven't tried that

A selection of Java classes are available, see babashka/impl/classes.clj in babashka's git repo.

The example I saw was at the end of this section: https://book.babashka.org/#_nrepl

user=> (def proc (:proc (babashka.process/process ["ls"])))
#'user/proc
user=> (def cf (.thenApply (.onExit proc) (reify java.util.function.Function (apply [this p] (.exitValue p)))))
#'user/cf
user=> (.get cf)
0

do I need to do something classpath-y in order to invoke an import like that?

@mathpunk It's just not available in bb yet. https://github.com/babashka/babashka/blob/cc21fcc9ef7f71ccde836e055c9d04f4a9dc798b/src/babashka/impl/classes.clj#L154

gotcha, didn't consider that a subset of http://java.net might be present

we could consider adding it if there is a good use case for it and if the added binary size is still reasonable. could you explain why you need it?

I am communicating with datadog via UDP datagram

I might should just use curl instead

Are you the same person I recently gave the statsd example to? I think that is all you need

https://github.com/babashka/babashka/blob/3dfc15f5a40efaec07cba991892c1207a352fab4/test-resources/babashka/statsd.clj

I am not, I bet I could reuse it

great, I'm sending events rather than incrementing a thing but, I think I can see enough to find my way toward that

thank you!

:thumbsup:

This worked beautifully, thanks.

I added InetSocketAddress for the next release so the next person won't run into this issue again

It's been a pretty successful day getting babashka to help me do a task. One thing though: I created a second .clj file, and went to require the other one, and it's not found. I figured: 1. I'll probably have to learn more about classpaths to do that; but also, 2. Maybe if I split up namespaces that's a sign I'm using babashka for too large a problem?

I'm adding functionality to existing shell scripts, which will call my program, so I thought it was the right thing to grab. I could be wrong and it's a good time in this project to figure out if I am 🙂