Clojurians Log v2

Finally found a feasible way to include proper clojure.pprint :)

$ ./bb '(clojure.pprint/cl-format true "~R~%" 63)'
sixty-three

(see #babashka_circleci_builds , core-pprint branch)

(sometimes GraalVM binary size is incomprehensible: with some commit the macOS binary size went down with 3MB and I just did a clean up commit and the binary is up with 3MB again)

either way, replacing fipp with clojure.pprint proper (patched for GraalVM) didn't have a significant binary size change

@alekcz360 if you need any help with pod impl let me know

wow @thomas.ormezzano's pod now also supports S3 and pre-compiled binaries are available https://github.com/tzzh/pod-tzzh-aws

Thanks for mentioning it 🙂 so I actually use babashka to generate a lot of the Go code as I realised I was doing a lot of copying and pasting to add more functions (as it’s kind of always the same functions but with different types) It’s only dynamodb and s3 for now as they are probably the most common (and the only ones I need at the moment) but I can easily add more services when they are needed (I could also automate that part and have all the sdk but I want to make sure there isn’t a better option that code generation first)

haha, how meta. I don't use AWS a lot but I do use S3 for work. When I try it I get this:

5: (s3/list-buckets)
   ^--- MissingRegion: could not find region configuration

where should I specify a region?

ah so I think there might be different ways to set it but what I personally have is these env variables

AWS_ACCESS_KEY_ID=zzzzzzz
AWS_SECRET_ACCESS_KEY=zzzzzzz
AWS_DEFAULT_REGION=eu-west-1
AWS_REGION=eu-west-1

ah right

$ export AWS_REGION="eu-west-1"
$ bb /tmp/s3.clj
{:Buckets [{:CreationDate "2020-07-07T07:32:16Z", :Name "rotocode.dre"}], :Owner {:DisplayName "wagner", :ID "01813cccafaa89781d4a555c4026b9abde7c33195cdf0b75236381b28056c4e8"}}

🍾

@thomas.ormezzano Have you see the cognitect aws library? They generate everything from the openapi specifications from aws. I'm using it myself from clojure but it might give inspiration on generating code. I think it might be less fragile than what you described, since if the spec changes you can regenerate your code from the new one without having to manually make changes. https://github.com/cognitect-labs/aws-api

Ah ok interesting I’ll take a look 🙂

ok yeah it looks good but I think it’s more helpful when trying to use the API directly. For the pod it’s more about the actual Go types that are used as input/output to the functions of the sdk so I don’t think that would work as well. I am thinking maybe I could parse the API docs or even the Go code to extract them but I am not really convinced yet.

About the pagination, is the problem that you can't pass functions over pods (or Clojure functions to go for that matter)? Is there a data representation possible that would make sense? I have no idea how much of a problem this is in practice

How does the Go SDK pass this function over HTTP to AWS itself for dealing with this?

or does it fetch everything and then filter?

so my understanding is that the go sdk fetches the pages over http one by one and keeps doing it until there is no more data to fetch or the function passed argument returns false after being eval’d on the page

ah right, so the function can check like: this is the nth page, this is enough?

yeah or I guess if you’re looking for an object in s3 or something you can go start going over the pages and e.g stop when you’ve found what you were looking for

I have not needed that in practice so it’s not really a problem for me but I wonder how often if might be for other people

with pods it's also possible to define macros and "client side" functions by just passing code which will be evaluated on the babashka side. maybe that's something that could be used for this.

just fyi

ah ok I am not 100% sure I understand what you mean but I need to look into this - I was thinking it could also be possible to send each page back to bb eval the fn there and then pass back the output to Go but that seemed a bit tricky and not really worth it

in the pod you could implement a function that's executed by bb: (loop [...] (let [p (aws/page x)] (if (f p) (recur ....) ...) so then only aws/page (or whatever, this is pseudocode) will be called from the pod, which only fetches a single page at a time and bb itself checks for more

the babashka-sql-pods have examples of how this is used.

e.g. database transactions are implemented using this

ok I see interesting I’ll think about it :thumbsup:

It's really cool to have a Go pod in the family of pods, another good example on which others can build

🙂 yes I think this https://github.com/tzzh/pod-tzzh-aws/blob/master/babashka/babashka.go can be reused as it is for any go pod to do the bencode communication between the pod and bb

did you find it a barrier to work with bencode or was it ok?

tbh I was a bit worried at first but it was actually not a problem at all. There was already a library to do the encoding/decoding https://github.com/jackpal/bencode-go and so the bencode part was really easy

nice

Running one bb on the server and opening an nREPL session there (or socket REPL, if you will) and then executing commands from another bb connected to that, yeah that can work

I don't know if ssh running in a java.lang.ProcessBuilder and then sending input to that works, I never tried this.

this is probably basic Clojure knowledge but is there a way to send commands programmatically to an nREPL session or do you need to actually just do a regular REPL type session?

@goomba I've got one example here: https://github.com/borkdude/babashka#communicating-with-an-nrepl-server

Wow! Awesome. You are a very prolific writer, give St. Thomas Aquinas a run for his money!

Ohhhh, that nrepl "client" is incredibly handy

i often use wikipedia to translate some english term to finnish. i made a simple babashka script to do this without browsers. if it finds an exact answer it shows what the corresponding finnish wikipedia entry would be called. if it gets a disambiguation page, it shows the alternatives. i thought about parameterizing this to enable search between any two languages but then decided that was creeping featuritis.

nice and simple :)

tested it locally:

$ translate.clj dog
"Hond – Dutch"

this is working SO WELL!

Made a slight modification for convenience: