Clojurians Log v2

:)

I have enabled puget in a branch. It does work, but it breaks piping.

$ /usr/local/bin/bb '{:a (repeat 10 {:a 1 :b 2 :c 3})}' | /usr/local/bin/bb '(:a *input*)'
({:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1} {:c 3, :b 2, :a 1})
borkdude@MBP2019 ~/Dropbox/dev/clojure/babashka (puget*) $ ./bb '{:a (repeat 10 {:a 1 :b 2 :c 3})}' | ./bb '(:a *input*)'

Maybe puget can be made into its own GraalVM binary so you can pipe EDN to that and colorize it at will

bb ... | puget --cprint

and then create an alias for it in your shell

Interesting. I'll try that, thanks!

I think such a tool will be very helpful in other contexts too. Any tool outputting EDN can then be piped into that

@isak There's also a zprint GraalVM binary: https://github.com/kkinnear/zprint Maybe that also supports colors?

I looked at that one before and it seems good, but doesn't support windows builds

I was gonna try wrapping babashka with another babashka script that adds puget

I don't think puget will work from source with babashka

ah I see

but puget does work for you from the JVM? which terminal are you using?

the reason I'm asking is, I have a pod for making TUIs: https://github.com/babashka/pod-babashka-lanterna but it doesn't work on Windows because of the terminal differences

I'm using Windows Terminal: https://docs.microsoft.com/en-us/windows/terminal/

and yea, puget does work for me on the JVM

Though I haven't tried piping it to the console, just HTML output so far

and which shell, cmd.exe or powershell?

powershell

I made a babashka puget branch. Can you maybe test it by running lein bb '{:a (repeat 10 {:a 1 :b 2 :c 3})}' and then see what it prints?

in PS

yea

Ah nice. It does seem to work.

sweet, yea just saw it

Also works in cmd.exe

yea, I just tested that too

What if pugetting the output was a switch? Similar to how you have the -O

I pushed a commit with --puget. We might want to generalize this to --print-fn which defaults to clojure.core/prn but can be set to clojure.pprint/pprint or puget/cprint or whatever maybe?

I want to pay attention to the increase in compilation memory usage and binary size first before deciding on this

splitting it into its own binary has the benefit of wider utility perhaps

Also made these issues for it: - https://github.com/borkdude/babashka/issues/620 - https://github.com/greglook/puget/issues/53

Nice! Yea, makes sense

@isak it's done building here: https://ci.appveyor.com/project/borkdude/babashka

it seems to increase binary size with 1 MB

Nice, I will use this

Hmm I get a problem:

PS C:\Users\isak.sky> bb '(range 5)'
(0 1 2 3 4)
PS C:\Users\isak.sky> bb --puget '(range 5)'
----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  File does not exist: --puget

put --puget as the last option. I implemented it so that --puget can get an optional option

ah ok

or use -e '(range 5)'

Ok now I don't get an error, but no colors/formatting:

PS C:\Users\isak.sky> bb -e '(into {:foo :bar} (for [i (range 10)] [i i]))' --puget
{0 0, 7 7, 1 1, 4 4, 6 6, 3 3, :foo :bar, 2 2, 9 9, 5 5, 8 8}
PS C:\Users\isak.sky> bb -e '(into {:foo :bar} (for [i (range 10)] [i i]))' --puget $true
{0 0, 7 7, 1 1, 4 4, 6 6, 3 3, :foo :bar, 2 2, 9 9, 5 5, 8 8}
PS C:\Users\isak.sky>

Are you sure you're using the right version. Doesn't look like it

ah it is the old one now, strange - i checked this before

I see it working now, sorry

I did make https://github.com/greglook/edn-tool as a standalone tool a couple years ago, in any case. 😄

@greg316 Nice. I just tried it. But I think the name should change to puget or pprint or something more akin to prettifying EDN

But the general idea is good!

I guess the only thing missing is a Windows release, which graalvm now supports

yeah it was going to be more general-purpose and support querying and transformation and such, but then https://github.com/borkdude/jet did all that 😅

semi related, thoughts on supporting CBOR in jet? https://github.com/greglook/clj-cbor

No thoughts on that, never used it

I might tackle it one of these days - CBOR is great, it’s basically binary EDN

How does it compare to nippy?

I mean, why would one choose CBOR over nippy?

Nippy is not standardized, maybe that is why. (So you're unlikely to get implementations in other languages)

good point

transit+messagepack?

anyway to get back on point, I think a puget tool that only focuses on pprinting might be nice. I'm not sure if that fits with jet since it should then also colorize json output etc

adding cbor and nippy to jet, I think that can work

PR welcome :)

Nippy is... It’s fast, but it’s not a standard so it’s clojure-only (including non-cljs). It does naive fallback to Java Serializable, so if you don’t define a custom formatter for some type like org.joda.time.DateTime it will blow up to hundreds of bytes when serialized. We also observed backwards-incompatible minor version changes, which broke all of our stored data. 😬 I would not recommend it.

I also like CBOR over transit+msgpack because it’s a lot simpler as a format and implementation, supports a wider variety of types, and is similarly more widely-supported (since transit is also clojure-specific). :man-shrugging:

@greg316 sounds interesting. if CBOR is widely supported, maybe it's also a good candidate to add to pods as a wire format

pretty wide: http://cbor.io/impls.html

it seems clojure is missing?

one of the explicit goals of the RFC is a simple/compact implementation in addition to compact message representation 😁 - so it’s pretty easy to write a codec

(on that page)

it’s just sorted poorly

oh right

or.. not sorted? :man-shrugging:

@greg316 So if you decided to support a custom format, say a Clojure set, then I guess there is no standard way of decoding that in, say Go, right?

it’s a lot like EDN’s tagged literals; there is an IANA registry entry for “sets”, which are represented as an array of elements with tag 258

obviously in clojure those turn into #{} but it’d be up to golang to decide what type to represent sets with - I’m guessing there’s similarly some native language type for them

https://github.com/greglook/clj-cbor#type-extensions

https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml

The types most likely to be “weird” in other languages are keywords and symbols, which are ubiquitous in clojure but likely don’t have as direct a representation in other languages. You could define a handler for those tags to stringify them for a simple lossy conversion though, which is probably fine for things like map keys.

then there's also https://github.com/FasterXML/smile-format-specification

(bb already supports this since it has cheshire built-in)

It seems that smile isn't that well supported outside of Java

yeah, I’ve only ever seen it in Jackson

a lot of these binary formats (smile, msgpack, jsonb, etc) all fell into the same trap of, “like JSON, but binary!” without solving the type extension problem 🤦

@isak @greg316 Made this now: https://github.com/borkdude/puget-cli

@borkdude nice, got it working!