Clojurians Log v2

@cgrand good morning, last night I did a more generic implementation of elision expansion, and I noticed something weird for maps, that I think it's related to the into used in unrepl.print/ednize

tell me

[14] user=> (zipmap (map char (range 97 (+ 97 11))) (range 11))
> {\a 0 \b 1 \c 2 \d 3 \e 4 \f 5 \g 6 \h 7 \i 8 \j 9 ...}
[15] user=> (zipmap (map char (range 97 (+ 97 11))) (range 11))
> { ... \a 0 \b 1 \c 2 \d 3 \e 4 \f 5 \g 6 \h 7 \i 8 \j 9}
[16] user=> (zipmap (map char (range 97 (+ 97 11))) (range 11))
> {\a 0 \b 1 \c 2 \d 3 \e 4 \f 5 \g 6 \h 7 \i 8 \j 9 ...}
[17] user=> (zipmap (map char (range 97 (+ 97 11))) (range 11))
> {\a 0 \b 1 ... \c 2 \d 3 \e 4 \f 5 \g 6 \h 7 \i 8 \j 9}
[18] user=> (zipmap (map char (range 97 (+ 97 11))) (range 11))
> {\a 0 \b 1 \c 2 \d 3 \e 4 ... \f 5 \g 6 \h 7 \i 8 \j 9}
[19] user=> (zipmap (map char (range 97 (+ 97 11))) (range 11))
> {\a 0 \b 1 \c 2 \d 3 \e 4 \f 5 \g 6 ... \h 7 \i 8 \j 9}
[20] user=> (zipmap (map char (range 97 (+ 97 11))) (range 11))
> {\a 0 \b 1 \c 2 \d 3 \e 4 \f 5 \g 6 \h 7 \i 8 \j 9 ...}

take a look at the ...

it changes positions

non deterministicly afaik

it always holds the last char though

in elisp you read maps as lists or alists?

hah.. good point

nevermind

it can be the parser

At some point I thought “it doesn’t matter” but if it makes life too hard...

yes it does

but pretty sure it's the parser, I'll take a look

oh no

it isn't

(zipmap (map char (range 97 (+ 97 11))) (range 11))
[:read {:from [1 1], :to [2 1], :offset 0, :len 52} 1]
[:started-eval {:actions {:interrupt (unrepl.replG__24139/interrupt! :session24584 1), :background (unrepl.replG__24139/background! :session24584 1)}} 1]
[:eval {\a 0, #unrepl/... {:get (unrepl.replG__24139/fetch :G__24587)} #unrepl/... nil, \b 1, \c 2, \d 3, \e 4, \f 5, \g 6, \h 7, \i 8, \j 9} 1]
[:prompt {:file "unrepl-session", :line 2, :column 1, :offset 52, clojure.core/*warn-on-reflection* false, clojure.core/*ns* #unrepl/ns user}]

there the elision is 2nd

yeah I know — I wasn’t blaming your parser, just trying to figure out things.

haha no worries, me too.

so my position was “it doesn’t matter — let revisit when clients authors show up with pitchforks”

do you have a fork?

no

phew

haha

I can fix it on my side though, I can just make sure to print the ... at the end

at least for now

yeah but...

the current solution is bad for all: • those who read maps as maps can’t lookup for elision because the key is random • those who read maps as list can’t rely on it being at the end. So in both cases you have to do a linear scan (yeah I know reading a map is linear anyway) instead of a get or looking at the last value

so you're thinking on changing the structure for ednizedkvs? kind of like with ednized strings

not going that far but requiring the server to print the elisions last AND to reverse padding and actual elision, so that the key would always be #unrepl/... nil

ok

sounds good

it’s really a genuine question but why do you need this-as if you don’t use it ?

e.G

(defn make-pr-str-stream
  []
  (let [transform (fn [v enc cb]
                    (this-as this
                      (cb nil (prn-str v))))]
    (Transform. #js {:writableObjectMode true
                     :transform transform})))

dead code left after refactoring?

@baptiste-from-paris check it out ^^

done

I dusted off a text layout algorithm that I adapted to data, here is the result https://gist.github.com/cgrand/fd67ff7459b1a446a14a533095261c14

So it’s an experiment in compact pretty printing

Do you like the produced layouts?

what would be the context? where/how do you see this coming into play?

I could see it used by unravel to format values.

cool... I don't know the details of the implementation but I guess it should take care special care of pair of values (kvs, tagged literals)

would love to see it inside unravel 🙂

Good point about pairs. The algorithm lays out unbreakable spans (and spans have an indent value). A first naive approach would be to merge the last span of the key and the first one of the value and add an extra indent for the value.

Actually I think there’s a better way

I think I found a bug with blob customization (trying it right now)

one second

so, the unrepl_make_blob task reads the session action map with {:default tagged-literal}

the thing is that this outputs the actual tagged literal into the blob

it puts something like :my.own/action (some.ns/fun #unrepl/param :bar)

whereas what's seem to be need is something like :my.own/action (some.ns ~(tagged-literal 'unrepl/param :bar))

I tried to fix it but couldn't to be hones I couldn't get it to work and i think it might be an easy fix, so I rather ask

the easiest way to reproduce the problem is by simply generating a new custom blob and try to upgrade to unrepl with it

it will raise a read error because the #unrepl/param is not recognized

AFK could you show me the relevant portion of the generated blob?

sure, one moment

:actions (into
                                                                {:exit `(exit! ~session-id)
                                                                 :start-aux `(start-aux ~session-id)
                                                                 :log-eval
                                                                 `(some-> ~session-id session :log-eval)
                                                                 :log-all
                                                                 `(some-> ~session-id session :log-all)
                                                                 :print-limits
                                                                 `(let [bak# {:unrepl.print/string-length p/*string-length*
                                                                              :unrepl.print/coll-length *print-length*
                                                                              :unrepl.print/nesting-depth *print-level*}]
                                                                    (some->> ~(tagged-literal 'unrepl/param :unrepl.print/string-length) (set! p/*string-length*))
                                                                    (some->> ~(tagged-literal 'unrepl/param :unrepl.print/coll-length) (set! *print-length*))
                                                                    (some->> ~(tagged-literal 'unrepl/param :unrepl.print/nesting-depth) (set! *print-level*))
                                                                    bak#)
                                                                 :set-source
                                                                 `(unrepl/do
                                                                    (set-file-line-col ~session-id
                                                                      ~(tagged-literal 'unrepl/param :unrepl/sourcename)
                                                                      ~(tagged-literal 'unrepl/param :unrepl/line)
                                                                      ~(tagged-literal 'unrepl/param :unrepl/column)))
                                                                 :unrepl.jvm/start-side-loader
                                                                 `(attach-sideloader! ~session-id)}
                                                                {:my.own/action (unrepl.repl/ensure-ns (foo/bar #unrepl/param :baz))})

there, now i'ts better

every other action's params are ~(tagged-literal 'unrepl/param :some/name), and the generated by the customization is #unrepl/param :some/name

if I try to upgrade with that blob, I get

$> cat test-blob.clj - | nc localhost 5555
user=> [:unrepl.upgrade/failed]
RuntimeException No reader function for tag unrepl/param  clojure.lang.LispReader$CtorReader.readTagged (LispReader.java:1245)

by now I'm just using ~(tagged-literal 'unrepl/param :my/param) to generate a custom blob, but maybe this is an easy fix

You are right

It’s a rather easy fix: need a reader and a record for unrepl/param

yeah the record for the unrepl/param was the one I didn't figure out, I had problems with the ~, I forgot about unquote

I'll submit a PR later tonight, I'm heading out.. thanks for the help!

@volrath there’s a way simpler fix: pass a default tag reader to read in gen-blob.

what blob.clj is used for ?

@cgrand Now I'm afk, but i'm under the impression that's a clojure.core/read, not a clojure.edn/read, is it possible to add readers to that function? If so, that would definitely be better.

humm lein unrepl-make-blob strange ^^

ohhhhh https://github.com/Unrepl/unrepl/blob/master/UPGRADE.md

Just to be sure, are you talking about this read? https://github.com/Unrepl/unrepl/blob/master/tasks/leiningen/unrepl_make_blob.clj#L15

so that’s what transfors classical nREPL message in unrepl ones

There’s no nREPL involved!

Hey I’ve got a new one: unREPL the underground REPL

so you telling me that I understood all wrong ^^

#embarrassed

when we say “repl” it’s a plain repl (generally socket one so clojure 1.8) as provided by clojure

oh

ok ok

However as a proof of concept we have the nREPL-bridge project which recreates a plain repl on top of nREPL. Then you can upgrade it to unrepl.

okay but blob.clj is transforming REPL message to unrepl protocol right ?

or I am completly lost

@cgrand I'm back, I don't see how I can pass a default tag reader to clojure.core/read, not sure if I'm missing out on something... I tried to go with the other approach, having a tag reader for #unrepl/param but I'm a bit blocked. Best I can get is

{:readers {'unrepl/param (fn [param-kw]
                                        '(unquote (tagged-literal 'unrepl/param param-kw)))}
              :default tagged-literal}

*default-data-reader-fn* but I’m blocked too by a subtle error related to pr-dup

Not so subtle, it’s just too late.