unrepl

discussing specification of an edn-based repl and its implementations.
2018-03-09T04:51:21.000130Z

Hmm.... the repl buffer should probably not do read tracking. However, then some special treatment of prompts is necessary.

2018-03-09T04:57:14.000147Z

Eg. user=> (read) :foo (read)<enter> would result in: user=> (read) :foo (read) :foo user=> <cursor here in stdin mode> So there is an empty prompt. However it must be ignored. So it is a bit difficult because the buffer is one channel. Not two like a terminal. Maybe it just works. Maybe it need some special magic. I have to think about this.

2018-03-09T08:36:10.000038Z

Yup. (read) :foo (read) works. :-)

2018-03-09T08:48:01.000133Z

-.- encountered the first problem where vim and clojure disagree on character count....

cgrand 2018-03-09T08:48:36.000152Z

endline or non-ascii?

2018-03-09T09:01:45.000087Z

Non-ascii. Vim has strchars and strlen. The latter gives bytes the former characters. I thought I needed that, but the failed now. Strlen however was correct. -.- Why can't we have cookies?

2018-03-09T09:03:09.000488Z

Oh. How are endlines counted in :len? That is also a point.

2018-03-09T09:03:50.000395Z

(Not in this case. Then the discrepancy would have been much higher.)

cgrand 2018-03-09T09:04:24.000205Z

endlines are normalized to a single \n

2018-03-09T09:11:02.000109Z

headache

2018-03-09T09:33:14.000319Z

Maybe it's not a problem. In this case it wasn't. I need to check this. How is the wire encoding chosen?

cgrand 2018-03-09T09:38:03.000226Z

You can have any encoding you want as long as it’s UTF8

2018-03-09T09:38:29.000384Z

Good.

cgrand 2018-03-09T11:08:45.000368Z

Previously people (at least @volrath and @kotarak) had requested a way to set the eval-id because having to track :read messages to find the matching id was cumbersome.

cgrand 2018-03-09T11:10:41.000062Z

Now since there’s always a prompt before an eval (previously (+ 1 2)(+ 3 4)\n would have triggered two :evals but one :prompt, the prompt is respobsible for allocating the eval-id.

cgrand 2018-03-09T11:13:36.000028Z

synchornizing is thus easier: you know that there’s nothing in flight when prompt offset matches the sent-chars count of the client (better add a tolerance for whitespaces)

cgrand 2018-03-09T11:14:21.000097Z

so if the repl has caught up with the input then the prompt bears the next eval-id

cgrand 2018-03-09T11:14:32.000334Z

Does it sound easier?

volrath 2018-03-09T11:15:50.000380Z

correct me if I'm wrong, but now if we want to produce the same output than a regular repl, the clients would have to know that some prompt messages should be ignored. i.e.

$&gt; clj
Clojure 1.9.0
user=&gt; (+ 1 2)(+ 3 4)
3
7
user=&gt; 
but with these extra prompt messages, the client would print a prompt between the first result 3 and the second 7
$&gt; clj
Clojure 1.9.0
user=&gt; (+ 1 2)(+ 3 4)
3
user=&gt;
7
user=&gt;

volrath 2018-03-09T11:17:26.000307Z

like that.. (i didn't actually test it)

cgrand 2018-03-09T11:17:32.000352Z

Yes, it behaves like

$ clj
Clojure 1.9.0
user=&gt; (clojure.main/repl :need-prompt (constantly true))
user=&gt; (+ 1 2)(+ 3 4)
3
user=&gt; 7
user=&gt;

cgrand 2018-03-09T11:18:48.000209Z

(badly multiplexed outputs notwithstanding)

cgrand 2018-03-09T11:19:42.000167Z

[:prompt {:file nil, :line 1, :column 1, :offset 0, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 1]
(+ 1 2)(+ 3 4)
[:read {:file “unrepl-reader-684”, :from [1 1], :to [1 8], :offset 0, :len 7} 1]
[:started-eval {:actions {:interrupt (unrepl.repl$NPYB29j9QDmbJ9ILAyMl3uHuLOs/interrupt! :session683 1), :background (unrepl.repl$NPYB29j9QDmbJ9ILAyMl3uHuLOs/background! :session683 1)}} 1]
[:eval 3 1]
[:prompt {:file nil, :line 1, :column 8, :offset 7, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 2]
[:read {:file “unrepl-reader-684", :from [1 8], :to [1 15], :offset 7, :len 7} 2]
[:started-eval {:actions {:interrupt (unrepl.repl$NPYB29j9QDmbJ9ILAyMl3uHuLOs/interrupt! :session683 2), :background (unrepl.repl$NPYB29j9QDmbJ9ILAyMl3uHuLOs/background! :session683 2)}} 2]
[:eval 7 2]
[:prompt {:file nil, :line 1, :column 15, :offset 14, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 3]

cgrand 2018-03-09T11:21:25.000261Z

@volrath are you just making sure you understand it or is it against your repl aesthetics?

volrath 2018-03-09T11:21:52.000124Z

both

volrath 2018-03-09T11:22:06.000340Z

would it be great to have it as an optional thing

cgrand 2018-03-09T11:23:24.000226Z

you can dedupe prompts client side, I think the extra messages are useful.

cgrand 2018-03-09T11:23:54.000123Z

maybe a flag to mark the “good” prompts

volrath 2018-03-09T11:24:05.000346Z

that'd also work

volrath 2018-03-09T11:24:29.000020Z

an easier way to filter in/out those prompts

volrath 2018-03-09T11:26:38.000034Z

brb, gonna grab lunch

cgrand 2018-03-09T11:26:44.000281Z

legacy behaviour is “you get a prompt when at the start of a line” so

(+ 1
  2) (+ 3 4)
triggers only one final prompt

cgrand 2018-03-09T11:34:54.000145Z

FWIW my own repl aesthetics: I see the repl split in two frames: upper frame is the log, divider holds the prompt info, bottom frame is the input buffer. Having actual input split in chunks (in the log) according to eval is ok for me.

cgrand 2018-03-09T11:35:08.000203Z

But ok I’m thinking about how to accomodate yours.

cgrand 2018-03-09T11:39:03.000302Z

ah it was simpler than I thought

cgrand 2018-03-09T11:40:00.000165Z

A “traditional” prompt is a prompt whose :column is 1. Look Ma, no flag!

2018-03-10T16:38:31.000055Z

This doesn't help. Since I now have a eval operator, the user might choose to send any span of text for evaluation. In particular multiple forms separated by empty lines. So I will get multiple :column one prompts. So I still need to track the read count. But that's ok. I think that works.

cgrand 2018-03-10T16:53:46.000078Z

Eval selection is a kind of framed evaluation and support is not there yet.

2018-03-10T17:11:29.000136Z

As I said: read tracking works! If you find that too funky on the unrepl side, there is no need to have it. Clients can do it, if they need it.

2018-03-10T17:13:23.000073Z

I just wanted to point out, that the :column one approach might be insufficient. Depending on your inputs.

cgrand 2018-03-09T11:41:25.000201Z

Now:

[:prompt {:file nil, :line 1, :column 1, :offset 0, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 1]
(+ 1 2)(+ 3 4)
[:read {:file "unrepl-reader-683", :from [1 1], :to [1 8], :offset 0, :len 7} 1]
[:started-eval {:actions {:interrupt (unrepl.repl$jr7rtNzCvE2Cb_3TntSHIMlqPCY/interrupt! :session682 1), :background (unrepl.repl$jr7rtNzCvE2Cb_3TntSHIMlqPCY/background! :session682 1)}} 1]
[:eval 3 1]
[:prompt {:file nil, :line 1, :column 8, :offset 7, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 2]
[:read {:file "unrepl-reader-683", :from [1 8], :to [1 15], :offset 7, :len 7} 2]
[:started-eval {:actions {:interrupt (unrepl.repl$jr7rtNzCvE2Cb_3TntSHIMlqPCY/interrupt! :session682 2), :background (unrepl.repl$jr7rtNzCvE2Cb_3TntSHIMlqPCY/background! :session682 2)}} 2]
[:eval 7 2]
[:prompt {:file nil, :line 1, :column 15, :offset 14, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 3]
; THE TWO LINES BELOW ARE WHAT CHANGED
[:read {:file "unrepl-reader-683", :from [1 15], :to [2 1], :offset 14, :len 1} 3]
[:prompt {:file nil, :line 2, :column 1, :offset 15, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 4]

cgrand 2018-03-09T11:46:10.000247Z

Now:

[:prompt {:file nil, :line 1, :column 1, :offset 0, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 1]
(+ 1 2)(+ 3 4)
[:read {:file "unrepl-reader-683", :from [1 1], :to [1 8], :offset 0, :len 7} 1]
[:started-eval {:actions {:interrupt (unrepl.repl$L5y82Z_WyPDOudROXJCUgcOs6ZY/interrupt! :session682 1), :background (unrepl.repl$L5y82Z_WyPDOudROXJCUgcOs6ZY/background! :session682 1)}} 1]
[:eval 3 1]
[:prompt {:file nil, :line 1, :column 8, :offset 7, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 2]
[:read {:file "unrepl-reader-683", :from [1 8], :to [1 15], :offset 7, :len 7} 2]
[:started-eval {:actions {:interrupt (unrepl.repl$L5y82Z_WyPDOudROXJCUgcOs6ZY/interrupt! :session682 2), :background (unrepl.repl$L5y82Z_WyPDOudROXJCUgcOs6ZY/background! :session682 2)}} 2]
[:eval 7 2]
[:prompt {:file nil, :line 1, :column 15, :offset 14, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 3]
; NO MORE :READ HERE
[:prompt {:file nil, :line 2, :column 1, :offset 15, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 4]

cgrand 2018-03-09T12:23:25.000243Z

@volrath ^^

2018-03-09T12:32:08.000227Z

I simply check if there is a prompt and whether the command is empty. If so the spurious prompt is removed. There is a race condition though.

cgrand 2018-03-09T12:33:06.000005Z

?

2018-03-09T12:33:33.000116Z

That's even more funky in vim. I rather keep one window with one buffer. Complicated enough.

cgrand 2018-03-09T12:34:20.000172Z

Can you make part of a buffer read-only?

2018-03-09T12:36:42.000152Z

Vim is async. I have to add a prompt because I don't know what else is coming. The second form then starts. And the spurious prompt is removed. However the event handling is async. So the user might already have typed something. The chance that this happens is pretty low. Or so I should hope.

2018-03-09T12:37:30.000150Z

Nope. "Cp1252" 😛

2018-03-09T12:39:39.000025Z

No.

2018-03-09T12:40:22.000073Z

Vim is very ... hmmm ... inept regarding buffer and window handling.

2018-03-09T12:41:35.000241Z

Maybe there should be a note that you might need -Dfile.encoding=UTF-8 on Windows.

volrath 2018-03-09T12:43:58.000309Z

🎉

volrath 2018-03-09T12:44:03.000212Z

cool, that works 🙂

cgrand 2018-03-09T12:46:16.000134Z

ouch just checked the code: I really thought that clojure.core.server was forcing UTF-8. No. Platform default. Argh!

2018-03-09T12:48:09.000165Z

I can live with setting the option. Now that I know it.

cgrand 2018-03-09T12:48:54.000268Z

With more prompts, :read messages are almost obsolete (its data could be derived by comparing two consecutive prompts). The only thing that it has for it is that you get the data earlier (you don’t have to wait until end of eval)

cgrand 2018-03-09T12:49:56.000380Z

how vim works with encodings?

2018-03-09T12:59:07.000231Z

I set the option. For me: utf-8. It was the clojure side.

2018-03-09T13:03:45.000469Z

Vim actually also has file and term encoding settings. For me these are all hardwired to utf-8.

2018-03-09T13:05:05.000364Z

It has a iconv(). So I could also convert things on the vim side. But I prefer the model T approach.

2018-03-09T13:08:21.000419Z

Öh. Evaling a form with four lines, I get now 4 prompts?

cgrand 2018-03-09T13:08:39.000237Z

No

2018-03-09T13:08:57.000186Z

Why do I see it in the log?

cgrand 2018-03-09T13:09:38.000316Z

1 form spanning 4 lines right?

2018-03-09T13:09:48.000115Z

Yes.

cgrand 2018-03-09T13:10:09.000475Z

(+
 1
 2
 3)
[:read {:file “unrepl-reader-683”, :from [2 1], :to [5 4], :offset 15, :len 12} 4]
[:started-eval {:actions {:interrupt (unrepl.repl$L5y82Z_WyPDOudROXJCUgcOs6ZY/interrupt! :session682 4), :background (unrepl.repl$L5y82Z_WyPDOudROXJCUgcOs6ZY/background! :session682 4)}} 4]
[:eval 6 4]
[:prompt {:file nil, :line 5, :column 4, :offset 27, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 5]
[:prompt {:file nil, :line 6, :column 1, :offset 28, clojure.core/*ns* #unrepl/ns user, clojure.core/*warn-on-reflection* false} 6]
`

2018-03-09T13:10:12.000151Z

Ok. 3 lines there was one empty line.

cgrand 2018-03-09T13:10:45.000373Z

I get two prompts: one after the form, one after the newline (newlines outside of forms)

2018-03-09T13:11:55.000239Z

:-( The newlines without read break my read tracking....

2018-03-09T13:12:25.000491Z

So now I have to check offset in prompt?

cgrand 2018-03-09T13:13:11.000460Z

You are too bleeding edge.

cgrand 2018-03-09T13:13:43.000315Z

Yes, :prompt

2018-03-09T13:13:57.000174Z

I noticed, I got cut.

cgrand 2018-03-09T13:38:29.000520Z

Okay. That’s written in stone 🙂 I almost specified a regex-like structure (like yesterday here) but it was too strong of a commitment for little gain. :prompts are the backbone of the output. There’s one at each start of the Loop. The eval-id should almost be renamed to iteration-id