unrepl

discussing specification of an edn-based repl and its implementations.
2018-03-29T12:27:57.000514Z

unrepl lives now in vimpire's shading scheme. Nice. šŸ˜Ž

cgrand 2018-03-29T12:42:33.000431Z

heh?

2018-03-29T12:51:14.000453Z

2018-03-29T12:51:14.000610Z

cgrand 2018-03-29T12:55:00.000267Z

However we went from

=> (let [a] 2)
IllegalArgumentException let requires an even number of forms in binding vector in hwlc5.core:1  clojure.core/let (core.clj:4333)

cgrand 2018-03-29T12:55:29.000528Z

to

=> (let [a] 2)
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
In: [0] val: () fails spec: :clojure.core.specs.alpha/bindings at: [:args :bindings :init-expr] predicate: any?,  Insufficient input
 #:clojure.spec.alpha{:problems [{:path [:args :bindings :init-expr], :reason ā€œInsufficient inputā€, :pred clojure.core/any?, :val (), :via [:clojure.core.specs.alpha/bindings :clojure.core.specs.alpha/bindings], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x64ba3208 ā€œclojure.spec.alpha$regex_spec_impl$reify__2436@64ba3208"], :value ([a] 2), :args ([a] 2)}, compiling:(NO_SOURCE_PATH:1:1)

pesterhazy 2018-03-29T12:57:20.000293Z

Heh

cgrand 2018-03-29T13:03:34.000784Z

With expound:

=> (let [a] 2)
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
-- Syntax error -------------------

  ([a] ...)
   ^^^

should have additional elements. The next element ā€œ:argsā€ should satisfy

  any?

-- Relevant specs -------

:clojure.core.specs.alpha/bindings:
  (clojure.spec.alpha/and
   clojure.core/vector?
   (clojure.spec.alpha/* :clojure.core.specs.alpha/binding))

-------------------------
Detected 1 error
 #:clojure.spec.alpha{:problems [{:path [:args :bindings :init-expr], :reason ā€œInsufficient inputā€, :pred clojure.core/any?, :val (), :via [:clojure.core.specs.alpha/bindings :clojure.core.specs.alpha/bindings], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x5aa360ea ā€œclojure.spec.alpha$regex_spec_impl$reify__2436@5aa360eaā€], :value ([a] 2), :args ([a] 2)}, compiling:(NO_SOURCE_PATH:3:1)

pesterhazy 2018-03-29T13:09:46.000077Z

Much better for sure

pesterhazy 2018-03-29T13:10:17.000398Z

I wonder if it should say "The vector should have additional elements." — That's what I would say in conversation

cgrand 2018-03-29T13:10:49.000568Z

And the formatting around

The next element ":args" should satisfy

  any?
is weird

pesterhazy 2018-03-29T13:11:24.000565Z

I'll bring this up to Ben, he's my coworker

cgrand 2018-03-29T13:11:37.000076Z

I’m not even sure to what it refers

cgrand 2018-03-29T13:11:47.000067Z

really? that’s awesome!

pesterhazy 2018-03-29T13:12:01.000197Z

Yup šŸ™‚ Happy coincidence

cgrand 2018-03-29T13:15:53.000058Z

So given the state of 1.9 exceptions, I believe shipping expound with unrepl is a necessity

cgrand 2018-03-29T13:16:18.000095Z

@bbrinck good job

pesterhazy 2018-03-29T13:17:10.000685Z

I wouldn't say it's necessary but it would be awesome

cgrand 2018-03-29T13:17:53.000363Z

s/necessity/would be a strong differentiator/

bbrinck 2018-03-29T13:25:15.000503Z

Hello šŸ‘‹ !

bbrinck 2018-03-29T13:26:15.000636Z

Feedback on expound messages is always welcome. I may also be able to provide a little context on why things are the way they are šŸ™‚

pesterhazy 2018-03-29T13:26:42.000389Z

the "next element" in (let [a 1]) would be the 1 right?

bbrinck 2018-03-29T13:27:16.000238Z

that’s correct, but I agree that’s not easy to see

bbrinck 2018-03-29T13:27:16.000700Z

Generally speaking, the errors around arg lists are the worst part of expound, but something I’m very interested in improving

bbrinck 2018-03-29T13:27:38.000499Z

(well, around any regex spec, but the most common case is around arglists)

pesterhazy 2018-03-29T13:28:03.000514Z

:args is the name of the spec?

bbrinck 2018-03-29T13:29:43.000160Z

Yes, that’s correct … when you call fdef for a function or macro, you can provide 3 parts: :args, :ret and :fn. Later, you do something like (:args (s/get-spec `let))`

bbrinck 2018-03-29T13:30:22.000706Z

hm, my backtick escaping is failing in slack

bbrinck 2018-03-29T13:30:54.000093Z

trying this: ā€œ(s/form (:args (s/get-spec `let)))ā€

bbrinck 2018-03-29T13:30:59.000471Z

eh, you get the idea

cgrand 2018-03-29T13:31:01.000488Z

the whole spec for :args is (clojure.spec.alpha/cat :bindings :clojure.core.specs.alpha/bindings :body (clojure.spec.alpha/* clojure.core/any?))

pesterhazy 2018-03-29T13:31:38.000207Z

user=> (s/get-spec 'clojure.core/let)
#object[clojure.spec.alpha$fspec_impl$reify__2451 0x732f29af "clojure.spec.alpha$fspec_impl$reify__2451@732f29af"]

pesterhazy 2018-03-29T13:31:42.000061Z

I'm a spec-newb šŸ˜ž

cgrand 2018-03-29T13:32:20.000594Z

so I was tempted to interpret

The next element ":args" should satisfy

  any?
as ā€œthe next element of :args out of :bindings

cgrand 2018-03-29T13:32:36.000035Z

@pesterhazy call s/form on it

pesterhazy 2018-03-29T13:36:01.000399Z

Using the -m friendly option from the expound README. Fun!

bbrinck 2018-03-29T13:36:28.000721Z

@cgrand Agreed, this looks like a bug in expound.

pesterhazy 2018-03-29T13:36:42.000458Z

And yeah s/form shows the right spec

bbrinck 2018-03-29T13:36:44.000376Z

I suspect I failed to account for some extra data spec puts in the explain-data in this case

bbrinck 2018-03-29T13:37:39.000422Z

I’ve created an issue, thanks for pointing this out!

bbrinck 2018-03-29T13:38:13.000338Z

In general, the problem with the core macros is that there is a lot of branching

bbrinck 2018-03-29T13:38:19.000289Z

but it’s something I’m working on

cgrand 2018-03-29T13:38:30.000160Z

thanks to you! I’m looking at the code, problems is the ns where most magic happens?

bbrinck 2018-03-29T13:39:26.000403Z

if by ā€œmagicā€, you mean ā€œchaosā€, then yes šŸ™‚

bbrinck 2018-03-29T13:39:57.000218Z

generally speaking the complexity of expound comes from a design constraint

bbrinck 2018-03-29T13:40:33.000062Z

the message after the announcement of spec was that anyone could print the explain-data however they wanted, so I really wanted to follow that to the logical conclusion

bbrinck 2018-03-29T13:41:21.000273Z

(as opposed to say, pinpointer, which analyzes the actual spec forms itself. really cool idea! but I wanted to flush out bugs w/ spec that would prevent others from building things like expound)

bbrinck 2018-03-29T13:41:56.000324Z

however, the trick with using explain-data is that the :in path is often ambiguous and, in any case, you can’t use the path to trivially find the bad value

bbrinck 2018-03-29T13:42:40.000650Z

i.e. you can’t just use the path in get-in or update-in. As a result, there’s a ton of code to a) disambiguate paths and b) highlight bad values.

bbrinck 2018-03-29T13:42:59.000664Z

this code is kind of a mess but I haven’t yet had time to rework it šŸ˜ž

bbrinck 2018-03-29T13:44:18.000198Z

anyway, a few things to point out: 1. If you don’t like the fact that expound adds ... for irrelevant values, that can be controlled 2. you can also omit the ā€œrelevant specsā€ section 3. I’ve got a branch right now that includes colorization

cgrand 2018-03-29T13:47:49.000272Z

for the record, here is with pinpoint

=> (let [a] 2)
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
Detected 1 spec error:
----------------------------------------------------------------------
(1/1)

    Cause: ([a ...] 2)
               ^^^
 Expected: any?
 Failure
   Reason: Insufficient input

----------------------------------------------------------------------
 #:clojure.spec.alpha{:problems [{:path [:args :bindings :init-expr], :reason ā€œInsufficient inputā€, :pred clojure.core/any?, :val (), :via [:clojure.core.specs.alpha/bindings :clojure.core.specs.alpha/bindings], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x1d6d1d42 ā€œclojure.spec.alpha$regex_spec_impl$reify__2436@1d6d1d42"], :value ([a] 2), :args ([a] 2)}, compiling:(NO_SOURCE_PATH:3:1)

bbrinck 2018-03-29T13:48:10.000172Z

Much nicer!

cgrand 2018-03-29T13:49:20.000032Z

@bbrink well I would be interested in a more data-ish/less formated output

pesterhazy 2018-03-29T13:49:45.000617Z

like the colors

pesterhazy 2018-03-29T13:50:04.000704Z

the ... is a nice touch

bbrinck 2018-03-29T13:51:01.000136Z

@cgrand Do you mean you’d like to get something like explain-data so that unrepl can format it accordingly?

cgrand 2018-03-29T13:53:03.000059Z

yeah somewhere between you annotated explain-data and your printer

cgrand 2018-03-29T13:53:27.000570Z

[warning: thinking out loud]

cgrand 2018-03-29T13:54:36.000228Z

the idea for unrepl integration would be to not bing *explain-out* but in the exception handler of the repl to sniff spec exceptions and pimp them

bbrinck 2018-03-29T13:56:03.000246Z

Also thinking out loud here …

bbrinck 2018-03-29T13:57:07.000159Z

it’s possible that expound could provide an option to just print a more parseable thing with the data, although IIRC, the result of the normal explain-data is not readable in some cases due to the actual specs included

bbrinck 2018-03-29T13:57:32.000288Z

(this may change with new version of spec that is supposed more data focused, but i don’t know what that will look like)

bbrinck 2018-03-29T13:58:10.000562Z

you could potentially bind *explain-out* and just capture the explain-data that is passed in, then pass it to expound to annotate it with more data

bbrinck 2018-03-29T13:58:28.000239Z

(that inner API is not stable right now, but eventually it could be)

bbrinck 2018-03-29T13:59:29.000684Z

also, in general, i’ll be curious to see how unrepl catches exceptions and improves them. there’s quite a lot of interest in trying to do that generally for any project, so I

bbrinck 2018-03-29T14:01:04.000667Z

I’m keeping my eye on efforts on this front - related to stuff in maria.cloud, pyro, a discussion in #clojure yesterday about heuristics for better errors, https://github.com/always-be-clojuring/issues/issues/7 … lots of interesting discussions around this

bbrinck 2018-03-29T14:02:28.000804Z

but AFAICT, spec doesn’t let a printer like expound really control the actual exception data, so I can’t attach more data to the exception.

bbrinck 2018-03-29T14:02:53.000238Z

(at least, that’s my memory of the situation, I haven’t actually tried it)

cgrand 2018-03-29T14:05:59.000624Z

so unrepl installs its own read eval print loop and there are two places where exceptions can be intercepted

cgrand 2018-03-29T14:13:33.000115Z

* when the exception is caught by the loop * when any exception is printed (including as part of a result value)

cgrand 2018-03-29T14:20:34.000371Z

so it would be quite possible to detect when ex-data contains ::s/problems and enrich the map before printing

bbrinck 2018-03-29T14:22:15.000637Z

I see, that’s cool

bbrinck 2018-03-29T14:25:14.000855Z

I’m biased of course, but I’d be happy to see expound used in some way within unrepl. Bug reports are always welcome and I’m open to expanding the API to make unrepl integration easier. Let me know how I can help!

cgrand 2018-03-29T14:28:39.000070Z

what I like about expound is that it has no deps (unlike pinpointer)

dominicm 2018-03-29T14:31:10.000825Z

Why is that a preference?

cgrand 2018-03-29T14:34:27.000017Z

I know sideloader, shading etc.

cgrand 2018-03-29T14:41:24.000338Z

@volrath @pesterhazy @kotarak you all use your own actions (or even no actions at all) for completion, right?

pesterhazy 2018-03-29T14:42:11.000828Z

yeah unravel doesn't use an action currently

dominicm 2018-03-29T14:42:23.000559Z

Are you implying they don't work transitively? I don't understand sorry.

volrath 2018-03-29T14:53:38.000137Z

I use my own

cgrand 2018-03-29T14:53:51.000035Z

IIRC the proposed general contract for a standardize completion actions was: • in: code (string) • in: offset (int) • in: ns-name (sym) • out: coll of candidates (strings)

cgrand 2018-03-29T14:55:27.000384Z

do you use any extra information (in or out)?

cgrand 2018-03-29T14:59:19.000072Z

I may imagine more output: • candidate (string) • relative offset (relative to the input offset) and length to specify what to replace by the candidate

cgrand 2018-03-29T15:02:28.000411Z

For example:

cgrand 2018-03-29T15:02:48.000115Z

=> (complete "(let [x (mapc" 13 'user) ; 13 is the end of the string
({:candidate "mapcat", :roffset -4, :len 4})

cgrand 2018-03-29T15:04:19.000033Z

No I just imply that it’s a matter of taste and that I tend to consider any dep as a liability

cgrand 2018-03-29T15:04:37.000316Z

Now for something a bit more controversial:

cgrand 2018-03-29T15:05:46.000529Z

=> (complete "(let [x (mapc" 12 'user)
({:candidate "map", :roffset -3, :len 3}
 {:candidate "map?", :roffset -3, :len 3}
 {:candidate "mapv", :roffset -3, :len 3}
 {:candidate "mapcat", :roffset -3, :len 3}
 {:candidate "map-entry?", :roffset -3, :len 3}
 {:candidate "map-indexed", :roffset -3, :len 3})

cgrand 2018-03-29T15:06:21.000835Z

so here the completion is asked as if the caret ws between p and c

volrath 2018-03-29T15:06:37.000302Z

with compliment I'm getting the type and ns of each candidate

volrath 2018-03-29T15:06:52.000459Z

type as in: function, keyword, namespace, blah blah

cgrand 2018-03-29T15:06:59.000245Z

do you use this info?

volrath 2018-03-29T15:07:02.000184Z

yes

cgrand 2018-03-29T15:07:09.000787Z

how?

volrath 2018-03-29T15:07:14.000301Z

sec..

cgrand 2018-03-29T15:07:59.000073Z

map|c so how would you substitue the candidate map?? and mapcat?

volrath 2018-03-29T15:08:05.000654Z

<f> as in function...

volrath 2018-03-29T15:08:24.000192Z

oh

volrath 2018-03-29T15:08:32.000242Z

I can't upload files to the channel

volrath 2018-03-29T15:08:33.000740Z

haha

cgrand 2018-03-29T15:08:49.000715Z

I see your screenshot just fine

volrath 2018-03-29T15:09:08.000321Z

oh ok... I got a message saying that the workspace doesn't have any storage space left

volrath 2018-03-29T15:10:20.000113Z

the screenshot comes from cider (I'm doing cljs right now), but I'm using the same underlying lib to display the candidates in spiral

cgrand 2018-03-29T15:11:00.000676Z

and do you pass the context or only the prefix to compliment?

dominicm 2018-03-29T15:12:17.000222Z

Vim has an option to show doc string while completing options. It would be good to support that.

cgrand 2018-03-29T15:13:08.000216Z

@dominicm can’t it be done in two steps? or does vim show all docstrings at once?

dominicm 2018-03-29T15:13:59.000632Z

@cgrand it can't, but not for that reason. It's not possible to show it in response to a callback.

volrath 2018-03-29T15:14:07.000198Z

I also pass the context

cgrand 2018-03-29T15:14:41.000885Z

also = in addition to the prefix?

volrath 2018-03-29T15:15:55.000283Z

if I remember correctly, I pass the context as a string with a __prefix__

cgrand 2018-03-29T15:16:07.000563Z

ok

volrath 2018-03-29T15:20:05.000641Z

yes I double checked, I pass context with a __prefix__ substring

2018-03-29T16:20:24.000532Z

2018-03-29T16:37:58.000325Z

2018-03-29T17:54:03.000222Z

@cgrand I use my own action which uses compliment under the hood and then transforms/enriches things for vim to digest.

2018-03-29T18:00:25.000907Z

Trying to make screenshot.

2018-03-29T19:02:29.000586Z

From compliment I get the candidate and it's type. I look up the various information based on the type. Compliment accepts the prefix and the namespace. The action looks like this:

2018-03-29T19:02:40.000513Z

`#:vimpire {:complete (vimpire.complete/completions #unrepl/param :prefix #unrepl/param :nspace)}`

2018-03-29T19:07:39.000541Z

Unrepl is now shaded as: vv-8zmXciM-6qVYjA3apmzDNEzIgiRFW7EXcizIgAqgAAU.un repl.repl.

2018-03-29T19:08:48.000465Z

This also gets rid of the unshaded unrepl.core since I can make it available to the other shading units in its unshaded form.

2018-03-29T22:42:50.000241Z