unrepl

discussing specification of an edn-based repl and its implementations.
richiardiandrea 2017-07-27T06:54:05.665860Z

@dominicm that is a very interesting question and I am not a lawyer but it seems like we are not reditributing/copying things but taking and running the files only

pesterhazy 2017-07-27T07:36:39.366187Z

@dominicm do you mean we should bundle the license with the blob?

dominicm 2017-07-27T07:37:37.383131Z

@pesterhazy potentially yesyes. This is what JavaScript minifiers do (see most cljs advanced builds contain the react license)

pesterhazy 2017-07-27T07:38:14.393939Z

I think that would make sense

richiardiandrea 2017-07-27T07:45:52.535271Z

I am thinking of repackaging https://github.com/Lambda-X/boot-pack-source, maybe a rename to boot-sources makes sense but naming is hard

richiardiandrea 2017-07-27T07:45:58.537206Z

suggestions? 😄

richiardiandrea 2017-07-27T07:46:28.547053Z

the lib contains a couple of packages for moving (source) files within the fileset

richiardiandrea 2017-07-27T07:46:49.553809Z

like pack-source takes deps and moves the sources files to a custom folder

richiardiandrea 2017-07-27T07:48:11.579692Z

and this will include an unrepl-blob task for making the blob like so boot pack-source unrepl-blob

pesterhazy 2017-07-27T08:17:22.166045Z

what would be the advantages over what @cgrand has already built?

dominicm 2017-07-27T08:42:43.705509Z

I like the names: uglifyclj, replpack and replify, named after their js counterparts 😛

richiardiandrea 2017-07-27T08:52:48.927148Z

@pesterhazy how do you mean? did you see the snipped above? I am basically blobifying/concatenating arbitrary deps

pesterhazy 2017-07-27T08:53:31.943005Z

@richiardiandrea so this is for building a blob that includes, e.g. compliment?

richiardiandrea 2017-07-27T08:53:38.945645Z

yep

pesterhazy 2017-07-27T08:53:41.946702Z

sorry I'm being a bit slow today

richiardiandrea 2017-07-27T08:53:58.952970Z

no no problem, I thought I was missing some point 😉

pesterhazy 2017-07-27T08:54:07.956Z

have you tried sending it to the repl?

pesterhazy 2017-07-27T08:54:13.958331Z

e.g. using netcat

richiardiandrea 2017-07-27T08:54:21.961115Z

not yet, today's goal

pesterhazy 2017-07-27T08:55:26.985460Z

(cat unrepl-blob.clj compliment-blob.clj - ) | java -jar clojure.jar -m clojure.main)

👍 1
richiardiandrea 2017-07-27T09:17:25.474148Z

uhm first attempt is weird:

cat target/unrepl/blob.clj | java -cp "$CLOJURE_EXT" clojure.main 
Clojure 1.8.0
user=> #'compliment.core/documentation
user=> 
$ ...exited to shell...

richiardiandrea 2017-07-27T09:19:23.517108Z

it is evaluated correctly though, because the printed var is the last var of compliment.core

richiardiandrea 2017-07-27T09:19:57.529651Z

Oh I need the - wow...bash magic

richiardiandrea 2017-07-27T09:20:12.535348Z

$ cat target/unrepl/blob.clj - | java -Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}" -cp "$CLOJURE_EXT" clojure.main 
Clojure 1.8.0
user=> #'compliment.core/documentation
user=> (compliment.core/completions "def")
({:candidate "def", :type :special-form} {:candidate "defn", :type :macro, :ns "clojure.core"} {:candidate "defn-", :type :macro, :ns "clojure.core"} {:candidate "defonce", :type :macro, :ns "clojure.core"} {:candidate "deftype", :type :macro, :ns "clojure.core"} {:candidate "defmacro", :type :macro, :ns "clojure.core"} {:candidate "defmulti", :type :macro, :ns "clojure.core"} {:candidate "definline", :type :macro, :ns "clojure.core"} {:candidate "defmethod", :type :macro, :ns "clojure.core"} {:candidate "defrecord", :type :macro, :ns "clojure.core"} {:candidate "defstruct", :type :macro, :ns "clojure.core"} {:candidate "defprotocol", :type :macro, :ns "clojure.core"} {:candidate "definterface", :type :macro, :ns "clojure.core"} {:candidate "default-data-readers", :type :var, :ns "clojure.core"})
user=>

🍻 1
🍾 1
🦜 1
🎉 3
richiardiandrea 2017-07-27T09:20:57.551931Z

this is awesome 😉

dominicm 2017-07-27T09:25:09.644270Z

Was the decision made to not to ns munging of some kind?

richiardiandrea 2017-07-27T09:32:49.816254Z

no well this is my own trial and error, no decision has been taken

dominicm 2017-07-27T09:33:32.831654Z

ah, okay. just checking.

pesterhazy 2017-07-27T10:10:43.627369Z

just so you know, beer and champagne doesn't mix well

✅ 1
pesterhazy 2017-07-27T10:12:14.657448Z

it should be pretty easy to swap this in in unravel https://github.com/pesterhazy/unravel/blob/master/src/unravel/loop.cljs#L77

pesterhazy 2017-07-27T10:13:03.673953Z

to add another blob you could add it here: https://github.com/pesterhazy/unravel/blob/7d4f48c033fe6da2f87d20764d614321e421fd52/src/unravel/loop.cljs#L123

richiardiandrea 2017-07-27T10:16:02.732424Z

@pesterhazy do you vendor the blob? or you build it programmatically on release

richiardiandrea 2017-07-27T10:16:30.741546Z

I yes I see it is just vendored

pesterhazy 2017-07-27T10:20:33.819835Z

correct, there's also scripts/update-unrepl

richiardiandrea 2017-07-27T10:23:59.887022Z

oh well, that will come later...I guess at this point what will go in the blob will only depend on the client

richiardiandrea 2017-07-27T10:24:41.900740Z

the script can change in order to use boot for building it

richiardiandrea 2017-07-27T10:36:51.130374Z

uhm I tried and [:connect true] is the only output I get unfortunately

pesterhazy 2017-07-27T10:40:20.194093Z

hmm

pesterhazy 2017-07-27T10:40:29.196787Z

could you put up a branch?

richiardiandrea 2017-07-27T10:41:03.207005Z

sure, on my fork? or you can give me access wink wink nudge nudge 😄

richiardiandrea 2017-07-27T10:52:00.402431Z

so just to be sure I did:

cat ~/git/unravel/resources/unrepl/blob.clj - | java -Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}" -cp "$CLOJURE_EXT" clojure.main
and it works.

pesterhazy 2017-07-27T10:59:33.536277Z

looking at it now

pesterhazy 2017-07-27T10:59:41.538689Z

how did you create the blob?

pesterhazy 2017-07-27T11:00:01.544869Z

could you add the compliment blob as a separate file to the repo, rather than pre-concatenating?

richiardiandrea 2017-07-27T11:28:57.052136Z

just a sec a bit busy now 😉 basically this is the core of it:

(let [code (->> blob-files (mapv slurp) str/join)]
            (butil/trace* "Blob:\n%s\n" (butil/pp-str code))
            (io/make-parents output-file)
            (butil/dbug "Writing blob to %s...\n" (.getName output-file))
            (spit output-file
                  (prn-str
                   `(let [prefix# (name (gensym))
                          code# (.replaceAll ~code "unrepl\\.(?:repl|print)" (str "$0" prefix#))
                          rdr# (-> code# java.io.StringReader. clojure.lang.LineNumberingPushbackReader.)]
                      (try
                        (binding [*ns* *ns*]
                          (loop [ret# nil]
                            (let [form# (read rdr# false 'eof#)]
                              (if (= 'eof# form#)
                                ret#
                                (recur (eval form#))))))
                        (catch Throwable t#
                          (println "[:unrepl.upgrade/failed]")
                          (throw t#)))))))

richiardiandrea 2017-07-27T11:35:37.167834Z

haven't really check the code that was in the blob, so maybe there is something else to do there (for example I see a replaceAll now

richiardiandrea 2017-07-27T11:36:13.177944Z

oh well, of course it does not work 🤦

richiardiandrea 2017-07-27T11:36:40.186164Z

there are no unrepl specific blobs

richiardiandrea 2017-07-27T16:14:32.996106Z

./scripts/run localhost 5555
Unravel 0.2.2 connected to localhost:5555

Type ^O for full docs of symbol under cursor, ^D to quit
Enter #__help for help

user=> (compliment.core/completions "def")
({:candidate "def", :type :special-form} {:candidate "defn", :type :macro, :ns "clojure.core"} {:candidate "defn-", :type :macro, :ns "clojure.core"} {:candidate "defonce", :type :macro, :ns "clojure.core"} {:candidate "deftype", :type :macro, :ns "clojure.core"} {:candidate "defmacro", :type :macro, :ns "clojure.core"} {:candidate "defmulti", :type :macro, :ns "clojure.core"} {:candidate "definline", :type :macro, :ns "clojure.core"} {:candidate "defmethod", :type :macro, :ns "clojure.core"} {:candidate "defrecord", :type :macro, :ns "clojure.core"} #__1)
user=> (print
print         printf        println       print-dup     print-str     print-ctor    println-str
print-method  print-simple  

user=> (print

richiardiandrea 2017-07-27T16:16:02.049530Z

compliment is super cool, I added the option as comment in the code, neat stuff can be achieved (like complete locals, but the surrounding form needs to be sent)

richiardiandrea 2017-07-27T16:16:28.064614Z

cmd-complete could actually accept the same options that we just forward to compliment

richiardiandrea 2017-07-27T16:42:44.978286Z

@bbrinck has also a very cool library for error parsing that we could make use of in unravel: https://github.com/bhb/expound

👀 1
richiardiandrea 2017-07-27T16:43:49.016144Z

this opens the door to some interesting question, how to send blobs conditionally? Expound makes sense only when clojure.spec is loaded server side

cgrand 2017-07-28T09:53:52.775043Z

Side loading requires a dedicated connection (and client code for this connection). This is the price.

dominicm 2017-07-28T09:59:15.886320Z

I don't understand the dedicated connection requirement.

richiardiandrea 2017-07-28T10:16:42.220207Z

I think that for solving the particular problem above, the blob that contains expound for instance, could refuse to eval itself if it does not find its requirements (`(find-ns 'clojure.spec.alpha)` for instance)

richiardiandrea 2017-07-28T10:19:20.266955Z

Also the number of blob combinations is potentially infinite and we might want to share them (example, the compliment one)...this actually makes the new tools.deps.alpha a good candidate for fetching things off topic 🙂

dominicm 2017-07-28T10:20:16.284489Z

or generally pomegranate, no?

cgrand 2017-07-28T10:23:10.336017Z

@dominicm loading can occur anywhere even in the middle of a read so it can't share the same input stream

dominicm 2017-07-28T10:23:56.349356Z

@cgrand gotcha, so unravel would be forced to maintain a separate (or multiplexed) connection in order to support this feature (if it wanted to)

cgrand 2017-07-28T10:24:51.364786Z

@richiardiandrea I'm under the impression we are slightly talking past each other.

👍 1
dominicm 2017-07-28T10:26:50.399245Z

This does somewhat open up the desire for a multiplexer again I suppose. I know it was canned for complexity, but just raising the point that it would remove the scary of maintaining a dedicated connection.

cgrand 2017-07-28T12:57:09.073409Z

@dominicm multiplexing was not canned for complexity but for not being core. It's one way to create multiple connections. So it's rather orthogonal and clients willing to use it should add it to their blobs.

dominicm 2017-07-28T13:32:21.026035Z

So not necessarily a dedicated connection then

cgrand 2017-07-28T16:36:00.209532Z

@dominicm it's still a connection (it needs to be handled separately by the client after demux)

richiardiandrea 2017-07-27T16:51:19.275816Z

in particular I say I so so like unravel as client but I want to be able to send some additional blob apart from the ones unravel provides. Maybe unravel can expose an option for this use case? This way the user controls what to send (in case she is using clojure alpha-1.9.0-alpha17) to some extent

richiardiandrea 2017-07-27T16:52:50.326792Z

Uhm or maybe a simple series of load-file might suffice once the repl is up

richiardiandrea 2017-07-27T17:24:17.396500Z

ah ! maybe I missed this one, you folks have thought about everything 😄 What is side-loading? Server side, client side? 😄

cgrand 2017-07-27T17:53:13.400171Z

Side-loading is an idea that has been floating around for quite a while (at some point I even had an implementation). Basically it's a way for the server to request a resource from the client. (In java it hooks up the classloading mechanism)

richiardiandrea 2017-07-27T17:59:45.632626Z

ok it is that inversion of control we were talking about yes, I just did not know the name then 😉

dominicm 2017-07-27T19:27:55.570763Z

did those messages send? they're all grey?

dominicm 2017-07-27T20:11:18.920924Z

Looks like no

dominicm 2017-07-27T20:11:31.927497Z

What were the objections to the side loading?

dominicm 2017-07-27T20:36:46.704926Z

https://giphy.com/gifs/nNksEd1ZGdqGk/html5 feels appropriate

cgrand 2017-07-27T21:32:57.322030Z

About the whole blob thing: maybe we make reasoning and problem solving harder by complecting blob generation and namespaces isolation (gensyms etc.)

richiardiandrea 2017-07-29T10:50:46.167517Z

For now I am going to publish a task that does the following things: resolves the deps (already part of boot-sources), sorts in dep order the files, concatenates. One step at the time. We can refine things once problems come up.

👍 1