Clojurians Log v2

Guy who wrote Clojure book in 2012 and wrote many Clojure OSS libs: https://twitter.com/cemerick/status/1111110485510942720

I certainly would like some hand-holding by the compiler, esp. when dealing with code (not data) — arity exceptions or unresolved symbols are examples of this, perhaps though we could have more of those?

@orestis 🙂 https://twitter.com/borkdude/status/1111238854525235200

The CLJS compiler already warns about this when you compile the code. It would be nice if CLJ did the same

The joker linter helps with unresolved symbols.

Yeah I’m following clj-kondo with great interest 😄

I had some fun years ago building some prototype of this for Python

Turns out that while people could write extremely indirect and dynamic code, most of the time they don’t — so covering for the usual use cases with static analysis can be a real time-saver, if you get feedback in your editor.

right, that’s what I’m trying. it won’t be perfect, but covers 90% of the cases

@borkdude are you going to eventually cover the joker use cases too? or always complement them?

I’ll prioritize things that it doesn’t do yet, but they may overlap. E.g. the wrong arity only worked in joker for calls within the same namespace, but for clj-kondo they also work across namespaces

How far could one go, theoretically, with these static analysis tools?

80%

cemerick is a notable troll on twitter (I still like him). OCaml's super cool tho and I do miss the typechecker at times

Sounds pessimistic

I don’t know. Runtime analysis is always easier

I mean, you can do more things.

What about about ephemeral runtimes for compile time analysis?

Maybe that’s what joker does, kind of, I’m not sure. Since it’s a light weight interpreter

right

I’m mainly focussing on a lightweight tool that I can use to get “stupid” errors really fast. E.g. arity checking while editing CLJS files without a backing REPL

I bet eastwood can detect a lot more stuff since it’s a runtime tool, for example

but then you need a JVM running

so tradeoffs. you can use these tools together in CI for example

do you have any experience in OCaml?

I guess you'd need some fixture data for a kind of static analyzer that could exercise runtime scenarios

do you mean a test set?

I wrote some small applications with it a little over a year ago. I participated a lot in the ReasonML community early on

yeah

I already have that. We have a 30kloc app with tons of deps. I just have to analyze the classpath to get a ton of warnings and errors 😉

30kloc CLJ and 18kloc CLJS actually. And this is only the main app, we have other components too 🙂

well, I don't mean test data for tests...

I mean, almost like for generative tests, so that a static type system can do some run-time-ish things

I don’t really see what you mean

Yeah, I'm not sure if it's really a coherent idea

I do expand certain macros already, so you can see that (-> 1 (inc 1)) is an arity error.

so everything that’s purely syntactic can be done

Just wondering how you could maybe capture some dynamicity of the repl for static analysis at compile time

I had a similar amount of experience with Clojure at the time, and our team was trying to decide what to build our new system on top of. I debated between Clojure and OCaml and landed on Clojure 🙂

been here ever since!

I plan to scan fdefs to offer a very limited/basic form of type checking on literals

joker takes this a bit further already, but only for core functions

but as soon as the arguments are non-literals, it gets very hard

Yeah, but joker recreates the interpreter in go... why not just use a full clojure interpreter?

then you’re basically implementing core.typed and I’m not planning to do that 🙂

I mean, joker is faster

I guess graal isn't going to help there

do you mean, implement a clojure interpreter in clojure and compile with Graal?

Well, whatever joker is doing, doing a lightweight interpreter

can one get further with a Graal solution?

that would be like implementing “eval” which is currently not supported with GraalVM 🙂

I mean, you could do the same thing as joker but then in Java.

right, that's what I thought

I’m not sure which problems the joker currently detects that cannot be done with static analysis

BTW, Cursive also uses static analysis, might also be a source of inspiration. I’m not using it ATM

cool 🙂

could your re-find utility be used to potentially provide a predictive-next-token thing for an ide?

For the “interpreter” bit, why not just use JVM/eastwood for that?

I would be surprised if I could use that, since I suspect that will require eval

I was thinking of a feature like:

(s/fdef foo :args (s/cat :i int?))
(foo "foo") ;;=> invalid argument
(foo _) ;;=> type int expected in hole

(like Haskell type holes, but much more limited)

I think spectrum is doing this, but much more advanced

Yeah, like, if the the thing that follows a thing is expected to be a literal string, maybe the IDE can start the string form for you, with the cursor between the quotations

... I don't know... just wondering if re-find's inference system can be used to infer code templates or whatnot for certain code contexts in an ide

maybe, who knows what the future holds 🙂

I made a toy little utility https://github.com/johnmn3/coal-mine2vec that can find analogies between relations between functions. I've been wondering how I could combine it with your re-find tool to do some interesting kinds of inference.

that’s cool! 🙂

I guess your tool can be used to give suggestions: “also see …” on clojuredocs for example

Or re-find could also use it. It would have to run in JavaScript though. Maybe the result of the analysis could be emitted to some static EDN graph structure

Re-find is pretty simple. It just tries to find functions whose specs match the given in/output. There’s not really anything “inferency” to it, or maybe I’m misunderstanding what you mean with inference.

Well, the word2vec thing is just looking over swaths of text and if it sees things in similar proximity to one another, but in different contexts, it'll fill in the blank for you... which seems similar to your re-find... Not sure how they could be combined yet though...

you can create an automatic network of related functions this way right? and this could be useful in suggesting things

RIght

Like, subdued text of what might come next

But in conjunction with your re-find thing, you could eliminate fns that don't work from a type perspective

Some kind of predictive coding tool

and if you're talking about pure functions, you could interpret some candidate options a priori and see if they're invalid suggestions

@borkdude I wonder if it'd be useful to gen-spec a mountain of code using speculative and run it through word2vec... See if it can do fast inference on code relations.

you mean generate code using speculative?

that should work, all the tests are already using generative testing