Clojurians Log v2

Hey folks. I’ve recently switched over to ClojureScript from JavaScript so I’m especially excited set up kaocha for my team at work and show them how watch mode will be so much easier to use than karma (from shadow-cljs). Unfortunately, I’m stuck with an exception like this issue where the JS namespaces aren’t loaded. https://github.com/lambdaisland/kaocha-cljs/issues/25

It seems like the issue author and I are both missing some understanding of the cljs.analyzer. Would it make sense that the next thing to try to debug the issue would be to start kaocha.runner with some debug breakpoints in cljs.analyzer to see why we’re getting that error from this function? https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/analyzer.cljc#L2575-L2610

I’d expect that the node-module-dep? would be true :thinking_face: :

(ns work.interop.axios
  (:require ["axios" :as axiosjs] ...))

The runner is running in Clojure, so really that’s checking if the module name is in (:node-module-index env/compiler). That seems to be updated in two spots: • https://github.com/clojure/clojurescript/blob/8f38049d543b04b8da55029f140b6577e3ec245a/src/main/clojure/cljs/closure.clj#L2871-L2874 • https://github.com/clojure/clojurescript/blob/9a8196ebfe4265feda88a06de84affb9df469012/src/main/clojure/cljs/env.cljc#L46-L57

So … would it make sense to debug this by figuring out what the compiler options are for the js and node dependencies are? If yes, any hints at how to debug Kaocha with a local copy of ClojureScript? I’m assuming you’d check out both and configure deps.edn to point to the local copies? 🤞

Kaocha is not yet compatible with shadow-cljs. You can try using it, but shadow's compilation model is different because of how it deals with node modules. For some of them you might be able to swap them out with cljsjs versions and keep the same require syntax but YMMV.

But yes looking into the compiler would still be a useful exercise, it does in principle have node support. And indeed checking out a local copy and using :local/root would be the way to go.

Sweet. Thanks Arne, and good morning! 😄

I’ll update the issue if/when I figure that out.

So, I’m curious if you need help supporting node modules but I’m relatively new to Clojure, don’t have much time, and don’t want to waste your time. How complicated would it be to set it up the way you’d like @plexus?

I haven’t fully converted to REPL-driven development and so I’m pretty motivated to set up my tests in watch mode for a CLJS project. 😄

Help is always very, very welcome. Especially when it come to CLJS support I don't have all the answers, and could really use people who are willing to dig in, get to an understanding of the various parts involved, and are able to propose solutions. Have a look at the shadow-cljs issue, there's been some discussion and proposals there, but it's mostly backseat driving. The truth is that I'm quite amazed how well kaocha-cljs's approach has worked so far, but there's a lot of complexity and room for improvement. I know a good bit of the clojurescript internals so far but we're basically on uncharted territory trying to reconcile clojurescript's static compilation model with kaocha's dynamic nature. https://github.com/lambdaisland/kaocha-cljs/issues/2

@s_zharinov has taken some initiative to start working towards a solution, we had a pairing session, and the result are these notes (might be a bit cryptic) https://github.com/lambdaisland/kaocha-cljs/blob/ws-controller/notes.org

the tl;dr is this: currently Kaocha uses a clojurescript REPL (modified prepl actually) backed by the standard clojruescript repl-env interface to get an evaluation environment where we can load and invoke tests on the fly. To capture and report test results in realtime we establish a websocket back that sends cljs.test events back to Kaocha to be reported with whatever kaocha (clojure.test) reporter you have configured. The plan is to move away from that and instead generate a "controller" namespace based on the test plan, which depends on any non-skipped tests in the test plan, and which establishes a bidirectional websocket. (currently the websocket is unidirectional). Then we would send events over the WS to invoke certain tests, instead of what we currently do which is sending forms to a REPL.

this would mean we no longer rely on the prepl abstraction, instead we decouple that. There will be a separate initial compilation step, then we launch a JS environment, and then use the WS to coordinate the test run from start to finish. So you can compile however you like (simple, advanced, with shadow, etc.), run however you like (node, browser, etc.).

That's an amazing summary. :)

yup, now someone just needs to do it 😛 I hope @s_zharinov manages to keep at it, it's a good challenge 😄

this should give us more control, the repl environments have been a pain because they are impossible to troubleshoot when things go wrong. From there on we can start looking at other improvements, like reusing JS environments (in the REPL, or when running multiple browser tests, you really want it to reconnect to the existing tab), and also do things like have a nice test output on the page when running in the browser

Sounds like a good challenge indeed. I've got to level up my .clj game and try to keep up. :)

We're working on an opencollective for lambdaisland open source, currently prepping the release announcement. This would be on the table for a planned project that we can tackle with http://gaiwan.co if we can get the budget. Although we'll probably start with smaller stuff, like porting deep-diff to clojurescript.

I'm not so familiar with Clojure's compilation process. Can you compile one file at a time if you're running in watch mode, or is that a full incremental build?

clojure or clojurescript?

Ooh... ClojureScript.

Clojure could do one at a time since it doesn't go through the Google Closure compiler after, right? ClojureScript could probably compile the JS for one file, then redo the Closure Compiler optimizations?

it depends... clojurescript compilation is typically done on a whole project starting from a :main namespace, the compiler env at least needs analyzer information from any dependent namespaces. If you're doing any optimization other than :none then google Closure compiler kicks in, which also does full program analysis, don't think there's an incremental way to to GCC. If you use the lower level API of the clojurescript compiler then you can hang on to the compiler env and reuse that, pretty sure that's for instance what figwheel does.

so yeah this is all stuff to think about. Normally a kaocha run doesn't leave any state around, but in this case we might want to save some state in case you are doing another run in the same process.

Huh. Cool! So, that's in our future too?

Thanks! I've got a lot more to look for when I go debugging and code reading.

I mean it could be, potentially, but none of this is trivial. Huge props to Bruce and Thomas for the work they've put in, building good tooling on top of ClojureScript with good ergonomics is not a small feat.

I'm quite grateful. The community is very welcoming and friendly (so far).

that it definitely is!

Alright, I feel like I'm being distracting. Thanks again, and have a lovely day! Keep at it @s_zharinov ;).

Thanks, I'll do my best 😀