Clojurians Log v2

Starting the main app at CircleCI takes 60 seconds on my machine (3.5 GHz Dual-Core Intel Core i7). I’ve been wondering what is happening for these 60 seconds for that time for some years, and today I spent a little time looking into it. In the past, I have booted our app with a sampling profiler attached, but the data produced is very difficult to work with – loading a Clojure app is recursive in nature – the evaluation the top-level namespace will compile + eval the required namespaces, which in turn compile and eval their own dependencies. The resulting profile has a call-tree which is very deep and has a very low branching factor. This time around I added some coarse instrumentation code to src/jvm/clojure/lang/Compiler.java. From the data that I collected, I was able to confirm that there are no bottlenecks in the launch of our app, we are just loading many files which all take time. This lead to wonder about two approaches to speeding up the time to load our app: • Parallelising the loading of code • Caching the result of evaluations disk Has anyone every tried either of these approaches? I was searching JIRA to find any prior art in this area, but I could not find anything.

Re caching, https://clojure.org/guides/dev_startup_time

Thanks Alex.

I’ll try that approach later on tonight or tomorrow. I’m excited to try it out.

Parallel compilation (really load) is something I’ve looked at but it’s inherently tricky due to the lack of immutable namespaces and locking around loads

Those are both things that we’ve talked about changing in a hand-wavey long term sort of way. Really, there are a bunch of sub problems around making load faster which are probably more tractable (and useful whether parallel or not).

FWIW, in my crude data, the time to eval namespaces dominates the time to compile them. I had difficulty instrumenting the time taken to load Java dependencies in isolation from the time take to eval a form.

your compilation speed will benefit from not compiling, as in the link posted above

what is the distinction between eval namespace and compile them?

FWIW @dnolen had some thoughts on caching compiled deps in a recent defn-podcast.

well not super concrete thoughts for Clojure - just that ClojureScript had to solve this problem out of necessity since we're fundamentally AOT

@ghadi when I say “compile” and “eval” I guess I mean Read and Eval as in REPL. I added some timers around the call to eval here: https://github.com/clojure/clojure/blob/30a36cbe0ef936e57ddba238b7fa6d58ee1cbdce/src/jvm/clojure/lang/Compiler.java#L7636

In my tests, no file took longer than 10ms to in the read phase, and the slowest file to eval was clojure/core/async.clj, which took 1.52 seconds to eval, (plus a further 2.07 seconds to read and eval files that clojure/core/async.clj itself requires).

@slipset - yup, I listened to that podcast recently after a having had to wait for my app to boot many times that day, which gave the imputus to do some research.

the steps involved here are really read, compile, and I'd call it load. I think the eval you're timing there really includes compile (which you can remove using the guide above). core.async is particularly painful due to the giant macro, but that's all cost paid at compile time.

load itself includes both loading classes and initializing vars

compile is probably the slowest part of this, but I don't know of any specific part that's slow, it's just the cost of emitting a lot of bytecode

loading vars is something with known costs and there are some options there to make that stuff lazier (Rich made a lazy-vars impl a while back and Ghadi has a version using some of the newer indy guards that mitigates some of the downsides of laziness)

pulling way back from an app perspective, I find it helpful to think about what you actually need to do as an app to be "up" and whether you can defer parts of that loading until later. can you load 100 namespaces instead of a 1000 before you are "ready" and then load the rest as needed? (usually you can)

Thanks Alex, I really appreciate your time on this, > can you load 100 namespaces instead of a 1000 Yeah, I’ve been thinking along the same lines. The situation that I find myself in is that our main Clojure app has grown to be cumbersome and resistant to change. It’s slow to start, the tests are slow to run, and it is hard to run the tests reliably locally. This hinders refactoring efforts. So yes, I could factor the app better, but making those changes will be slow, and the slow compile time feeds back into that cycle.

I think it’s time for bed - my experimenting with emitting classes has hit some road-blocks which have dampened my spirits! Someone decided a namespace should have the following two functions:

(defn- identity->Identity

...

(defn- Identity->identity

Execution error (NoClassDefFoundError) at java.lang.ClassLoader/defineClass1 (ClassLoader.java:-2).
federations_service_client/core$identity__GT_Identity (wrong name: federations_service_client/core$Identity__GT_identity)

yes, that's pretty evil

it's actually totally fine but bad things will happen on case insensitive file systems

when compiling in-memory, nothing hits disk and this isn't an issue, but you'd have this same problem if you aot compiled or uber jarred on a case insensitive file system like mac