Hmmm. I'm facing an interesting issue. Running clj-kondo --parallel --lint "$(lein classpath)"
takes about 37s to write ~73kb of cache. Running kondo/run!
from the lein repl with the same options actually results in OOM after 1.5minutes of running. The repl has a xmx of 1G.
Isn't a Xmx value of 1G pretty low though? I tend to set it around 18G, even in laptops with 32GB available (64GB currently).
Note that such a Xmx
does not allocate that much memory upfront: other processes aren't affected unless your java process actually uses that much RAM (which tends to be a transient peak anyway - then GC kicks in and RAM is available system-wide again)
Personally I tend to have the opinion that Clojure is simply more memory-hungry than Java, so sophisticated programs can have surprising memory requirements, given a sufficiently large input.
There's a slight (but not strictly related) hint of this here: https://github.com/clj-commons/rewrite-clj/issues/69#issuecomment-767914874 a sophisticated clj program needed a higher stack size simply because of it choice of clojure and recursive algorithms (`clojure.walk`).
The interesting part being that there was no 'bug' - simply one needed a -X
value that is suitable to the caliber of the program being run
...one cheap experiment you can make is increasing Xmx very (but not exceedingly) generously, say to 8G
. Then run clj-kondo while you observe the Heap
graph in your visual profiler of choice.
Say the heap peaked at 1.6GB. Then it becomes an educated guess to set Xmx
at 2G (if you still want that minimalism) and assume that peaks won't OOM you
That's a good idea. I'll try it out soon. 1Gb is relatively low i guess, but it's more than enough for the project i'm running. So I expected (maybe unduly) that clj-kondo wouldn't use a gig just to analyse that project.
This is the stacktrace.
I'm not sure where to start looking. Although i think it's interesting it's reporting around infinite / lazy sequences.
If i 'window' over the dependencies (e.g. call run! in steps of 50/100 files at a time) it's fine.