clj-kondo

https://github.com/clj-kondo/clj-kondo
2021-02-22T21:03:22.009200Z

Hmmm. I'm facing an interesting issue. Running clj-kondo --parallel --lint "$(lein classpath)" takes about 37s to write ~73kb of cache. Running kondo/run! from the lein repl with the same options actually results in OOM after 1.5minutes of running. The repl has a xmx of 1G.

vemv 2021-02-23T20:39:18.019Z

Isn't a Xmx value of 1G pretty low though? I tend to set it around 18G, even in laptops with 32GB available (64GB currently). Note that such a Xmx does not allocate that much memory upfront: other processes aren't affected unless your java process actually uses that much RAM (which tends to be a transient peak anyway - then GC kicks in and RAM is available system-wide again)

vemv 2021-02-23T20:40:50.019200Z

Personally I tend to have the opinion that Clojure is simply more memory-hungry than Java, so sophisticated programs can have surprising memory requirements, given a sufficiently large input.

vemv 2021-02-23T20:43:44.019400Z

There's a slight (but not strictly related) hint of this here: https://github.com/clj-commons/rewrite-clj/issues/69#issuecomment-767914874 a sophisticated clj program needed a higher stack size simply because of it choice of clojure and recursive algorithms (`clojure.walk`). The interesting part being that there was no 'bug' - simply one needed a -X value that is suitable to the caliber of the program being run

vemv 2021-02-23T21:08:02.019900Z

...one cheap experiment you can make is increasing Xmx very (but not exceedingly) generously, say to 8G. Then run clj-kondo while you observe the Heap graph in your visual profiler of choice. Say the heap peaked at 1.6GB. Then it becomes an educated guess to set Xmx at 2G (if you still want that minimalism) and assume that peaks won't OOM you

1
2021-02-24T15:43:03.035300Z

That's a good idea. I'll try it out soon. 1Gb is relatively low i guess, but it's more than enough for the project i'm running. So I expected (maybe unduly) that clj-kondo wouldn't use a gig just to analyse that project.

2021-02-22T21:04:08.009300Z

This is the stacktrace.

2021-02-22T21:04:39.009700Z

I'm not sure where to start looking. Although i think it's interesting it's reporting around infinite / lazy sequences.

2021-02-22T21:09:43.009900Z

If i 'window' over the dependencies (e.g. call run! in steps of 50/100 files at a time) it's fine.