testing

Testing tools, testing philosophy & methodology...
seancorfield 2021-06-16T01:43:07.041200Z

I’ve been playing around with Cloverage a bit over the last couple of days — https://github.com/cloverage/cloverage — using it via the Clojure CLI / deps.edn. How useful do folks find test coverage? Are there quirks of Cloverage that make it less useful? (I suspect so, based on my experiments so far) For folks who are looking at coverage under tests, do you find yourself adding “minimal” tests just to improve coverage, rather than actually useful tests?

vemv 2021-06-16T01:51:12.044600Z

Last time I used it in a team setting, indeed its issues made it hard to make it a "hard check" in CI. We ran it as an informative step - people could click around and see the stats. (Other informative steps can be e.g. reflection / boxed-math warnings) At least the situation has improved a bit over the last two years

vemv 2021-06-16T01:52:13.045400Z

One thing that has worked well for me is having a checklist as a PR template. It can have a checkbox for test coverage self-assessment

vemv 2021-06-16T01:55:21.045500Z

in some projects we couldn't run it at all luckily the architecture avoided "mega" codebases - by using multirepos or clean monorepos - one has N codebases, N test suites. So one can discard Cloverage only in a subset of those test suites

seancorfield 2021-06-16T02:47:47.045700Z

Yeah, I tried it on our whole codebase about over 400 tests threw exceptions. Running it on individual subprojects seems to be safe enough so far. There seem to be some constructs it just doesn't do very well with -- it flags clojure.tools.logging calls, for example, as not being "fully-exercised" (even when those lines are executed, so I'm not sure what path it thinks has been missed?), and defonce also seems to trip it up.

lread 2021-06-16T12:54:45.048100Z

I make use of cloverage for rewrite-clj, and have https://github.com/clj-commons/rewrite-clj/blob/main/doc/02-developer-guide.adoc#code-coverage.

1👍