kaocha

Official support channel: https://clojureverse.org/c/projects/kaocha
2020-04-02T00:11:54.067600Z

so it feels like kaocha-midje doesn't filter running the tests properly, only reporting them.

2020-04-02T00:13:01.068800Z

I'm trying to --focus-meta on a (facts :focus "blah blah" ...) and it recognizes the focus and reports only those tests, but based on the runtime it's running my expensive 1000x spec.gen tests too.

2020-04-02T00:13:09.069100Z

instead of the small, trivial test I asked for.

2020-04-02T00:13:30.069500Z

similarly when it says "failing tests pass, running all tests". it's actually running the lot both times.

bbrinck 2020-04-02T02:41:12.069600Z

I can reproduce here. I’ll dig into it more tomorrow.

bbrinck 2020-04-02T02:52:50.069800Z

It’s a bug in Expound: https://github.com/bhb/expound/issues/191

bbrinck 2020-04-02T02:53:37.070100Z

The workaround is to redefine the fdef spec (or disable instrumentation for the functions in expound):

(s/fdef expound/printer
        :args (s/cat :explain-data (s/nilable map?))
        :ret nil?)

plexus 2020-04-02T07:15:09.071800Z

kaocha-midje is really a PoC. Midje is a strange animal making it quite hard to integrate, and there doesn't seem to be a lot of demand for it anymore. The general recommendation would be to migrate to fudje, since that sits on top of clojure.test it should work well with Kaocha

plexus 2020-04-02T07:15:39.072400Z

My guess is that those tests are not being run by kaocha, by somewhere else e.g. as a side-effect of loading namespaces

plexus 2020-04-02T07:16:15.073Z

You can verify this with --print-test-plan, all testables except the one with :focus should be marked as :kaocha.testable/skip

2020-04-02T13:38:38.076400Z

hm. let me look into fudje. I'm not using any esoteric features of Midje, I just prefer it's style of (expression-under-test) => result and the ergonomics of the checkers.

2020-04-02T18:31:55.082400Z

I've got it running against Fudje now, but it's 0 tests, 0 assertions, 0 failures even after emitting hundreds of failures.

2020-04-02T18:34:40.082700Z

it seems to be working, but the reporter is busted.

2020-04-02T18:34:54.083100Z

I'm going to focus on fixing my tests for now; I can debug into the reporting problem later.

2020-04-02T18:52:39.083500Z

ah, I figured that out in passing - they need to be wrapped in a deftest or the reporting doesn't work.

2020-04-02T19:00:59.084900Z

wow, that's frustrating. a lot of my tests end up comparing my.complicated.Deftype => [similar vector]. (= deftype vector) is true, (= vector deftype) is false, because my equiv is flexible.

2020-04-02T19:01:33.085700Z

Midje => did the right thing, clojure.test's (is (= expected actual)) puts it the wrong way around. I'll have to define a custom checker for this...

2020-04-02T23:29:05.086500Z

I'm running comfortably on Fudje now, with a mix of my original Midje-style tests (lightly massaged to be compatible) and a few troublesome ones converted to clojure.test style.

2020-04-02T23:29:22.086900Z

thanks for the pointer, this is a faster, lighter environment better integrated with Kaocha.