`:reporter-fn`
A callback function that will be called at various points in the test
run, with a map like:
;; called after a passing trial
{:type :trial
:args [...]
:num-tests <number of tests run so far>
:num-tests-total <total number of tests to be run>
:seed 42
:pass? true
:property #<...>
:result true
:result-data {...}}
;; called after the first failing trial
{:type :failure
:fail [...failing args...]
:failing-size 13
:num-tests <tests ran before failure found>
:pass? false
:property #<...>
:result false/exception
:result-data {...}
:seed 42}
It will also be called on :complete, :shrink-step and :shrunk. Many
of the keys also appear in the quick-check return value, and are
documented below.
☝️ an arg to clojure.test.check/quick-check
, which you could monkey-patch if you aren't actually calling it directlyThanks for that tip.
I know that shrink-loop in my case is not finishing within 10,000 iterations, since I put in some logging and let it run for a while.
I guess I might let it run for a day or so, out of curiosity to see if it actually finishes. In the mean time, it does seem like some kind of option to force shrink-loop to return early would be useful in some cases, as I think you may have mentioned earlier.
Or perhaps there is one, and I haven't read the docs yet.
I would have added that feature a long time ago, but I got stuck being unable to decide on a coherent way of configuring test.check given the >=2 different usage styles
also once you start returning from shrinks early people will want a feature for resuming shrinks, and that's even more complicated
looks like there's no error-handling on the reporter-fn
, so technically you could just throw an exception from there 😄
That is certainly one way to do it. Let me try that.
it might not be trivial to get it to accurately return a "smallest failing case so far"
shouldn't be hard to get it to return a "most recently failing case"; maybe that's the same thing, I'm not quite sure 🙂
I see current-smallest
oh, perfect
which seems to be passed to reporter-fn whenever a new one is found