test-check

2020-12-05T13:53:58.085100Z

`:reporter-fn`
    A callback function that will be called at various points in the test
    run, with a map like:
      ;; called after a passing trial
      {:type            :trial
       :args            [...]
       :num-tests       <number of tests run so far>
       :num-tests-total <total number of tests to be run>
       :seed            42
       :pass?           true
       :property        #<...>
       :result          true
       :result-data     {...}}
      ;; called after the first failing trial
      {:type         :failure
       :fail         [...failing args...]
       :failing-size 13
       :num-tests    <tests ran before failure found>
       :pass?        false
       :property     #<...>
       :result       false/exception
       :result-data  {...}
       :seed         42}
    It will also be called on :complete, :shrink-step and :shrunk. Many
    of the keys also appear in the quick-check return value, and are
    documented below.
☝️ an arg to clojure.test.check/quick-check , which you could monkey-patch if you aren't actually calling it directly

2020-12-05T15:13:10.085400Z

Thanks for that tip.

2020-12-05T15:13:42.086200Z

I know that shrink-loop in my case is not finishing within 10,000 iterations, since I put in some logging and let it run for a while.

2020-12-05T15:15:30.087500Z

I guess I might let it run for a day or so, out of curiosity to see if it actually finishes. In the mean time, it does seem like some kind of option to force shrink-loop to return early would be useful in some cases, as I think you may have mentioned earlier.

2020-12-05T15:16:10.087900Z

Or perhaps there is one, and I haven't read the docs yet.

2020-12-05T15:16:42.088700Z

I would have added that feature a long time ago, but I got stuck being unable to decide on a coherent way of configuring test.check given the >=2 different usage styles

2020-12-05T15:17:17.089300Z

also once you start returning from shrinks early people will want a feature for resuming shrinks, and that's even more complicated

2020-12-05T15:18:26.089800Z

looks like there's no error-handling on the reporter-fn , so technically you could just throw an exception from there 😄

2020-12-05T15:18:51.090100Z

That is certainly one way to do it. Let me try that.

2020-12-05T15:20:49.090700Z

it might not be trivial to get it to accurately return a "smallest failing case so far"

2020-12-05T15:21:10.091100Z

shouldn't be hard to get it to return a "most recently failing case"; maybe that's the same thing, I'm not quite sure 🙂

2020-12-05T15:21:19.091300Z

I see current-smallest

2020-12-05T15:21:31.091700Z

oh, perfect

2020-12-05T15:21:34.091900Z

which seems to be passed to reporter-fn whenever a new one is found