Clojurians Log v2

Uploaded an alternative patch to http://dev.clojure.org/jira/browse/TCHECK-126 (Refactor c.t.c/quick-check as a state machine to provide more extension points). The new patch refactors c.t.c/quick-check "in-place", instead of introducing a separate namespace like in the old patch. Most important change is that it replaces the reporter-fn with a step-fn which allows feeding back the quick-check loop to drive/augment it by returning a modified quick-check state (for example, adding a timestamp, or dynamically changing the number of tests left). reporter-fn was for side-effects only, step-fn allows to do the side-effects AND modify the qc state. I understand this is not a breaking change because the reporter-fn has not been part of a release yet AFAIK

so step-fn is part of the public API then?

yes, through the new step-fn option that replaces reporter-fn. There's also a c.t.c.clojure-test/default-step-fn now

here's the new default-step-fn: https://github.com/clojure/test.check/compare/master...nberger:test.check2-reporter-fn#diff-fb4b33c15ed48f22f9f6c3655f16da02R29

hm

I'd want to be sure that creating a well-behaved and useful step-fn isn't more difficult than a reporter-fn

yeah, that's a risk... we could check for nil, but not sure if much more than that... maybe check that we get a map with certain keys back? But I don't know, passing a step-fn is a bit of "advanced usage", and reading a little bit about it or seeing an example is not such a big expectation, maybe... Here's an example of a step-fn: https://github.com/clojure/test.check/compare/master...nberger:test.check2-reporter-fn#diff-5eb045ad9cf20dd057f8344a877abd89R985

and here's a more useful one: https://github.com/clojure/test.check/compare/master...nberger:test.check2-reporter-fn#diff-5eb045ad9cf20dd057f8344a877abd89R1042

perhaps we could provide some specs for the step-fn...

another option is to keep separate functions for the side-effecty part (`reporter-fn`) and the modify-qc-state part (`step-fn`? will need help with the names 😄 )

but I don't know... maybe that's more appropriate for a layer over this (a step-fn that composes a reporter-fn and a change-qc-state-fn or whatever)

what uses do you imagine a user having for a step-fn that aren't possible with reporter-fn?

1. implementing the stats feature 2. add timestamps at different stages of the quick-check test 3. dynamically change for how long the test should run (by changing the num-tests key, or by introducing a new abort? key in the future) 4. bound tests and shrinks in time (maybe this goes under the previous item, but it's a long-standing request in TCHECK-8 and this way it could be implemented as an extension)

and others that users will come up with 🙂. Another benefit IMO is that it the qc state has the same shape at all stages of the quick-check run (except for added keys during shrink that don't make sense before it starts, for example), so it should make it marginally easier for say an editor to integrate with this to provide live status of the test.check progress

Not sure if it gives any benefit for clojure.spec, that's something to be explored...

Another benefit: the periodic reporter could be re implemented to not use a global atom quite easily

I keep wondering if a rewrite of the generators namespace could result in a more coherent notion of size; it seems hard to come up with one

an initial thought is that sizing generators could be easier to understand if they tried to interpret size as a suggested value for (comp count pr-str)

or a suggested maximum value

but there are lots of cases where that's not a useful default, so users would have to be more involved with sizing to get what they want

and it would also be hard to pull off, especially for combining collection generators with custom generators using gen/bind

it'd be useful to know how well the other quickchecks do at this

I don't like that clojure.spec needed to do some weird depth-tracking thing