testing

Testing tools, testing philosophy & methodology...
2020-06-28T06:47:43.024600Z

is it possible to reuse clojure.test.check.generators/such-that so that it returns a value from an alternative generator if it runs out of tries?

2020-06-28T07:16:04.026300Z

I would like to implement a function (preferably-such-that pred gen opts) but don’t know how to do it without accessing the internals of clojure.test.check .

2020-06-28T07:16:38.026700Z

any help is welcome

2020-06-28T08:49:35.029Z

Here is my working function. If someone knows a way to avoid tapping into the private function make-gen, please let me know. cc @gfredericks

(defn preferably-such-that
  "A generator that tries to generate values satisfying a given predicate,
   but won't throw an tantrum if it can't."
  [pred gen max-tries]
  (#'gen/make-gen (fn [rng size]
                    (loop [tries-left max-tries
                           rng rng
                           size size]
                      (if (zero? tries-left)
                        (gen/call-gen gen rng size)
                        (let [[r1 r2] (random/split rng)
                              value (gen/call-gen gen r1 size)]
                          (if (pred (rose/root value))
                            (rose/filter pred value)
                            (recur (dec tries-left) r2 (inc size)))))))))

2020-06-28T13:15:31.029700Z

off the top of my head I don't think you can do what you're describing using the public API

2020-06-28T13:19:10.031Z

The intention is that if you use such-that, you ensure that there's a (hopefully high) positive lower bound on the probability of the generator succeeding, such that (ha!) there's some reasonable value of max-tries that makes failure unlikely before the universe ends

2020-06-28T14:14:35.031300Z

@gfredericks thanks

2020-06-28T14:17:36.034300Z

I have another problem, I want a generator of sets which take as input a list of element generators. Formulated differently, I want a generator of tuple where the generated elements can be placed into a set (i.e. they should be distincts). @gfredericks Do you know a simple way to do that with the public API? That’s for my budget-based generator of recursive structures, each elements in my set is given a different part of the parent’s budget.

2020-06-28T14:19:30.034900Z

I don't quite understand what you mean, but presumably you're aware of the *-distinct generators, so you're describing something those can't do?

2020-06-28T14:20:26.036Z

I saw them, but they all take 1 generator as input, and I have n generators, 1 for each element of my set, each of them generating elements on a different budget.

2020-06-28T14:23:22.036800Z

so you could call this tuple-distinct, right?

2020-06-28T14:24:31.037600Z

I did not know that it existed, give me a few minutes to take a look …

2020-06-28T14:25:01.037900Z

it doesn't

2020-06-28T14:25:20.038700Z

I'm just saying that if it did exist, that's what you're describing

2020-06-28T14:25:23.038900Z

yes, I could call it tuple-distinct 🙂

2020-06-28T14:29:51.040300Z

I think it's a useful point because tuple-distinct seems like a simple concept and fits well with the existing distinct generators, compared to some more complicated & idiosyncratic thing that you might have been describing

2020-06-28T14:30:08.040700Z

so I can more easily imagine tuple-distinct being in test.check

2020-06-28T14:32:27.041400Z

I will implement it locally in my library for now, but if it makes its way to test.check, I will be happy to link to it.

👍 1
2020-06-28T14:33:41.043300Z

You could create a jira ticket suggesting it if you want; also you probably figured out that you can implement a crude version of this in userland using tuple and such-that

2020-06-28T14:33:53.043800Z

I'm actually not sure what a fancier version would do exactly

2020-06-28T14:34:05.044300Z

Would it only generate things in order?

2020-06-28T14:35:27.045600Z

as for preferably-such-that , I think that such-as could be augmented with an optional fallback function that returns a generator to use when the tries-left reaches zero.

2020-06-28T14:38:41.045700Z

yes, in order.

2020-06-28T14:44:32.047700Z

in fact, the extension to such-as would be super-useful for tuple-distinct , as the user’s code could have a change to reduce the number of elements if it fails to create distinct elements.

2020-06-28T14:45:15.048100Z

tuple-distinct could be then easily made in userland.

2020-06-28T14:45:30.048300Z

by which I mean, if it fails to get a distinct element for the last generator, will it give up, or do something fancier?

2020-06-28T14:47:11.048700Z

reduce the number of elements? Like return a smaller tuple?

2020-06-28T14:47:19.048900Z

I was thinking about doing something fancier, but that’s better done in userland rather than in test.check: the retry behavior highly depends on what the user wants to do with his data.

2020-06-28T14:47:25.049200Z

that's an odd definition of a tuple

2020-06-28T14:47:28.049400Z

yes

2020-06-28T14:47:46.049800Z

In my use case, I want my data to be a set, in the end.

2020-06-28T14:48:02.050300Z

in my mind a tuple is inflexible w.r.t. number of elements

2020-06-28T14:48:44.050600Z

I agree that this is a coherent thing to ask for; it feels less obviously useful, but it's not too intrusive to the API not my call anymore; in any case I guess I'd say I'm on the fence about it

2020-06-28T14:51:33.052400Z

My usecase is somehow experimental, I will see later if my idea of test.check extension still holds. Thanks again for your help.

👍 1