testing

Testing tools, testing philosophy & methodology...
2015-09-29T06:42:53.000053Z

@nberger: I wonder if it is more constly to create n samples of any generator than create n samples with choose. Or asked differently: does the feature of 'growing' values with sample size add to the cost of any generator?

nberger 2015-09-29T14:56:30.000054Z

Sorry, I don't follow you. You really mean choose? gen/choose docs says "Create a generator that returns long integers in the range lower to upper, inclusive."

nberger 2015-09-29T14:59:19.000055Z

Also, I understand that sample size represents how many values you want to generate from the generator, and "growing values" has more to do with the "size" of those values. I'm not clear on which one we are talking about

nberger 2015-09-29T15:01:31.000056Z

About the size, I see it like it starts small and it grows on each subsequent test, until a failure is found, then it shrinks until a pass is found or there's no more shrinking possible and it reports the result, something like: {:result false, :seed 1439064390947, :failing-size 48, :num-tests 49, :fail [10], :shrunk {:total-nodes-visited 4, :depth 0, :result false, :smallest [10]}}

nberger 2015-09-29T15:03:33.000057Z

Sorry, this example is more interesting: {:result false, :seed 1442512581845, :failing-size 12, :num-tests 13, :fail [8], :shrunk {:total-nodes-visited 9, :depth 2, :result false, :smallest [5]}}. :failing-size is the size when the first failure was found, :fail the value that made the tests to fail, and [:shrunk :smallest] well, the smallest value that was found :simple_smile: