Clojurians Log v2

Hi. I’ve a question about property based testing and test.check and their suitability to the following integration validation. If I wanted to check that function

(first-available-timeslot {:attendees 3 :starts 2017-02-03 :ends 2017-02-03 :duration 60})

I assume the function can fail if the constraints aren't satisfiable

Are you planning on just testing with that piece of example data you showed, or generating that input as well?

Correct, and generating input also.

I think the first two properties are good

The third has slightly more of the "duplicating the business logic" flavor

But might be easy enough as to be worth it

okay. I’ll have a bash. My concern was that integration tests are expensive, and having many tests with large datasets even more so, although they obviously offer far more confidence.

Expensive meaning dev time? Waiting for tests?

Waiting for tests. Further, the query might operate against a db, or possibly google calendars api so there’s timeouts and error conditions complicating things.

I would try to mock out things you don't control

Unreliable tests are even more annoying in PBT

Shrinking can be almost meaningless

Yeah, that’s what I was thinking. I actually wonder whether syncing a copy of the calendar into a clojure owned data structure before querying might allow better reasonability. It’s only a small dataset.

Or generate the calendar

A property I just thought of that is a corollary of the "earliest time slot" requirement:

"Given two reservation requests where the first is strictly easier to satisfy than the second, processing them in order should never result in the second being scheduled before the first

Meaning they should have the same date range, and the second should involve at least as many people and minutes

Just parsing that...

Got it, yes. It’s these property identifications that make it difficult, and require a lot of experience.

Yep

I'm about to sacrifice my phone for the sake of providing entertainment to my offspring, so that may be all the pondering I can do for now

@gfredericks thanks for the help. I will explore this further tomorrow and enjoy the rest of your evening!

is there any way to instrument a set of generators for a test to track performance?

I've got what, to me, seems like a relatively simple generation set that takes 20+ seconds per iteration for some reason

there's no recursion, and the only such-that checks that a basic keyword produces something not part of a previously-generated set

hm

that definitely will slow things down

depending on what you are doing, you could generate a set, then use fmap to remove an element and return a tuple of [removed-element set] to get a similar effect without thrashing such-that

that could work

as it is I'm using a modified version of test.chuck's testing that will output things on each iteration

I'm modifying that to display gen vs. test times

and then doing simple one-off tests with each of my generators

I doubt thrashing such-that would have such severe consequences unless you manually raised the iteration limit

It gives up after 10 tries

yeah, it's not that so far as I can tell

heh, as usual, my takeaway is "don't use generators for things you don't need to test the properties of"

replacing (gen/map gen/keyword some-other-gen {:min-elements 1 :max-elements 20}) with

🙁

reduced the time spent running generators to about an eighth of the origional

That's unfortunate

yeah

But sounds plausible

the keys don't really have any special properties here, it's mostly, I'm testing the map of some-other-gen here and how that effects the code I'm testing

but it literally went from

...this being the output of my runner

I have another project where I did something similar

it's a parser of sorts and I needed a bunch of word-looking text

I started with (gen/not-empty (gen/string-alphanumeric)) and ended up switching to (gen/elements a-set-of-words-from-lorem-text) because it was a similar magnitude faster

so I've started having to ask on these things, ok, what am I actually testing the properties of here?

and what can I get away with not generating