Clojurians Log v2

Is there an api in test.check to get the collection of inputs and outputs used when exercising a test? For example, when running

(defspec first-element-is-min-after-sorting ;; the name of the test
         100 ;; the number of iterations for test.check to test
         (prop/for-all [v (gen/not-empty (gen/vector gen/int))]
           (= (apply min v)
              (first (sort v)))))

({:input [1] :output [1]},
 {:input [2 1] :output [1 2]}
 ...

you can recreate it using the random seed that quickcheck gives you

@ghadi I can get the data literals using the seed? I know I can re-run the test again, i’m curious about the data itself though

right, I'm not sure the exact process

the seed is there so that you can get reproducibility

@lanejo01 may I ask what do you plan to do with that data? I'm truly curious, mostly because a while ago I wanted to add a way to get a glimpse on what's the distribution of the generated values

args is the generated data

The return value isn't given, you can just call your function yourself once you have the args

@gfredericks can you do something like sample-seq while providing the seed?

I had another instance of a recurring dream last night. The word cloud looks like this: clojure.spec generative testing + consumer driven contracts (like http://pact.io) except for libraries, not web apis + spec-ish type-system-ish code inspection/analysis (e.g. (-> (makes-random-int) expects-only-odds) would be analyzed and the system would show there is a problem) + service for publishing specs to a public repository. The big thing here is consumer driven contracts for function apis. A library author could know the expectations of its consumers based on properties they provide, and then know if they make a breaking change or not.

There are a LOT of conflated ideas here, please dont take this as a proper description of all goals.

I suppose the ideas are 1. consumer driven contracts for public library functions 2. analysis of clojure programs using spec to determine areas for potential bugs 3. maven for specs.

The reason I’m asking is about inputs and outputs is to create a system like haskell’s hoogle (https://www.haskell.org/hoogle/) with spec.

Thanks @gfredericks!

@lanejo01 The way something like that could work is if consumers published specs that the upstream author could consume, and those specs would have to be more constrained versions of the specs the author already publishes. The upstream author could swap those more specialized specs in their own tests. That reminds me of "specialization by constraint" in Date and Darwen's "Databases, Types, and the Relational Model". I'm not well-versed in type-theory in general, so it might have other names, but that's where I came across it.

The idea there is that any proper subtype is only a subtype if it satisfies the definition of its parent type plus some additional specialization constraint.

That’s exactly how http://pact.io does it, except with json and regex, not something more expressive like spec + edn.

How would "maven for specs" be different from maven (or similar tool) in general? Specs are code and just like when you use a library, you still need to know the names of the functions (or specs) once you have it.

It could exist, but might not be immediately useful since you couldn't easily pick out the failing args. Do you have another use case?

It was a bit of a mischaracterization of what I’m going after ( which, is admittedly, a moving target). It’s more about a place for consumer “contracts” to be placed. In addition to that, the idea of a central place that I can look up what functions take this input and produce this output could be really exciting for a variety of reasons. Not so much maven for specs. Sorry for the confusion.