Clojurians Log v2
speculative
It’s too long since I’ve been in the speculative code base, so maybe it’s already done, but I was thinking that now that we have a bunch of specs for core, how about another project which uses those specs to property-base testing core?
At least in the beginning of speculative, we write tests to see if the specs we wrote were correct, now I imagine we should be able to use those specs to generate random input and see how the spec’ed functions hold up?
I’m already doing that for all of the functions in speculative, except for some in the set namespace which I’m working on now
This has led to some changes in the specs. Sometimes I discovered that some examples didn’t work. One example is:
;; TODO: fix, the first fails but the second works. maybe deprecate the use of java maps?
#_(set/join [{:a 1} {:a 1}] [(java.util.HashMap. {:b 2})])
#_(set/join [{:a 1} {:a 1}] [(java.util.HashMap. {:b 2}) (java.util.HashMap. {:b 2})])
One place where I had not used generative testing was around the regex functions. It turned out the ret spec was not sufficient, which I could have found using generative testing. So I added those now too, by generating random regexes and matching strings.
Nice!
Curious, though. For eg assoc, a property of that operation would probably be that the values that are being assoc’ed are available in the returned map under the keys that are passed. Do you have those kinds of tests?
You mean in the :fn part of the fdef, yeah, that makes sense.
But it’s also kind of closed, isn’t it? I mean if I write a spec for my function foo, and I omit the :fn part, and you come along and figure out a property that you want to hold for foo, you have no way of augmenting the spec with this information?
The registry is open - you could always supply your own spec that augments
Ie tests that do more than just checking the input and ret vals.
For group-by there is such a test. But in general no, the tests are focused on verifying the validity of the specs, not as a test suite for Clojure core.
Cool. I’d say that’s another library which would depend on speculative.
https://github.com/borkdude/speculative/issues/70#issuecomment-465922518
ALL functions are now generatively checked 🙂
Next up: decouple test.check from the specs again, since I don’t want people to require this lib when only loading/instrumenting the specs…
Nice boring task while on a train ride to a Clojure conference in Berlin
done
These are exactly the kind of thing rich wants to be able to say directly in function specs
Using generative testing I think we found a bug in CLJS set/union:
$ clj -A:test -m cljs.main -re node
ClojureScript 1.10.520
cljs.user=> (require '[clojure.set :as set])
nil
cljs.user=> (set/union #{} nil)
#{} ;; <- empty set, correct
cljs.user=> (set/union #{} nil nil)
() ;;<- not a set!
cljs.user=> (set/union #{} nil nil nil)
() ;; <- not a set!