Clojurians Log v2

How can I specify the number of tests run when using clojure.spec.test.alpha/check ? By default its running 1000 tests for a check against 1 spec (the playing cards example from https://clojure.org/guides/spec#_a_game_of_cards ) and takes around 80 seconds to complete. The docs mention :num-tests within clojure.spec.test.check/opts but either I have the syntax wrong or missing something

;; runs 1000 tests
(spec-test/check `deal-cards
                 {:num-tests 1})

;; java.lang.RuntimeException
;; Invalid token: ::clojure.spec-test-check/opts
(spec-test/check `deal-cards
                 {::clojure.spec-test-check/opts {:num-tests 1}})

java.io.FileNotFoundException
   Could not locate clojure/spec/test/check__init.class,
   clojure/spec/test/check.clj or clojure/spec/test/check.cljc on classpath.

@jr0cket There's no such namespace: https://github.com/practicalli/spec-generative-testing/blob/prime/src/practicalli/spec_generative_testing.clj#L6

The option should be a qualified keyword -- but that doesn't mean a namespace exists.

:clojure.spec.test.check/opts

If you want to use ::stc you can introduce an alias:

user=> (alias 'stc (create-ns 'clojure.spec.test.check))
nil
user=> ::stc/opts
:clojure.spec.test.check/opts
user=>

Your ::clojure.spec-test-check/opts is going to fail because :: will try to auto-resolve clojure.spec-test-check which is not an alias.

Spec2 seems so cool. Increased programmability is such a good advantage. I have been using custom macros to generate spec, and it was a bit odd.

Sorry if this question has an easy answer discoverable that I missed, didn't do a deep dive. I see that https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#select is available in alpha2 but not in alpha. Obviously lot of people use spec already, but which version?

"spec1" (spec.alpha)

it's not really clear what will/wont change in spec2 and there are also a few bugs

Thanks! is there something similar to spec/select already existing, perhaps written by someone else, published under a different name?

It would never work as I got the namespace wrong, using - instead of . Now I have the namespace correct in the fully qualified name, surprisingly it works

(spec-test/check 
  `deal-cards
  {:clojure.spec.test.check/opts {:num-tests 1}})

whats the difference between those two?

(s/def ::name (s/with-gen string? #(gen/return "Margaret")))
(s/def ::name (s/spec string? :gen #(gen/return "Margaret")))

effectively, not much

any one is preferred?

not really. I usually do the first one.

thanks for the info 🙂

Running with 10 tests is pretty instantaneous and 100 tests is only a couple of seconds. Much better. Now to start figuring out how to run tests as part of the development workflow...

@ashnur Spec 2 will eventually become the official clojure.spec. It's just not ready for use yet as it is still being actively designed and changed. Spec 1 will remain available for everyone already using it. I don't think anyone has tried to copy Spec 2 -- because it is not yet complete (and why would anyone try to recreate an official part of Clojure when Rich himself hasn't fully figured out parts of the design?).

My point was that, even if you got the namespace right, ::some-namespace/key isn't going to work because some-namespace is not an alias. It needed to be a single : if you're using the whole name (and you get it right 🙂 ).

I am just curious if I want to use select or something that does similar stuff to select, what are my best options currently.

@ashnur If you're just building toy stuff or learning/experimenting, you could use Spec 2. It's just not ready for production use.

We were tracking it at work, with a branch of our (95k lines) codebase, and I really like the changes in Spec 2 -- above and beyond the schema/`select` stuff -- but there's been a lot of churn in Spec 2 and Alex has said that Rich will likely overhaul s/fdef completely before it is released (and it may drop s/keys completely as well), so we stopped tracking it months ago.

We're just going to wait for it to be "fully baked" at this point.

Once Alex signifies that it is stable and just needs testing to help iron out the bugs, we'll pick it up again.

I will signify that by making a release :)

so, if I understand that correctly, there is nothing else that targets the same problem domain to be used in production in the interim?

clojure.spec.alpha targets that domain and can be used in production. Spec 2 is the "next generation" of that and will be the preferred solution when it becomes ready.

Spec 2 is definitely "better" than Spec 1 -- because it's designed to incorporate lessons learned from the first version. But Spec 1 definitely has value in production.

I like what s/select does and would like to use something like it in production, even if it's not necessarily exactly the kind of s/select that is planned by Rich, since as I understand it, there are still months, maybe years until that version will be ready.

Starting to find my way and looking through the next-jdbc project was a good example of using spec. Lots to still learn about spec and testing though 😁

And just when you get it all down pat, Spec 2 will come along and change everything you know 🙂

There will still be a lot of spec 1 examples out there to work with, so good to have spec1 skills for my semi-retirement plan over the next 18 years 😁

god I hope it's not years :)

Based on our experience with Spec 2, I suspect our own codebase will contain lots of Spec 1 code for many years...

(Spec 2 is much better, but also very different, so migration would be painful)

how can not be expressed in spec? to maximize built-in generators reuse and composability (really just ability to wrap any spec in "it" without looking at spec/form I am wrapping)

(before you look at me funny, I am writing a translator from json-schema to clojure-spec, particularly https://json-schema.org/understanding-json-schema/reference/combining.html#not)

anything better than this?

(do
  (s/def ::foo string?)
  (s/def ::bar (s/with-gen
                 (complement (partial s/valid? ::foo)) 
                 #(s/gen any?)))
  (s/exercise ::bar))

not really

not is weird and I would generally avoid doing it :)

at this time, I am trying to generate spec as close to schema as possible, as code you than paste into file, and then might chose to change

Alex, is there an (out the box) way to conform unqualified map and get qualified conformed map back?

(do
  (s/def :my/foo string?)
  (s/def ::map (s/keys :req-un [:my/foo]))
  (s/magic-conform ::map {:foo "x"})  #_=> {:my/foo "x"})

I'm not Alex @misha but I can't think of any easy way to do that. You'd probably have to derive the (qualified) keys from the s/form of the Spec, and then zipmap with a version of those keys that had been mapped to unqualified keys, and then use clojure.set/rename-keys on your validated data.

(but that won't work with nested data structures/specs or anything more complex than just s/keys)

You could unform

I guess you still wouldn’t get unqual

So I’ll go with no :)

I thought about just including both :req and :req-un sets of keys, but it does not solve "I have a map from example page, show me the specs it uses", and screws up the generators, which you probably want to generate either entirely qualified or entirely unqualified deep tree.

If we were starting again from scratch with Spec available, and using next.jdbc instead of clojure.java.jdbc, I think we would only have unqualified keys at the boundary of our system: either as API input or user input (forms, URLs), and at outgoing boundaries for JSON-based systems. So our use of :req-un/`:opt-un` would be a lot smaller, and we'd explicitly transform validated input into a domain model that always used qualified keys. Interacting with JDBC via next.jdbc means you can use qualified keys going out to the DB and you would get qualified keys coming in from the DB as well, automatically.

I'm playing around with custom generators, since there's a type which it seems like spec is having a hard time generating for some tests.

(s/def ::value pos-int?)
(s/def ::name keyword?)
(s/def ::symbol (s/keys :req [::value ::name]))
(s/def ::symbols (s/coll-of ::symbol :kind set?))
(s/def ::rows pos-int?)
(s/def ::columns (s/and pos-int?
                        #(>= % 3)))
(def machine-gen
  (gen/let [machine (gen/fmap
                     (fn [[cols rows]]
                       {::rows rows ::columns cols})
                     (gen/tuple (gen/fmap (partial + 3) gen/nat)
                                (gen/fmap inc gen/nat)))
            symbols (gen/vector-distinct (s/gen ::symbol)
                                         {:min-elements (inc (::rows machine))})]
    (assoc machine ::symbols symbols)))
(s/def ::machine (s/with-gen
                   (s/and (s/keys :req [::symbols ::rows ::columns])
                          #(> (count (::symbols %)) (::rows %)))
                   (constantly machine-gen)))

Here's a simpler version of the generator:

(def machine-gen
  (gen/let [cols (gen/fmap (partial + 3) gen/nat)
            rows (gen/fmap inc gen/nat)
            machine (gen/return {::rows rows ::columns cols})
            symbols (gen/vector-distinct (s/gen ::symbol)
                                         {:min-elements (::rows machine)})]
    (assoc machine ::symbols symbols)))

Sean, my initial motivation is exploration, specifically of https://vega.github.io/ So I want to generate spec from schema (which is 9999km long), then take an example json, and with magic-qualify-conform see, which spec is that, and then navigate through keywords and specs in my IDE, instead trying to find things in huge json schema: https://vega.github.io/schema/vega-lite/v4.json or https://vega.github.io/schema/vega/v5.json This, and, the usual spec goods: exercise, etc.

so it seems I'd have to come up with "qualiform" too.

Yeah, I can definitely see the utility of this and it would be nice as an option in s/conform.

I think it's interesting that Spec 2 takes a different approach, where you can specify unqualified keys inline in a hash map spec or else qualified keys in a schema, to be select'ed

Search is being very unhelpful for this problem, seems like few people run into it. @seancorfield would you happen to know of anything I could do to debug this issue or to alter the generator so that it'll work?

@suskeyhose what is gen/ in your code above? I gather it's not clojure.spec.gen.alpha...?

It's clojure.test.check.generators

My understanding was that clojure.spec.gen.alpha was just a namespace that re-exposed some of the test.check vars.

Which seems to be true looking at the source.

Hahaha... OK, it took me a while... What is ::symbols? What does it generate?

And then in machine-gen, what type is symbols?

symbols is just a set of ::symbol, which are just maps with ::name and ::value

::symbols is a set. symbols in machine-gen is a vector.

Oh boy, of course that's it

That was a nice Friday afternoon debugging diversion -- thank you! 🙂

Thanks for helping me out! I feel so dumb when I just get my types misaligned like that 🙃

No worries. I couldn't see it either. And I was simplifying the ::machine spec trying to figure out what the problem was... I was quite bewildered by it!