clojure-spec

About: http://clojure.org/about/spec Guide: http://clojure.org/guides/spec API: https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html
2021-04-09T10:25:36.010200Z

Background: I want to create a spec which can validate values in different branches of a document in a manner that what I get out of spec/explain-data points to the right value. For example:

{:min 0 :max 10 :values [0 1 2 3 11 2]}
I want the spec to be able to say that because 11 is not between the values given in :min and :max, it is in error. I’m trying to implement the Spec protocol to add a wrapper for a spec which works like this
(defmacro value-binding [pred bind]
    `(let [wrapped# (delay (#'spec/specize ~pred))]
       (reify
         spec/Specize
         (specize* [s#] s#)
         (specize* [s# _] s#)

         spec/Spec
         (conform* [spec x#]
           (binding [~bind x#]
             (spec/conform* @wrapped# x#)))
         (unform* [spec y#]
           (binding [~bind y#]
             (spec/unform* @wrapped# y#)))
         (explain* [spec# path# via# in# x#]
           (binding [~bind x#]
             (spec/explain*
              @wrapped#
              path# via# in# x#)))
         (gen* [spec overrides# path# rmap#]
           (spec/gen* @wrapped# overrides# path# rmap#))
         (with-gen* [spec gfn#]
           (spec/with-gen* @wrapped# gfn#))
         (describe* [spec]
           (spec/describe* @wrapped#)))))
The idea is that when the validation passes this spec, the validated value is bound to the given dynamic variable. The wrapped spec can then use the value in that dynamic variable. I got this working with spec/valid? but not with spec/explain-data, which always returns an empty collection. Any ideas why? I made a test spec which prints if the dynamic variable has been bound to a value or not. Inside conform* it is bound, but inside explain* it is not.

2021-04-09T11:01:07.010400Z

Some further info… I can make a minimal spec in unit tests, but in a larger context this breaks down

ikitommi 2021-04-09T11:07:58.010800Z

for that given example:

(s/explain (s/coll-of (s/int-in 0 10)) [0 1 2 3 11 2])
; 11 - failed: (int-in-range? 0 10 %) in: [4]

2021-04-11T03:42:43.016500Z

The most annoying thing is when I make a simple example to see where it breaks down it works fine.

ikitommi 2021-04-11T06:13:35.016700Z

the data-spec is not safe as it mutates the global registry. If you have two documents with different min & max, the last one overrides the first.

ikitommi 2021-04-11T06:13:49.016900Z

How did you make it work @matti.uusitalo?

benoit 2021-04-11T20:10:20.019700Z

Yes, you have to be careful to not use data-spec for multiple documents at once. Sometimes it is an acceptable trade-off.

benoit 2021-04-11T20:35:17.019900Z

I'm not seeing a way around it if you want to use s/keys and the global registry. It does not make sense to me to define a spec on the global keyword ::values that is specific to a given map. The contract that the integers in ::values must be between :min and :max is a property of the map, not the global ::values keyword. If you still want to benefit from the s/explain infrastructure, you can always write a "local spec" like this:

(defn validate-map
  [{:keys [min max values] :as data}]
  (let [s (s/coll-of (s/int-in min max))]
    (when-not (s/valid? s values)
      (throw (ex-info "Invalid map."
                      {:explain (s/explain-data s values)})))))

2021-04-12T04:45:22.020300Z

So I finally figured out what was the problem. I had to wrap the nested spec/explain* call to a doall, because apparently explain returns a lazy sequence which can’t access the bound value if it is returned out of the binding block before realizing the sequence

2021-04-12T04:51:12.020500Z

@ikitommi i have now

(defmacro value-binding [pred bind]
  (let [pf #?(:clj (#'spec/res pred)
              :cljs (res &env pred))]
    `(let [wrapped# (delay (#'spec/specize ~pred ~pf))]
       (reify
         spec/Specize
         (specize* [s#] s#)
         (specize* [s# _] s#)

         spec/Spec
         (conform* [spec x#]
           (binding [~bind x#]
             (spec/conform* @wrapped# x#)))
         (unform* [spec y#]
           (binding [~bind y#]
             (spec/unform* @wrapped# y#)))
         (explain* [spec# path# via# in# x#]
           (binding [~bind x#]
             (doall (spec/explain*
                     @wrapped#
                     path# via# in# x#))))
         (gen* [spec overrides# path# rmap#]
           (spec/gen* @wrapped# overrides# path# rmap#))
         (with-gen* [spec gfn#]
           (spec/with-gen* @wrapped# gfn#))
         (describe* [spec]
           (spec/describe* @wrapped#))))))
and then for example
(testing "Bound values can be referred to in specs"
    (let [test-spec
          (sut/value-binding
           (fn [v]
             (= *test-binding* v))
           *test-binding*)]
      (is (spec/valid? test-spec 123))))
at some point that breaks down without that doall because of the lazyness & bindings

2021-04-12T04:51:51.020700Z

that cljs stuff is there because clojurescript has a different implementation of clojure.spec.alpha

2021-04-09T11:18:29.010900Z

The issue here is that values in the collection should be able to be valid or invalid depending on values in a different part of the document. The example I wrote is a minimal example that tries to convey the idea

2021-04-09T11:22:56.011100Z

I would use my spec like this

(def ^:dynamic *document*)
(spec/def ::min int?)
(spec/def ::max int?)

(spec/def ::value (fn [v]
                      ; compares that v is between min and max
                   ))
(spec/def ::values (spec/coll-of ::values))

(spec/valid?
 (value-binding
  (spec/keys :req-un [::min ::max ::values])
  *document*)
{:min 0 :max 10 :values [0 1 2 3 4 5 11]})

ikitommi 2021-04-09T11:45:42.011300Z

if spec supported maps with inlined entry definitions, that would be easy to do - you could create a new spec with new ::values subspec based on the whole document, but now it would require going through the global registry. I believe spec2, plumatic & malli all make this easy to do. there is also spec-tools, with a working(?) dynamic var for stuff like this, but don’t recommend it.

benoit 2021-04-09T11:47:20.011500Z

I would usually put the constraint on the map itself since it is a constraint between its elements.

benoit 2021-04-09T11:48:10.011700Z

(s/and (s/keys ...)
       (fn [{:keys [min max values]}] (every? #(< min % (inc max)) values)))

benoit 2021-04-09T11:51:57.011900Z

But you want explain-data to return the value ... I see. Good luck with that 🙂

benoit 2021-04-09T12:00:02.012100Z

Could you just redefine the spec for every document?

benoit 2021-04-09T12:00:17.012300Z

(defn data-spec
  [{:keys [min max values]}]
  (s/def ::min int?)
  (s/def ::max int?)
  (s/def ::values (s/coll-of (s/int-in min max)))
  (s/def ::data (s/keys :req-un [::min ::max ::values])))

(let [data {:min 0 :max 10 :values [0 1 2 3 11 2]}]
  (data-spec data)
  (s/explain-data ::data data))

2021-04-09T19:37:07.013600Z

given that all map keys are validated against the spec registry, what is the significance of the :opt argument to s/keys? documentation?

2021-04-09T19:53:39.013700Z

I think it adds the generator stuff for the optional key

2021-04-09T19:53:52.013900Z

(so it will sometimes be generated)

🙌 1
alexmiller 2021-04-09T20:04:35.014200Z

also ends up in the doc output

🙏 1