Clojurians Log v2

that's the way to do it now.

Is it by design that schema transformations like mu/assoc don't play well with an unregistered schema?

not by design, could you repro if there is something that doesn’t work?

(def Foo
  [:map
   [:a int?]])

(mu/assoc Foo :b ::bar)

Another issue I managed to stumble on, I defined a dependent schema like https://github.com/metosin/malli#content-dependent-simple-schema It works well but throws when I pass it to reitit routes when the coercion is compiled

Great, now I'm unable to reproduce it 😞

I managed to get myself into this corner like so: • wanted content dependent schema • wanted to parametrize the schema (makes it extensible) • figured out I'd do it by delaying registry building and schema compilation to run-time. • With registry I need ::my-schema • Can't transform anything with ::my-schema at compile time I can create a placeholder registry for it but it seems like it would lead to errors down the line

I'd be happy to adopt a better idea

me too 🙂 spec partially checks the references eagerly, partially lazily (e.g. s/keys), malli is currently eager.

there is an internal escape hatch: :ref doesn’t check the reference if :malli.core/allow-invalid-refs option is truthy.

it is used with local registries, which can have… holes.

(m/validate
  [:schema {:registry {::foo [:ref ::bar]}} ;; incomplete registry
   [:tuple {:registry {::bar int?}}
    ::bar ::foo]]
  [1 2])
; => true

ideas welcome how to make this good.

I don't know if it's good, but perhaps a :delay or :defer schema, which delays registry lookup to validation, with ample warning, care, etc.

Are there built-in functions to throw errors on invalid input? The plans for instrumentation in the above issue are nice, but I'm looking for something simple like spec/assert

Perhaps even wrap it in a function which will always emit warnings when it's called, i.e.

(mu/assoc Foo :b (m/schema (m/defer ::bar)))
STDERR: Deferred Warning *at* - instances of deferred schema must be provided with a registry at run time!

could it be just [:ref {:lazy true} ::bar]?

Ah, laziness has to be explicit

oh, ref’s are lazy already :thinking_face:

yeah, this didn't work 🙂

we need lazier laziness

try (m/-lazy ::bar options)

refs resolve eager by default, but one can create lazy refs with that.

(let [-ref (or (and lazy (-memoize (fn [] (schema (mr/-schema (-registry options) ref) options))))
                      (if-let [s (mr/-schema (-registry options) ref)] (-memoize (fn [] (schema s options))))
                      (when-not allow-invalid-refs
                        (miu/-fail! ::invalid-ref {:type :ref, :ref ref})))

Cool, it worked 🙂

Always good to know some black magic

could make a version of that which doesn’t require the options.

It makes me wonder why [:ref {:lazy true} ::bar] didn't work

it’s a property of the IntoSchema, not Schema instance.

by design, all the IntoSchemas are crated using a function, which can take properties how the IntoSchema works. Easy to extend the system that way and DCE drops all the unneeded schemas.

for example, it’s reletively easy to create custom collection schema types:

(defn -collection-schema [{type :type fpred :pred, fempty :empty, fin :in :or {fin (fn [i _] i)} :as opts}] ...)

:ref has:

(defn -ref-schema
  ([]
   (-ref-schema nil))
  ([{:keys [lazy type-properties] :as opts}] ...))

I have a follow up question from https://clojurians.slack.com/archives/CLDK6MFMK/p1618257034389400 regarding emitting configuration for clj-kondo to pick up (which is an awesome feature by the way!). I have schematised the following code:

(m/=> hash-map-by
  [:=> [:catn [:f [:fn ifn?]] [:coll coll?]] map?])

(defn hash-map-by
  "Returns a map of the items in `coll` keyed by `f`."
  [f coll]
  (into {} (map (juxt f identity)) coll))

{:lint-as #:malli.schema{defn schema.core/defn},
 :linters {:type-mismatch {:namespaces {example.hash-map {hash-map-by {:arities {2 {:args [:fn :coll], :ret :map}}}}}}}}

but, could lift the lazy into a :ref schema property too. so one can say [:ref {:lazy true} ::bar] as data.

if you need that, please write an issue.

currently there is no way to override per schema instance how the clj-kondo works, but would be easy to add. also, having an ifn? schema built-in, it could have the correct clj-kondo type. interested in a PR?

for the latter that is.

for the first, for the second, something like:

Thank you!

I wonder if I should settle for m/-lazy

[:fn {:clj-kondo/type :ifn} ifn?]

If I should consider functions prefixed with - as implementation detail, then I'd say that I shouldn't and open that issue

I'm very interested in implementing this as we'd need it to complete the replacement of clojure.spec with Malli in our codebase, I think.

things starging with - are ok to use: https://github.com/metosin/malli#alpha

I can create a PR for sure. 💯

> might evolve during the alpha That's a risk I'm willing to take. I think if m/-lazy develops in any direction it won't be one which will have friction with what I'm trying to do, on the contrary. Thanks again for the help and guidance, you rock

You can always (assert (thingy-validator dada)) but the error is not so useful

Yeah, I wrote my own for now:

(defn malli-assert
  ([schema value]
   (malli-assert schema value ""))
  ([schema value msg]
   (when-not (malli/validate schema value)
     (throw (ex-info (clojure.string/join "/n"
                       (cons msg
                         (flatten
                           (malli.error/humanize
                             (malli/explain schema value)))))
              {:value value})))))

Make a PR?

It's not ideal because humanize returns nested messages according to the path of the error, which I just flatten into a single string

Ok I'll submit an issue, just wanted to check if it was a design decision not to have an assert

Maybe AssertionError would be more appropriate :thinking_face:

And maybe use *assert* and make it a macro

Spec assert seems to use ex-info and a separate *assert* equivalent var

@ikitommi can I just clarify what you're thinking in terms of a PR, please? I can add #'ifn? to the predicate-schemas and then these tests pass:

(testing "ifn schemas"
    (let [schema (m/schema ifn?)]
      (is (true? (m/validate schema (fn []))))
      (is (true? (m/validate schema (constantly 1))))
      (is (true? (m/validate schema :keyword)))
      (is (true? (m/validate schema (reify clojure.lang.IFn
                                      (invoke [_] "Invoked!")))))))

yes, but also mappings for transformers, generators, json-schema, humanized errors and clj-kondo.

I'll take a look at implementing the full feature set for the ifn? domain. 👍

I'll not implement proper generation of interesting functions. Don't want to put us all out of a job. 😉

Oh, I think I see what you mean. You want ifn? to be parameterised so you can schematize the args and return values…?

So in the schema generator you can do something more than this:

(defmethod -schema-generator 'ifn? [_ _] (gen/return ::ifn))

I'd need to generate a function that returns valid data given valid arguments.

really, why?

there is already :=> and :function which have proper input & output generators

Because I thought that was what you wanted. 🙂

I think I misunderstood.

no, just the simple thing, lke fn? but bit different 🙂

If ifn? can remain a simple predicate, I should be able to have a first pass at a PR before the lunchtime walk. 🙂

👍

Is it possible for schemas to self-reference? Naively trying to make a recursive schema causes a stack overflow:

(malli/schema
  ::tree
  {:registry (merge (malli/default-schemas)
               {::node :int
                ::tree [:or
                        ::node
                        [:tuple ::tree ::tree]]})})
 

@qythium use the :ref, luke:

(mg/generate
  (m/schema
    ::tree
    {:registry (merge (m/default-schemas)
                      {::node :int
                       ::tree [:or
                               ::node
                               [:tuple [:ref ::tree] [:ref ::tree]]]})})
  {:seed 3})
; => [[-26764 [[1 73136] [13307055 -1]]] [-381 [[-3 587742] -243724556]]]

awesome, thanks!

So schema references only in :ref and :map have to be qualified keywords? Strings appear to work too, but not plain keywords

Also it seems that the ::foo shorthand syntax doesn't work on the http://malli.io playground

Seems like it, I found malli.core/-reference? in the source code which checks for qualified-keyword? or string?, but this doesn't seem to be documented

yes, references should be qualified keywords or strings. http://malli.io … must be a sci-thing.

Ok, filed an issue on the http://malli.io github

Trying to use unqualified keys as references tripped me up quite a bit at the beginning, since this requirement isn't documented and the error message just says :malli.core/invalid-schema

doc enhancement PRs are most welcome.

(the error keyword could be better here)

I'll just file issues for now if that's ok - still in the early stages of experimenting with the library and not confident of writing docs

Another strange thing I encountered:

;; This works as a schema
[:map {:registry {::foo :int}}
 ::foo]

;; so does wrapping the keyword in a vector
[:map {:registry {::foo :int}}
 [::foo]]
;; to pass it options
[:map {:registry {::foo :int}}
 [::foo {:optional true}]]


;; These are ok too
[:map {:registry {::foo [:tuple :int :int]}}
 ::foo]
[:map {:registry {::foo [:tuple :int :int]}}
 [::foo {:optional true}]]


;; But not this??
(malli/schema
  [:map {:registry {::foo [:tuple :int :int]}}
   [::foo]])
;; => Execution error (ExceptionInfo) at malli.impl.util/-fail! (util.cljc:16).
;;    :malli.core/invalid-schema {:schema [:tuple :int :int]}