Clojurians Log v2

@ikitommi for what it's worth, i still got a huge performance increase by actually caching the validators as well.

(crit/with-progress-reporting
  (crit/quick-bench (m/validate schema value)))
;; => Execution time mean : 297.880813 ms

(def schema' (m/schema schema))
(crit/with-progress-reporting
  (crit/quick-bench (m/validate schema' value)))
;; => Execution time mean : 533.885193 µs

(def validator (m/validator schema))
(crit/with-progress-reporting
  (crit/quick-bench (validator value)))
;; => Execution time mean : 1.830348 µs

i suspect in your specific benchmark, the schema is fairly simple so then a larger share of the benchmark is actually about performing the validation

@lmergen there was a cljs-issue, just merged the cached satisfies. could you retry with the latest master?

sure!

1 minute

there is still a lot of room for improvement for maps (`-parse-entries` is really slow) and for handling property-based registries. I would guess can make schema creation 2-5 times faster. But then again, after malli is used to validate schema properties & children, it will slow things down again.

(crit/with-progress-reporting
  (crit/quick-bench (m/validate schema value)))
;; before: => Execution time mean : 297.880813 ms
;; after:  => Execution time mean : 12.194964 ms

(def schema' (m/schema schema))
(crit/with-progress-reporting
  (crit/quick-bench (m/validate schema' value)))
;; before: => Execution time mean : 533.885193 µs
;; after:  => Execution time mean : 517.890217 µs


(def validator (m/validator schema))
(crit/with-progress-reporting
  (crit/quick-bench (validator value)))
;; before: => Execution time mean : 1.830348 µs
;; after:  => Execution time mean : 1.952607 µs

i'm caching the explainers in my own defn macro, but it requires quite a bit of macro magic to make this work, so i was looking for a more generic way to make this happen -- possibly some kind of registry

what should be in the registry? validator + explainer + generator + decoder(s) + encoder(s) + …?

if possible, i'd say all of them yes

in reitit, there is a Coercion protocol to cache things relevant there: https://github.com/metosin/reitit/blob/master/modules/reitit-malli/src/reitit/coercion/malli.cljc

right, so then you lazily cache things

which would be the best middle-ground

one would be to add a wrapper Schema impl, that is returned from registry instead of the real one. And that impl would have a cache -> first call to -validate would store the validator.

oh i see

could be just an option to the registry to return caching proxys instead of normal ones…

yes

this would be very effective

i'll experiment with this approach

… actually, just a new option key that m/schema uses would do fine (to wrap the returned thing if the option is present)

so then you cache it inside the actual schema, rather than a wrapper around it ?

I would wrap it outside, e.g. the return value wrapped

ah, right -- and the option to m/schema would then tell it whether to return the wrapped schema or the "regular" schema

yes. Or there could be a memoized-schema etc. as a separate fn? (-> :string m/schema m/memoized)

yes

well that's a detail

let me experiment with creating that memoized / cached schema in the first place

(crit/with-progress-reporting
  (crit/quick-bench (m/validate schema value)))
;; => Execution time mean : 11.782073 ms

(def schema' (memoized-schema (m/schema schema)))
(crit/with-progress-reporting
  (crit/quick-bench (m/validate schema' value)))
;; => Execution time mean : 2.095245 µs

exactly which schema am i supposed to wrap here -- it's just the regular malli.core/Schema, right ? the into-schema is meant more for building a hierarchy of parent/child schemas ?

yes Schema. IntoSchema is the factory-protocol for creating a Schema out of the Schema AST, each Schema is responsible for it’s own props & children.

right

awesome

i'll send a PR once i have all the functions working