Clojurians Log v2
malli
@ikitommi could it be related to into transformer using satisfies? It's notoriously slow and GC intensive
it could, easy to see from flamegraphs.
How do I provide sci options in malli decode? Trying to use java class with no luck.
(m/decode [:int {:decode/string '(fn [_] (.toEpochMilli (java.time.Instant/now)))}]
"whatever"
{:classes {'java.time.Instant java.time.Instant}}
mt/string-transformer)I can already guess from my experience with recursive calls to satisfies? , you'll see 95% of CPU is wasted on it
no atm, but could be. Thing is that you can present things with malli that are impossible to present as specs, so it would not be complete like malli->JSON Schema , which is lossy. Still, I think 95% would work ok. There is malli-instrument library for doing the same with malli, if that’s ok for your use case.
looking at the code, it’s :malli.core/sci-options key in options, see https://github.com/metosin/malli/blob/master/src/malli/core.cljc#L1792-L1813
@ben.sless merged the loop unrolling perf PR, great for common cases and as it’s just clj, doesn’t make the cljs bundle size any bigger, big thanks!
;; 164ns -> 36ns
(let [valid? (m/validator [:and [:> 0] [:> 1] [:> 2] [:> 3] [:> 4]])]
(cc/quick-bench (valid? 5)))merged the other perf PR too:
;; 150ns -> 39ns
(let [valid? (m/validator [:map [:a :any] [:b :any] [:c :any] [:d :any] [:e :any]])
value {:a 1, :b 2, :c 3, :d 4, :e 5}]
(cc/with-progress-reporting
(cc/quick-bench (valid? value))))
@ikitommi I've created https://github.com/metosin/malli/issues/472 with a test case which demonstrates the issue.
@richard.hundt I profiled you example, it looks like CPU mostly goes towards parsing the schema. If you wrap every def-ed schema with a schema constructor it seems to be way more efficient, would you mind trying?
D'you mean just wrapping it in a function which returns the structure? (I'm not sure what you mean by "schema constructor")
like so;
(def A1
(m/schema
[:map
{:encode/test {:leave encode} :decode/test decode}
[:type [:= "A1"]]
[:value float?]
]))thanks, I'll try
m/schema parses the children once
your implementation has no way to cache these results
It's a bother to do from scratch, you can take this
thanks slack, very helpful
thank you
https://gist.github.com/bsless/28a20f7e8f21f01fd93a788cf65444d4
there, more convenient
yeah, that's pretty much instant now, I'll try it on our production code
👍
Perfect. Thanks Ben! What shall I do with the github issue? Do I close it, or are you planning on making changes to parsing?
I closed the issue, no plans on making the parser faster or caching at the moment. Thanks for @ben.sless for digging into this 🙇
I think it is worth to leave a comment on the issue and perhaps a note in the readme regarding references to uncompiled schemas
And you're welcome 🙂
Turns out I was completely wrong regarding the transformers and satisfies?
I see you left a comment with a link to the gist, so that would leave only the readme
if you have idea what and where to write this, please do 🙂
1👍@ben.sless, great work. Saw the thread on Clojureverse about compiler optimizations. Would be great if things got automatically or magically faster. While waiting, writing performant code inside the libraries (on the hot perf path) is just being smart.
11🙏Thank you, I appreciate it. Blame it all on the talk you gave on ClojureTRE 2019
Nice 😁
If -into-transformer was turned into a protocol, how would you make it work with cljs? fn? there is very different from in clj and won't be covered by a simple extend.
Possible to do in cljs what was done for Schemas in clj?
Makes sense. I wouldn't mind losing some of the malli features when instrumented as a spec.
Some of the libraries (like integrant's pre-init-spec) support only spec.
Although in my case, I use spec mostly for instrumentation of some bits. I was asking for such a conversion, as it sounds like the non-invasive way of trying malli in an existing project.
Anyway, I will take a look at malli-instrument. Thanks.
tried this
(m/decode [:int {:decode/string '(fn [_] (.toEpochMilli (java.time.Instant/now)))}]
"whatever"
(merge
(m/-default-sci-options)
{:malli.core/sci-options {:classes {'java.time.Instant java.time.Instant}}})
mt/string-transformer)but can’t see it in https://github.com/metosin/malli/blob/eea10aa8c668a16f88d9fc66fce0616a7f9137b4/src/malli/sci.cljc#L10