Clojurians Log v2

Hoe does one combine validate and transform? E.g. this doesn't crash:

(prn (m/decode int? :foo mt/string-transformer))

Should I first call m/valid, if not valid, then m/explain and else m/decode, effectively traversing the structure twice?

If you need decoding, the flow should be: 1. decode 2. validate 3. explain on error

there is 2-3 walks in the error case.

for happy case, 1-2

having a seoarate optimized validate makes the happy case fast

the docs could have examples on this...

the m/decoder doesn't have to walk the structure, it returns an function to transform just the parts that need to be decoded. In case there is nothing to do, it returns identity

@ikitommi The concrete example I was going to try:

$ cat deps.edn
{:deps {metosin/malli {:git/url "<https://github.com/metosin/malli>" :sha "2bd749f7148e28a379f1e628a32188e7f6cf0bc4"}
        borkdude/edamame {:git/url "<https://github.com/borkdude/edamame>" :sha "64c7eb43950eb500ba7429dded48257cd15355ae"}}}

$ cat src/edamalli/core.clj
(ns edamalli.core
  (:require [edamame.core :as e]
            [malli.core :as m]
            [malli.transform  :as mt]))

(defrecord WrappedNum [obj loc])

(defn postprocess [{:keys [:obj :loc]}]
  (if (number? obj) (->WrappedNum obj loc) obj))

(defn -main [& args]
  (prn (e/parse-string "[:foo 42]" {:postprocess postprocess}))
  ;; TODO:
  ;; - validate that the WrappedNum contains value < 42
  ;; - then transform it to only that number
  ;; - else raise error, printing the location metadata of that number
  )

$ clojure -m edamalli.core
[:foo #edamalli.core.WrappedNum{:obj 42, :loc {:row 1, :col 7, :end-row 1, :end-col 9}}]

@ikitommi I am parsing a schema into a malli-schema. The original might contain recursive references to other "entities" in the schema. I do not know this up front. Are there any trade-offs in putting all my potential recursive entity references in a [:ref ], even if they turn out not to be?

@borkdude would [:foo 42] be transformed to [:foo 42] , as would [:foo :bar #{42}] to itself and {:a 41} would fail on the fact that there was a number that was not 42?

happy to help, sample inputs -> outputs would help.

@ikitommi No, [:foo (WrappedNum. x y)] would be transformed to [:foo x] only if x < 42, else error with explain using y

could that validation happen already in the :postprocess?

I just want to feed this data to malli and not intertwine parsing data from text to sexprs with malli validation

ok. can the input be anything, e.g [:foo {:bar (WrappedNum. x y)}]?

[:foo y] could also be {:foo y} if that's easier for you

No, it's more like person: {:name "foo" :age 42}, let's say

so attribute+value

so unwrapping would just be (:obj wrapped), that would be the transform step

The use case for this is: normally edamame doesn't let you have location metadata for numbers and strings, but using a wrapped value you can have that

so I want to use malli like normally, but just use the location metadata in the wrapped value for reporting errors and discard it if the value is valid

I believe in spec you would write a conformer for this

yes, this is kinda tricky with current malli, as there is not yet a parsing api, like conform.

also, there is no custom overridable validator, so one needs to describe the given data structure (here, a tuple with keyword and a record).

ok, so one would write a schema using the records and if everything's ok, then postwalk yourself, unwrapping them?

but, something like this:

;; schemas
(def <42 [:and int? [:< 42]])
(def Schema [:tuple keyword? [:map {:encode/success :obj, :encode/failure :loc} [:obj <42]]])

;; validator and encoders for both success & failure
(def valid? (m/validator Schema))
(def success (m/encoder Schema (mt/transformer {:name :success})))
(def failure (m/encoder Schema (mt/transformer {:name :failure})))

;; in action
(defn parse-validate-and-transform [s]
  (let [x (e/parse-string s {:postprocess postprocess})]
    (if (valid? x) (success x) (failure x))))

(parse-validate-and-transform "[:foo 41]")
; => [:foo 41]

(parse-validate-and-transform "[:foo 42]")
; => [:foo {:row 1, :col 7, :end-row 1, :end-col 9}]

yes, postwalk would do. or a recursive schema definition. if the wrapped records can be anywhere

let me try your snippet

$ clojure -m edamalli.core
Execution error (ExceptionInfo) at malli.core/-fail! (core.cljc:76).
:sci-not-available {:code ":obj"}

oh, should not.

(def Schema [:tuple keyword? [:map {:encode/success (fn [x] (:obj x)), :encode/failure (fn [x] (:loc x)) [:obj <42]}]])

pushed e19872273c3660fbc482dcff4c2d8439dbcbb2a6, which should allow naked keywords as functions.

(def Schema [:tuple keyword? [:map {:encode/success :obj, :encode/failure :loc} [:obj <42]]])

I'm still getting the sci-not-available error

When I do include sci, I get:

[:foo {:row 1, :col 7, :end-row 1, :end-col 9}]

I guess I should throw my own error in :encode/failure?

yes, that’s one place to do it. will check why sci is needed. just a sec.

(defn parse-validate-and-transform [s]
  (let [x (e/parse-string s {:postprocess postprocess})]
    (if (valid? x)
      (prn "SUCCESS" (success x))
      (prn "ERROR" (failure x)))))

$ clojure -m edamalli.core
"ERROR" [:foo {:row 1, :col 7, :end-row 1, :end-col 9}]

haha, when I do this:

:encode/failure {:message "should be lower than 42"}

$ clojure -m edamalli.core
Execution error (ExceptionInfo) at sci.impl.utils/throw-error-with-location (utils.cljc:54).
Could not resolve symbol: should [at line 1, column 1]

I have no idea what I'm doing, since I don't know these APIs well. I'll take a look after work again some time

8e067b3d004d1692cbfc695bc73d7e032ecb6e7f

the code used sci for all non fn?s, no to all non ifn?s. need to add tests.

@ikitommi I now have this:

(def Schema [:tuple keyword? [:map {:encode/success :obj, :encode/failure {:message "should be lower than 42"}} [:obj <42]]])

"eval!" "should be lower than 42"
Execution error (ExceptionInfo) at malli.core/-fail! (core.cljc:76).
:sci-not-available {:code "should be lower than 42"}

I might be doing something wrong, but it seems there's a debug println in there?

:picard-facepalm: my bad. but this kinda works (but is bad, should be better when we have the parsing api):

➜  ~ clojure -Sdeps '{:deps {metosin/malli {:sha "230b1767729aad3e02568f1320855e2b45d2d9b5", :git/url "<https://github.com/metosin/malli>"}, borkdude/edamame {:sha "64c7eb43950eb500ba7429dded48257cd15355ae", :git/url "<https://github.com/borkdude/edamame>"}}}'
Checking out: <https://github.com/metosin/malli> at 230b1767729aad3e02568f1320855e2b45d2d9b5

Clojure 1.10.1
user=&gt; (ns edamalli.core
  (:require [edamame.core :as e]
            [malli.core :as m]
            [malli.transform :as mt]))

(defrecord WrappedNum [obj loc])

(defn postprocess [{:keys [:obj :loc]}]
  (if (number? obj) (-&gt;WrappedNum obj loc) obj))

(defn fail! [{:keys [:obj :loc]}]
  (throw (ex-info (str "so bad " obj "/" loc) {})))

(def &lt;42 [:and int? [:&lt; 42]])
(def Schema [:tuple keyword? [:map {:encode/success :obj,
                                    :encode/failure fail!} [:obj &lt;42]]])

(def valid? (m/validator Schema))
(def success (m/encoder Schema (mt/transformer {:name :success})))
(def failure (m/encoder Schema (mt/transformer {:name :failure})))

(defn parse-validate-and-transform [s]
  (let [x (e/parse-string s {:postprocess postprocess})]
    (if (valid? x) (success x) (failure x))))

edamalli.core=&gt; (parse-validate-and-transform "[:foo 41]")
[:foo 41]

edamalli.core=&gt; (parse-validate-and-transform "[:foo 42]")
Execution error (ExceptionInfo) at edamalli.core/fail! (REPL:2).
so bad 42/{:row 1, :col 7, :end-row 1, :end-col 9}

@zclj there is a small (have not measured) penalty for using ref-schemas, one function hop basically as the values are memoized.

works, thanks

ok, that's fine since I have to do something to solve it anyway, by post-walking or such. Doing it up-front with malli ref considerable make the design simpler. Thanks for the info!