Clojurians Log v2

https://www.reddit.com/r/Clojure/comments/jhq899/did_anyone_migrate_a_nontrivial_project_from_spec/

I did, yes it is more involved 🙂

To be more specific, this project is a (conceptual) port from a clojure.spec project https://github.com/jeroenvandijk/aws.cloudformation.malli

I was generating the clojure.spec definitions from the AWS Cloudformation spec. This generation part was much easier with Malli

There are enough differences that I cannot consider it as a 1 on 1 port

Feel free to respond on Reddit as this was not my own question, just forwarded it here

yeah thanks. I’m not using reddit. I hope the reddit user finds it here 😅

Is there any guidance on how to write your own transformer?

@borkdude there is no guide yet, but lot of tests and some samples in the README.

happy to help if you need any.

@ikitommi I think I asked this before, but I can't find any docs on this, nor can I find the conversation on Zulip. So here it goes:

(def Schema
  [:map [:x int?]])

(defrecord Wrapper [obj loc])

parsed 
;;=> {#script.Wrapper{:obj :x, :loc {:row 1, :col 2, :end-row 1, :end-col 4}} #script.Wrapper{:obj 1, :loc {:row 1, :col 5, :end-row 1, :end-col 6}}}

(defn wrapper-transformer []
  (mt/transformer
   {:name :wrapper
    :default-decoder :obj
    :default-encoder (fn [obj]
                       (->Wrapper obj nil))}))

(prn (m/decode Schema parsed wrapper-transformer)) ;;=> nil ... ???

so I want to validate/transform the wrapped value according to the schema

any value that is not Wrapped should just be transformed as is

but a value that is wrapped should be unpacked using :obj

and I want to give an error message based on :loc

oh, i recall. need to dig in my histories too if there was/is a good answer.

I tried this:

(defn unwrap [x]
  (if (instance? Wrapper x)
    (:obj x)
    x))

(defn wrapper-transformer []
  (mt/transformer
   {:name :wrapper
    :default-decoder unwrap
    :default-encoder (fn [obj]
                       (->Wrapper obj nil))}))

it seems it doesn't handle the value recursively

the problem is that the decoder is first given a map, it calls :obj on it, which return nil.

I would use clojure.walk/pre-walk for first sweep of decoding.

the decoding here walks: 1. the :map 2. the keys (`:x`)

unless the Wrappedis already decoded on 1, there is no :x and the decoding stops.

so, with malli, you would need to decode all the keys & values on :map step also. doable, but extra noise 😞

(ns user
  (:require [malli.core :as m]
            [malli.transform :as mt]))

(def Schema
  [:map [:x int?]])

(defrecord Wrapper [obj loc])

(def parsed {(map->Wrapper {:obj :x, :loc {:row 1, :col 2, :end-row 1, :end-col 4}})
             (map->Wrapper {:obj 1, :loc {:row 1, :col 5, :end-row 1, :end-col 6}})})

(defn unwrap [x]
  (if (instance? Wrapper x) (:obj x) x))

(defn wrapper-transformer []
  (mt/transformer
    {:name :wrapper
     :decoders {:map (fn [x] (reduce-kv (fn [acc k v] (assoc acc (unwrap k) (unwrap v))) {} x))}
     :default-decoder unwrap
     :default-encoder (fn [obj] (->Wrapper obj nil))}))

(m/decode Schema parsed wrapper-transformer) ;;=> nil ... ???
; => {:x 1}

but, this is much simpler:

(clojure.walk/prewalk unwrap parsed)
; => {:x 1}

Yes, but how do I get validation errors if I first call prewalk on this thing?

based on :loc

I’ll check more of this later.

Thanks. Me too, cooking dinner :)

Hmm, in spec I would maybe have to spec a key as ::wrapped-int and then coerce it after it was checked?

which is not ideal

can the wrapped by anywhere? Wrapped map/vector/set? All values are wrapped (any nested edn value)? Or just keys and values in the map?

I have an idea, but would like to understand the case first.

@ikitommi The use case is preserving location information for non-iobjs and using that for error messages while validating malli schemas

e.g. you want to validate an EDN file and you get an error: this should be an int, on line 5, row 12

@ikitommi This is the complete code: deps.edn:

{:deps {metosin/malli {:mvn/version "0.2.1"}
        borkdude/edamame {:git/url "<https://github.com/borkdude/edamame>"
                          :sha "ba93fcfca1a0fff1f68d5137b98606b82797a17a"}}}

(ns script
  (:require [edamame.core :as e]
            [malli.core :as m]
            [malli.transform :as mt]))

(def Schema
  [:map [:x int?]])

(defrecord Wrapper [obj loc])

(defn iobj? [x]
  (instance? clojure.lang.IObj x))

(def parsed
  (e/parse-string "{:x 1}"
                  {:postprocess
                   (fn [{:keys [:obj :loc]}]
                     (if (iobj? obj)
                       (vary-meta obj merge loc)
                       (->Wrapper obj loc)))}))

(defn unwrap [x]
  (if (instance? Wrapper x)
    (:obj x)
    x))

(defn wrapper-transformer []
  (mt/transformer
   {:name :wrapper
    :default-decoder unwrap
    :default-encoder (fn [obj]
                       (->Wrapper obj nil))}))

;; (prn parsed)
(prn (m/decode Schema parsed wrapper-transformer))

@ikitommi Found the conversation here: https://clojurians.zulipchat.com/#narrow/stream/180378-slack-archive/topic/malli/near/206235546

@ikitommi So the way it can work is:

(defn fail! [schema {:keys [:obj :loc]}]
  (throw (ex-info (str obj " did not satisfy " schema
                       " [at " (str (:col loc)":"(:row loc)) "]") {})))

(def <42 [:and 'int? [:< 42]])

(defn lift-non-iobj-schema [schema]
  [:map {:encode/success :obj,
         :encode/failure (partial fail! schema)} [:obj schema]])

(def Schema [:map [:a (lift-non-iobj-schema <42)]])

(def valid? (m/validator Schema))
(def success (m/encoder Schema (mt/transformer {:name :success})))
(def failure (m/encoder Schema (mt/transformer {:name :failure})))

E.g. this will print:

(prn (parse-validate-and-transform "{:a 42}"))

Syntax error (ExceptionInfo) compiling at (script.clj:32:1).
42 did not satisfy [:and int? [:< 42]] [at 5:1]

The downsize of this is that I have to wrap schemas myself in case I want to check something non-iobj-ish (strings, keywords, numbers)

And this doesn't seem to work for keywords for example:

(def <42 [:and 'int? [:< 42]])

(defn lift-non-iobj-schema [schema]
  [:map {:encode/success :obj,
         :encode/failure (partial fail! schema)}
   [:obj schema]])

(def Schema [:map [(lift-non-iobj-schema :a) (lift-non-iobj-schema <42)]])

anyway, maybe this is a too niche use case

have an idea, need 30min to test & play

take yer time

if this can be fixed, potentially it would also work for rewrite-cljc structures: normal malli specs, but they run over the rewrite-cljc nodes using a simple function that looks at the actual value

it's basically a projection from wrapped thing to the essential value while the wrapped thing has more info to produce useful output in case of failure to parse/validate