malli

https://github.com/metosin/malli :malli:
willier 2021-03-19T03:12:39.076500Z

spec->malli was mentioned here a while ago. is anyone aware if such a thing exists?

ikitommi 2021-03-19T06:03:07.078300Z

@emccue currently it’s a let. Could be something else too. What kind of better errors would you expect? The reference information is available over there, just not used I guess in explain:

(m/explain
  [:map {:registry {"ID" :int}}
   [:id "ID"]]
  {:id "123"})
;{:schema [:map {:registry {"ID" :int}} [:id "ID"]],
; :value {:id "123"},
; :errors (#Error{:path [:id], :in [:id], :schema :int, :value "123"})}

ikitommi 2021-03-19T06:03:23.078700Z

@willier not yet. interested in doing? πŸ˜‰

willier 2021-03-19T08:43:01.080200Z

hmm, probably need a black belt in macro-fu to do this job

nilern 2021-03-19T08:44:07.080600Z

Or a spec-tools belt?

borkdude 2021-03-19T08:56:31.082Z

@willier you can inspect specs at runtime. spec-tools has something called a walker and coax is a library from exoscale to do coercion based on specs, I think they are doing something similar

πŸ‘ 1
ikitommi 2021-03-19T08:58:01.083200Z

spec-tools has both walker and a visitor πŸ™‚ visitor is the right tool for the job i believe: https://cljdoc.org/d/metosin/spec-tools/0.10.5/doc/spec-visitor

ikitommi 2021-03-19T08:58:47.083400Z

json schma with it: https://cljdoc.org/d/metosin/spec-tools/0.10.5/doc/json-schemas

willier 2021-03-19T09:00:58.085500Z

oh interesting... i will have a look into it - thanks!

ikitommi 2021-03-19T09:01:30.086200Z

also, there are progression tests in spec-tools for broken forms: if the core would be fixed: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/visitor_all_test.cljc

ikitommi 2021-03-19T09:03:32.086600Z

oh, no more, tests disabled and the last form is fixed. great!

danielneal 2021-03-19T09:43:19.087400Z

If you have a map that references other schemas, like this [:map ::some/ref ::some-other/ref] how do you make ::some/ref and ::some-other/ref optional?

danielneal 2021-03-19T09:47:50.089500Z

I tried [:and {:optional true} ::some/ref] but that was invalid

nilern 2021-03-19T09:49:23.091Z

You just have to use the sugar-free [:map [::some/ref {:optional true} ::some/ref]] You could get clever and [:map (#(vector % {:optional true} %) ::some/ref)] or name and reuse that little fn

nilern 2021-03-19T09:52:57.093Z

[:map ::some/ref] is just some sugar in :map, we don't have "first-class properties" beyond that. And :optional is specific to map entries while :maybe does something else.

danielneal 2021-03-19T09:53:14.093300Z

aha ok thanks πŸ™‚

ikitommi 2021-03-19T10:05:29.094400Z

this should also work: [:map [::some/ref {:optional true}]], no need to repeat the schema for refs.

☝️ 1
danielneal 2021-03-19T10:21:02.094700Z

nice!

danielneal 2021-03-19T10:59:04.096Z

If I'm walking a schema and want to convert all schemas of type ::sq/ref into :string, but I also want to deref all other refs, how do I do this/ This is what I've got so far:

(defn output-schema
  [schema]
  (malli/walk
   schema
   (malli/schema-walker
    (fn [schema]
      (malli/type schema)
      (cond
        (= schema ::sq/ref) :string
        (= (malli/type schema) :map)
        .... do some other stuff
        :else schema)))
   {::malli/walk-schema-refs true
    ::malli/walk-refs true}))
But the schema is expanded by the time the check happens, so it fails

ikitommi 2021-03-19T11:00:48.096900Z

what does (malli/walk schema (malli/schema-walker identity) {::malli/walk-schema-refs true, ::malli/walk-refs true}) do?

ikitommi 2021-03-19T11:01:16.097300Z

that might expand all automatically

ikitommi 2021-03-19T11:09:37.097800Z

(m/walk
  [:schema
   {:registry {"Country" [:map
                          [:name [:enum :FI :PO]]
                          [:neighbors [:vector [:ref "Country"]]]]
               "Burger" [:map
                         [:name string?]
                         [:description {:optional true} string?]
                         [:origin [:maybe "Country"]]
                         [:price pos-int?]]
               "OrderLine" [:map
                            [:burger "Burger"]
                            [:amount int?]]
               "Order" [:map
                        [:lines [:vector "OrderLine"]]
                        [:delivery [:map
                                    [:delivered boolean?]
                                    [:address [:map
                                               [:street string?]
                                               [:zip int?]
                                               [:country "Country"]]]]]]}}
   "Order"]
  (m/schema-walker #(mu/update-properties % assoc :type (m/type %)))
  {::m/walk-schema-refs true})
;[:schema
; {:registry {"Country" [:map [:name [:enum :FI :PO]] [:neighbors [:vector [:ref "Country"]]]],
;             "Burger" [:map
;                       [:name string?]
;                       [:description {:optional true} string?]
;                       [:origin [:maybe "Country"]]
;                       [:price pos-int?]],
;             "OrderLine" [:map [:burger "Burger"] [:amount int?]],
;             "Order" [:map
;                      [:lines [:vector "OrderLine"]]
;                      [:delivery
;                       [:map [:delivered boolean?] [:address [:map [:street string?] [:zip int?] [:country "Country"]]]]]]},
;  :type :schema}
; [:malli.core/schema
;  {:type :malli.core/schema}
;  [:map
;   {:type :map}
;   [:lines
;    [:vector
;     {:type :vector}
;     [:malli.core/schema
;      {:type :malli.core/schema}
;      [:map
;       {:type :map}
;       [:burger
;        [:malli.core/schema
;         {:type :malli.core/schema}
;         [:map
;          {:type :map}
;          [:name [string? {:type string?}]]
;          [:description {:optional true} [string? {:type string?}]]
;          [:origin
;           [:maybe
;            {:type :maybe}
;            [:malli.core/schema
;             {:type :malli.core/schema}
;             [:map
;              {:type :map}
;              [:name [:enum {:type :enum} :FI :PO]]
;              [:neighbors [:vector {:type :vector} [:ref {:type :ref} "Country"]]]]]]]
;          [:price [pos-int? {:type pos-int?}]]]]]
;       [:amount [int? {:type int?}]]]]]]
;   [:delivery
;    [:map
;     {:type :map}
;     [:delivered [boolean? {:type boolean?}]]
;     [:address
;      [:map
;       {:type :map}
;       [:street [string? {:type string?}]]
;       [:zip [int? {:type int?}]]
;       [:country
;        [:malli.core/schema
;         {:type :malli.core/schema}
;         [:map
;          {:type :map}
;          [:name [:enum {:type :enum} :FI :PO]]
;          [:neighbors [:vector {:type :vector} [:ref {:type :ref} "Country"]]]]]]]]]]]]]

ikitommi 2021-03-19T11:10:53.098300Z

not sure if that’s near what you want to do.

danielneal 2021-03-19T11:11:33.098400Z

yep, that expands all

danielneal 2021-03-19T11:13:52.099500Z

Ah I think I said :type wrong, the type is :malli.core/schema, what I'm checking for is the schema itself to be ::sq/ref

danielneal 2021-03-19T11:14:25.100300Z

So is what you're suggesting to do two walks, one to capture information and put it in the properties, and then a second to do the other transformations?

danielneal 2021-03-19T11:18:48.100500Z

hmm

danielneal 2021-03-19T11:20:01.101Z

I suppose I could put some property on the ::sq/ref schema itself, but it feels like I'm missing a trick

nilern 2021-03-19T11:28:54.103600Z

To do it in one traversal you would need a more Visitor-like prewalk where you manually deref the ones you want. I think that can be done with -walk and Walker but only schema-walker and walk are documented and stable ATM

danielneal 2021-03-19T11:32:20.104200Z

thanks

ikitommi 2021-03-19T12:38:31.107600Z

ok, I guess I missed the original point. But there are public walkers like https://github.com/metosin/malli/blob/master/src/malli/util.cljc#L37-L51

danielneal 2021-03-19T15:00:29.108200Z

is it safe to use those functions and protocols?

nilern 2021-03-19T15:03:02.108400Z

https://github.com/metosin/malli/#alpha

nilern 2021-03-19T15:03:12.108600Z

> extender api: public vars, name starts with -, e.g. malli.core/-collection-schema. Not needed with basic use cases, might evolve during the alpha, follow https://github.com/metosin/malli/blob/master/CHANGELOG.md for details

danielneal 2021-03-19T15:27:40.109Z

nice, i see

raymcdermott 2021-03-19T17:36:21.110Z

doing the graphviz thing is a bit whack it turns out cos it passes through all of the constraints to be visualised and it soon gets ugly

raymcdermott 2021-03-19T17:36:25.110100Z

raymcdermott 2021-03-19T17:38:42.111500Z

So I guess you really have to pare back everything to the simplest of all possible models for nice visuals πŸ™‚

nilern 2021-03-19T17:42:06.112600Z

Maybe we could have custom visualization props like we have :error/message?

raymcdermott 2021-03-19T18:45:05.114700Z

yes, or like {:swagger}

raymcdermott 2021-03-19T18:48:19.117300Z

if one had a Member, it would be nice for example to have {:viz/parent Org} on the map and {:viz/type string} on the name property

raymcdermott 2021-03-19T18:49:32.117800Z

and then one could select for those properties when transforming to DOT

raymcdermott 2021-03-19T21:54:47.118800Z

@ikitommi I might have been going about this wrong, so maybe this is now a better explanation of what I would like to have ...

raymcdermott 2021-03-19T21:55:09.119200Z

(def Id [:re #"^[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}$"])
(def Name [:string {:min 3 :max 255}])

(def org-ref :ref)
(def org-ref-viz [:map [org-ref :string]])
(def org-ref-description "Reference (name or ID) of the organisation")
(def org-ref-valid [:map {:title org-ref-description} [org-ref Name]])
(def org-ref-swagger {:swagger {:description org-ref-description
                                :example     "Acme tech, Houston TX"}})
(def Org-Ref [:map {:title (name org-ref)}
              [org-ref org-ref-swagger Name]])

(def org-id :id)
(def org-id-viz [:map [org-id :string]])
(def org-id-description "The organisation ID")
(def org-id-valid [:map [org-id Id]])
(def org-id-swagger {:swagger {:description org-id-description
                               :example     (->id)}})

(def Org-Id [:map {:title (name org-id)}
             [org-id org-id-swagger Id]])

(def org-viz {"Org" (mu/merge org-id-viz org-ref-viz)})
(def Org (mu/merge Org-Id Org-Ref))

raymcdermott 2021-03-19T21:56:50.120400Z

I can't find a way to create org-ref-full by combining org-ref-valid with org-ref-swagger

raymcdermott 2021-03-19T21:57:50.121400Z

I have added org-ref-viz as that's what I would like to derive from org-ref-valid

raymcdermott 2021-03-19T21:59:33.122500Z

At the moment I am OK with doing it the way as it is above cos there is still a lot of value but obviously there seems to be quite a bit of boiler-plate