Clojurians Log v2

sure @ognivo, would like to do that, just super busy at work atm, between projects. If the code - and the goal - is available somewhere, I could first try to read that, between things.

@joel380 not atm. The new malli schemas of malli schemas feature allows us to describe the available properties formally as schemas, but it’s just the mechanism atm, no content shipped. Once all schmas are described (both properties and children), it will be THE documentation. For now, you need to read the sources. Documentation PRs always welcome.

e.g.

;; currently
(m/properties-schema :string)
; => nil

;; after
(m/properties-schema :string)
; [:map
;  [:min {:description "length must be at least"} :int]
;  [:max {:description "length must be at most"} :int]]

after the children & property schemas are done, one can also generate valid schema-hiccup out of a given registry 🙂

Yep, you can look at it here: https://github.com/dvingo/malli-code-gen/blob/main/src/malli_code_gen/gen_eql.clj We have a branch merge pending. If we merge it before you look at the code – we’ll leave you a link in https://github.com/dvingo/malli-code-gen cc @ikitommi Not rushing you with the review. If you can say at what period you will be probably available for that (in a week or three) that would help. Good luck with the workload!

Is there a way to forbid some combination of keys in a map? For instance a map with either :a and :b or :c and :d

probably using another predicate?

or using two different maps

What do you mean by “using another predicate”?

[:and [:map ...] [:fn (fn [...] ...)]]

Same as with clojure.spec

(def my-schema
  [:and
   [:map
    [:x int?]
    [:y int?]]
   [:fn (fn [{:keys [x y]}] (> x y))]])

I was just going to ask if there's a way to specify constraints on relations between keys which does not involve defining an ad-hoc function

Maybe using two separate closed maps also works. I think it depends on your domain perhaps?

Likely, although I was just thinking of a case like the one you illustrated, a constraint where the value at one key is greater than another

This pattern repeats so often, should it be a schema?

Something like:

(def preds {:> >, :>= >=, :< <, :<= <=, := =, :not= not=})

(defn -rel
  []
  (m/-simple-schema
   (fn [_ [pred a b]]
     {:type :rel,
      :pred (m/-safe-pred #((preds pred) (get % a) (get % b))),
      :min 3,
      :max 3})))

(m/validate
 (m/schema
  [:and
   [:map
    [:x :int]
    [:y :int]]
   [:rel :< :x :y]]
  {:registry
   (mr/composite-registry
    m/default-registry
    {:rel (-rel)})})
 #_{:x 1 :y 2}
 {:x 1 :y 1})

reminds me of jet :)

$ echo  '{:a 1 :b 2}' | jet --query '(< :a :b)'
true
$ echo  '{:a 1 :b 2}' | jet --query '(> :a :b)'
false

I think it’s good idea to experiment new relational schemas in the user space, both -simple-schema & -collection-schma are good ways to make these, like @ben.sless you demoed. The more stuff pushed into these data-languages, the more it looks like a simple sci.

there is also things like :select-keys -> https://malli.io/?value=%7B%3Aa%201%2C%20%3Ab%202%7D&schema=%5B%3Aschema%0A%20%7B%3Aregistry%20%7B%22ABCD%22%20%5B%3Amap%20%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Aa%20%3Aany%5D%20%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Ab%20%3Aany%5D%20%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Ac%20%3Aany%5D%20%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Ad%20%3Aany%5D%5D%7D%7D%20%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%5B%3Aor%20%0A%20%20%5B%3Aselect-keys%20%22ABCD%22%20%5B%3Aa%20%3Ab%5D%5D%0A%20%20%5B%3Aselect-keys%20%22ABCD%22%20%5B%3Ac%20%3Ad%5D%5D%5D%5D.

I also had some thoughts on the subject here https://clojureverse.org/t/declarative-rules-for-relations-between-inputs/7623/5?u=bsless

A big plus it could have over sci is getting the benefit of the JIT while still keeping the schema serializable

only 👍

true, sci introduces a lot of overhead. and is big on cljs.

would meander be great at describing the relations, as data?

much smaller than self-hosted CLJS though

but how many people are really using schema serialization?

I think most people are not

i think so too.

Does select-keys translate to mu/get of all the keys then closing the schema?

but a lightweight map key dependency utility might be a good fit for many things. just (somone) needs (to invent) more syntax.

The schema serialization isn't my first (or third) priority here tbh, it's more about having more expressive schemas for the common 90% of cases and not having to define ad-hoc functions and errors for them, not to mention generators.

https://github.com/metosin/malli/blob/master/src/malli/util.cljc#L233-L239

https://github.com/metosin/malli/blob/master/src/malli/util.cljc#L233-L239

yeah, I think that's nice. I think clojure spec also doesn't have a good answer to this yet and it's pretty common

malleander!

using modified ben’s syntax:

[:and
 [:map
  [:a :any]
  [:b :any]
  [:c :any]
  [:d :any]]
 [:rel
  [:or
   [:and :a :b]
   [:and :c :d]]]]

The only objection I can come up with is that meander is too powerful, (i.e. the result will be hard to work with)

I think it can be confusing to overload the meaning of :or within :rel

"Here, have this fighter-jet fueled by liquid plutonium" Scary

Maybe just [:and [:map ....] [:or [:required-keys :a :b] [:required-keys :c :d]] [:gtk :a :b]]

(gtk = greather than for keys, or something)

The rel syntax needs to be properly thought out, I just invented something ad-hoc to see if it could work

[:keys/> :a :b]

How about :constraints?

[:keys/require :a :b]

Then constraints unify by default (like datalog) or disjoin when specified (via :or)

[:k> :a :b], [:kreq :a :b]

this works already:

(m/validate
  [:schema {:registry {::a :int
                       ::b :int
                       ::c :int
                       ::d :int}}
   [:or
    [:map ::a ::b]
    [:map ::c ::d]]] 
  {::a 1, ::b 2})
; => true

yeah, that's what I said in the start of the conversation: two disjunct map defs

just not with non-qualified keys (for no good reason)

[:constraints
 [:or
  [:and
   [:requires :a :b]
   [:> :a :b]]
  [:and
   [:requires :c :d]
   [:< :c :d]]]]

but for rels between keys you could just have specialized ops like [:k> :a :b]: the :a key must be greater than the :b key, without introducing some new concept

i kinda like it.

special ops rub me the wrong way, for some reason. What's wrong with just :>?

but question is: from whom the declaration are for? schema writer? user? external docs?

(m/validate [:> 6] 4)
; => false

it already exists

Is there an answer to this in TypeScript?

hm, yes, but as a schema not an argument (if we go back to the [:rel x y z] or [:constraint ,,,] suggestion

> What's wrong with just :> That it means different things in different contexts, this can be confusing imo.

What if you want to really do numeric comparison like [:> :a 5] and 5 is also a key in a map?

In jet I chose [:> :a #jet/lit 5]

but quickly this became tedious so I wrote a clojure interpreter

I think you have to use union types with ts

But it doesn’t scale. When you have other set of constraints you’d need to write down all the combinations of valid keys.

The most shorthand constraint syntax I can think of would be :! An alternative which conforms to your suggestion would be :!/>

Downside - it looks like Perl Upside - concise and unique syntax

you should always make a good trade-off between adding extra syntax (= complexity) and how much people are really going to use this

[:and
 [:map
  [:a :any]
  [:b :any]
  [:c :any]
  [:d :any]]
 [:keys/or
  [:keys/and :a :b]
  [:keys/and :c :d]]]

yeah, I like that a lot better

why even :keys/or, you can just use the normal :or here?

I was about to write the same thing

Then adding a predicate function on the keys would look like

[:and
 [:map
  [:a :any]
  [:b :any]
  [:c :any]
  [:d :any]]
 [:or
  [:and
   [:keys/and :a :b]
   [:keys/< :a :b]]
  [:keys/and :c :d]]]

The disjunction must not be about the keys

true

:keys/or should mean "at least k1 or k2 should exist"

what does :keys/and mean: both keys must be present? what does [:keys/and :a] mean then? Maybe [:keys/req :a] is a better name?

And now, where would we put a custom predicate [:fn …]?

Maybe even without a shorthand, keys/require

or :keys/with and :keys/without ;P

:keys/req doesn’t convey the fact that :a and :b are related

off to cook some food. good discussion, as always 👍

but what exactly does [:keys/and :a :b] mean then, how are these things related?

Thank you, feel like we hit on something useful and needed Bon appetit!

I didn’t say I liked :keys/and

Maybe [:keys/present :a :b]?

By the way, my use case is real for a project at work.

but you also didn't say what the semantic relationship between those keys is according to that op. > :keys/req doesn’t convey the fact that :a and :b are related :present also doesn't communicate a relationship, just as :required doesn't, it just means all those keys must be required/present

same

We are writing a Hbase driver

I guess you can already express required keys like this as well? [:and [:map ...] :a :b :c] or do keywords not behave like predicates in malli?

The scan function receives a map that could contain either :from and :to or :starts-with

(m/schema [:and :a :b])

It looks like malli is becoming really popular in Israel?

ok, [:and [:fn :a] [:fn :b]] ? :thinking_face:

(ugly)

There are not so many Clojure shops i Israel. Therefore, we could already say that 10% of Israeli Clojure shops use malli 😁

And non informative, and useless for generation

(me/humanize (m/explain (m/schema [:and [:fn :a]]) {:b 1}))
;; => #:malli{:error ["unknown error"]}

agreed

I’m struggling to write the schema of the map received by scan

[:map    
   [:starts-with :string]
   [:from :string]
   [:to :string]
 [:xor 
   [:keys/req :starts-with]
   [:keys/req :from :to]
   [:keys/req :from]
   [:keys/req :to]
 ]
]

:or is not good in my case. It should be exclusive or

if it is xor, then why not use disjunct maps?

Because there are other keys

maybe add a :type field or something?

:time-range , :limit …

[:map    
   [:starts-with :string]
   [:from :string]
   [:to :string]
   [:limit :number]
   [:time-range [:map [:from-ms :int] [:to-ms :int]]
 [:xor 
   [:keys/req :starts-with]
   [:keys/req :from :to]
   [:keys/req :from]
   [:keys/req :to]
 ]
]

What is a :type field?

I think you could have different types of ranges

{:range-type :closed}, {:range-type :open}

In this case I'd start with some base schema, merge the different options with it and or between them

I was wrong. There is no relationship between the keys. But somehow I find it weird to say “required” inside an or condition

Does it help to define constraints on map keys

?

I don’t get what both of you suggested

@viebel I think this requirement is a bit weird:

:xor
[:keys/req :from :to]
[:keys/req :from]

I did not look at the other requirements, but I think your logic can be "refactored"

by modeling it differently

hmm, this is a tricky one

[:or [:map {:closed true}
       [:from ...]
       [:to   ...]]
     [:map {:closed true}
       [:starts-with ...]]]

OR is an exclusive or if the types don't intersect

[:map 
  [:from ...]
  [:to   ...]]

but if you have it closed you can't have anything other than from and to

and if you have it open you leave open the possibility that :starts-with is in the map as well

what you need is basically

[:or [:and [:map 
             [:from ...]
             [:to ...]]
           [:not [:map 
                   [:starts-with ...]]]]
     [:and [:map 
             [:starts-with ...]]
           [:not [:map 
                   [:from ...]
                   [:to   ...]]]]

OR

[:or [:and [:map 
             [:from ...]
             [:to ...]]
           [:not [:map 
                   [:starts-with ...]]]]
     [:and [:map 
             [:starts-with ...]]
           [:not [:map [:from ...]]
           [:not [:map [:to ...]]]]]

not that not exists, but conceptually thats what you would need to represent the constraint

since you only want to be closed against specific keys

if you are willing to explicitly enumerate all the keys you can just do the version with {:closed

I meant something like

(def Base
  [:map
   [:limit :number]
   [:time-range [:map [:from-ms :int] [:to-ms :int]]]])

(def S1 (mu/merge Base [:map [:starts-with :string]]))
(def S2 (mu/merge Base [:map [:from :string] [:to :string]]))

Did some generalizing work, still not set on the syntax

(defn comparator-relation
  [sym msg]
  (let [f @(resolve sym)
        type (keyword "!" (name sym))]
    [type
     (m/-simple-schema
      (fn [_ [a b]]
        (let [fa #(get % a)
              fb #(get % b)]
          {:type type
           :pred (m/-safe-pred #(f (fa %) (fb %))),
           :type-properties
           {:error/fn
            {:en (fn [{:keys [schema value]} _]
                   (str
                    "value at key "
                    a ", "
                    (fa value)
                    ", should be "
                    msg
                    " value at key "
                    b
                    ", "
                    (fb value)))}}
           :min 2,
           :max 2})))]))

(defn -comparator-relation-schemas
  []
  (into
   {}
   (map (fn [[sym msg]] (comparator-relation sym msg)))
   [['>  "greater than"]
    ['>= "greater than or equal to"]
    ['= "equal to"]
    ['== "equal to"]
    ['<= "lesser than or equal to"]
    ['< "lesser than"]]))


(me/humanize
 (m/explain
  (m/schema
   [:and
    [:map
     [:x :int]
     [:y :int]]
    [:!/> :x :y]]
   {:registry
    (mr/composite-registry
     m/default-registry
     (-comparator-relation-schemas))})
  {:x 1 :y 1}))

I don’t see how to refactor the logic. All the followings are valid • :from 10 :to 20 • :form 10 • :to 20 • :starts-with aaa

The followings are invalid

• :from 10 :starts-with aaa • :to 20 :starts-with aaa • “from 10 :to 20 starts-with aaa

I'm surprised that TypeScript is sophisticated enough to be used as inspiration, I guess I need to learn more. I've presumed Elm/Haskell are more sophisticated, eg.: https://www.youtube.com/watch?v=IcgmSRJHu_8

Hey @danvingo > do you have any links or materials on alloy as it relates to crux? Nothing published from me yet, but this is worth a look https://www.hillelwayne.com/post/formally-modeling-migrations/ and, whilst not directly Alloy-related, I think principles of https://en.wikipedia.org/wiki/Object-role_modeling are also relevant

ooohh this looks great, thanks for sharing! I have some reading to do 🙂