clojure-spec

About: http://clojure.org/about/spec Guide: http://clojure.org/guides/spec API: https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html
practicalli-john 2020-06-06T07:05:04.165400Z

Question about spec keys. It seems I can use either qualified keys with :req or unqualified keys with :req-un . I cannot use qualified keys with :req-un. So if I have keys that could be qualified or unqualified, would I have req and req-un sections in the keys

(spec/def ::customer-details
  (spec/keys
    :req [::first-name ::last-name ::email-address ::residential-address ::social-security-id]
    :req-un [::first-name ::last-name ::email-address ::residential-address ::social-security-id]))
Or I assume I should just choose to use one or the other. I could just add the #:: auto-resolve macro to hash-maps with unqualified keys...

practicalli-john 2020-06-06T07:14:57.166200Z

Adding a specific namepace in front of the hash-map, #:practicalli.bank-account-spec seems to work too and feels like a better approach

(spec/explain :practicalli.bank-account-spec/customer-details
              #:practicalli.bank-account-spec
              {:first-name          "Jenny"
               :last-name           "Jetpack"
               :email-address       "<mailto:jenny@jetpack.org|jenny@jetpack.org>"
               :residential-address "42 meaning of life street"
               :postal-code         "AB3 0EF"
               :social-security-id  "123456789"}  )

Aron 2020-06-06T07:26:30.168900Z

I haven't really had time to watch all the videos, is there one where the organization of the namespaces (especially in relation with models, specs, entities)? I am coming from js where circular dependencies are allowed in certain circumstances and I find it a bit too much work to create new files to create new namespaces (note, it's not the problem that I have to create files, I love lots of tiny files, I just don't like the now additional work that means I have to constantly rewrite all my files if I want to reorganize just a couple.)

practicalli-john 2020-06-06T07:28:51.170800Z

@ashnur I will be covering a bit of organisation in the broadcast today in about 30 minutes https://youtu.be/jUgm4zh-vF4 - I create a src, test and spec namespace and have a design-journal namespace to show my working 🙂 It still fairly basic, so you dont need to watch the other videos

Aron 2020-06-06T07:29:33.171100Z

thanks, I will be watching

đź‘Ť 1
dev.4openID 2020-06-06T11:36:05.178700Z

There seems some confusion about :req and :req-un in regards to specs. In regards to a s/valid? or s/explain-str a lot of error info is generated. So you s/def uses :: and can compose other s/defs with :: However when using it with s/explain-str and it has a :req means (e.g. (s/explain-str ::myspec {:test1 "a" :test2 "b"))} it will generate errors versus (e.g. (s/explain-str ::myspec {::test1 "a" ::test2 "b"})) - assuming test1 and test2 are s/def somewhere. At the moment the workaround is just use req-un and that gets past the problem. However, that seems a hack. Anyone with a good explanation anywhere as to the appropriate use of :req and :req-un. This seems more complicated to the eyeball than "it's good enough". I see similar being asked above

dev.4openID 2020-06-07T20:59:57.203700Z

@alexmiller Hi Alex

(s/def ::timestamp string?)  ;; =&gt; :parceltrax.tracking.spec/timestamp
(s/def ::status string?) ;; =&gt; :parceltrax.tracking.spec/status
(s/def :o/status (s/keys :req-un [::timestamp ::status])) ;; =&gt; :o/status
(s/valid? :o/status {:timestamp "2020-09"
                     :status "abc" }) ;; =&gt; true
(s/explain-str :o/status {:timestamp "2020-09"
                          :status "abc" }) ;; =&gt; "Success!\n"

(s/def ::mapx (s/keys :req-un [:o/status])) ;; =&gt; :parceltrax.tracking.spec/mapx
(s/valid? ::mapx {:status {:timestamp "2020-09"
                           :status "abc" }}) ;; =&gt; true
(s/explain-str ::mapx {:status {:timestamp "2020-09"
                                :status "abc" }}) ;; =&gt; "Success!\n"
I accept is works and maybe I'm being a little thick here: The solution depends of the "temp" creation of a namespace as a workaround. In this case a :o/status in order to differentiate from the actual :req-un namespace of :status. My perception is this in not very elegant or consistent with the spec model. As is encouraged by the clojure team, developers are to use :: more often. In the simple example provided - the code is not "portable" as such. I recognize it may seem may seem nitpicking - perhaps I have missed the understanding or this is just accept "it works this way" In the real workplace there are many occasions of "recursive dup fields in JSON data. Could you provide a comment or two on this? Much appreciate your efforts

alexmiller 2020-06-07T21:29:41.205300Z

I’m not trying to encourage more use of :: and it’s typically not what you need (but it’s very convenient for examples)

alexmiller 2020-06-07T21:30:17.206Z

Personally I most often use full namespaces for data specs

dev.4openID 2020-06-08T09:32:03.211800Z

Hi, I was not stating you were the advocate - the spec literature does - just to be clear Any thought about my point of "making up kw names" in conjunction with the point of recursive/nested JSON map structures that employ a keword such as my example :status - is there no recursive strategy for example? I don't know thus I ask.

dev.4openID 2020-06-08T09:32:12.212Z

Thanks for your patience

tianshu 2020-06-06T13:18:50.179800Z

Is it expected if :a/b is not defined in this case?

(s/def ::c
  (s/keys :req [:a/b]))


(s/valid? ::c {:a/b {}})
;; =&gt; true

alexmiller 2020-06-07T13:33:35.200700Z

Yes, in general resolution should be delayed for all specs and it’s not entirely true right now. Not going to make any updates on this in spec 1 though, will work on it in spec 2

hadils 2020-06-06T13:34:58.182400Z

I am having a very difficult time organizing my specs with my code. This is for code that interfaces to an external API. I have a single namespace for the code and an specs.namespace for the specs. The problem is that I have to rename fields in different args and returns for the API code so they won't collide with different uses. Help!!!

dev.4openID 2020-06-07T16:45:02.203300Z

@hadilsabbagh18 Do you mean 2 fields in the JSON structure, where 1 is subordinate to another? {:name {:person "abc" :name}} or do you mean recurring pasterns that are in a map?

dev.4openID 2020-06-07T16:45:42.203500Z

For the former: (s/def :inner/status string?) ;; => :inner/status (s/def :inner/timestamp string?) ;; => :inner/timestamp (s/def :outer/status (s/keys :req-un [:inner/timestamp :inner/status])) ;; => :outer/status (s/def :outer/map (s/keys :req-un [:outer/status])) ;; => :outer/map (s/valid? :outer/map {:status {:timestamp "2020-09" :status "abc" }}) ;; => true (s/explain-str :outer/map {:status {:timestamp "2020-09" :status "abc" }}) ;; => "Success!\n"

dev.4openID 2020-06-06T13:43:01.182500Z

No. Imagine they are defined It's not about a/b nut when :req or :req-un is appropriate

dev.4openID 2020-06-06T13:45:23.183Z

I have just done this and it is OK for code and spec to be in different files I assume you have converted the JSON to a structure. Now define the spec "structure" accordingly Apply s/conform ::spec "JSON structure" - note the spec has :: and your JSON map has : for all elements - in spec make sure all :req are :req-un and it will work

dev.4openID 2020-06-06T13:50:50.183500Z

in my case the file my namespace is ns: myproj.trail.clj and myproj.trial.spec.clj

dev.4openID 2020-06-06T13:52:11.185100Z

Lesson learnt model the API JSON map example in spec file fisrst and all becomes more clear. I started from leaves to trunk and not trunk down as seemingly many do

alexmiller 2020-06-06T13:52:24.185600Z

I would say yes, this is expected. :req means it’s required (and it’s there). If there are specs for any keys, they must be followed.

hadils 2020-06-06T13:53:00.185800Z

Thanks! @dev.4openid

hadils 2020-06-06T13:54:43.187800Z

What about for multiple JSON outputs in the same spec file? How do I do that?

hadils 2020-06-06T13:57:20.190700Z

That's the real problem I'm having.

dev.4openID 2020-06-06T13:57:47.191300Z

If you break down the JSON maps into components and composed elements that are reusable. You then compose complete "trees" for all the responses for all the API calls. There are commonalities in the APIs returns and it goes faster in the compositions as you do more. i.e do 1 at a time. I went with the most complex one (in order to lean spec) and it just became easier to gen the rest.

hadils 2020-06-06T13:58:23.192300Z

How do you qualify your spec keys?

alexmiller 2020-06-06T14:00:00.194600Z

First, spec know nothing about :: - this is just a Clojure shorthand to fully resolve a keyword within the current namespace at read time. Spec will just see the full kw as if you had typed :user/myspec. s/keys will always validate all fully qualified keys in the map with their associated registered specs. It will check that the map has all of the specified :req keys (match fully qualified) or :req-un keys (match only the unqualified name). And specially for :req-un it will verify the values validate against the spec for the qualified key.

dev.4openID 2020-06-06T14:00:04.194800Z

It becopmes more and more obvious. Just build the leaves and branches slowly - use s/valid? and s/explain-str against sample data - extracted from the JSON map eventually build more complex branches and tada! you will compose the trunk. It seem tedious at first (as I was leaning) but it become faster

hadils 2020-06-06T14:03:08.195200Z

I am hung up on the qualified keys like ::foo instead of using :node/foo. The latter seems more appropriate for my needs. How should I approach this?

dev.4openID 2020-06-06T14:04:17.195400Z

look at the toy example I made first here <script src="https://gist.github.com/dev4openid/5c9320fb8ef2f8f5383836f379ff507e.js"></script> It is tedious at first BUT it gave me insights on how to do things I will leave it up for a while

dev.4openID 2020-06-06T14:05:26.195700Z

in the spec file use :: format for your definitions - saves typing

hadils 2020-06-06T14:05:41.195900Z

Thanks!

dev.4openID 2020-06-06T14:09:00.196100Z

note the :reg-un !!

hadils 2020-06-06T14:25:04.196300Z

Your Gist does not show up properly on my browser.

dev.4openID 2020-06-06T14:30:34.196500Z

(def exampl {:abc [{:loca {:addr {:country "USA"}}
                    :dets {:some{:timestamp "2020-04-09T15:27:00"
                                 :Url "<https://mywebpage.com/answer>"
                                 :prod {:name "this product"}}}
                    :totalNumber 399
                    :pieceID ["ABC1" "DEF2" "GHI3"]}]})


(s/def ::country string?)
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un [::addr]))

(s/conform ::country "USA") ;; =&gt; "USA"
(s/conform ::addr {:country "USA"}) ;; =&gt; {:country "USA"}
(s/conform ::loca  {:addr {:country "USA"}}) ;; =&gt; {:addr {:country "USA"}}
(s/valid? ::loca  {:addr {:country "USA"}})  ;; =&gt; true
(s/check-asserts?)
(s/check-asserts true)                       ;; by default false!!
(s/assert ::loca  {:addr {:country "USA"}})  ;; =&gt; {:addr {:country "USA"}}
(s/assert ::loca  {:addr {:countryx "SA"}})  ;; note the exception error BUT look at the repl

(s/def ::timestamp (and not nil?
                        string?))     ;; this needs work for T-time issue
(s/def ::Url string?)                 ;; need to validate URLs - technique
(s/def ::name string?)
(s/def ::prod (s/keys :req-un [::name]))
(s/def ::some (s/keys :req-un [::timestamp ::Url ::prod]))
(s/def ::dets (s/keys :req-un [::some]))

(s/conform ::timestamp "2020-04-09T15:27:00")
(s/conform ::Url "<https://mywebpage.com/answer>")
(s/conform ::name "this product")
(s/conform ::prod {:name "this product"})
(s/conform ::some {:timestamp "2020-04-09T15:27:00"
                   :Url "<https://mywebpage.com/answer>"
                   :prod {:name "this product"}})
(s/conform ::dets {:some {:timestamp "2020-04-09T15:27:00"
                           :Url "<https://mywebpage.com/answer>"
                           :prod {:name "this product"}}})

(s/def ::totalNumber (and pos? int?))
(s/def ::pieceID (and (s/coll-of string?)
                      vector?))
(s/conform ::totalNumber 399) ;; =&gt; 399
(s/conform ::pieceID ["ABC1" "DEF2"]) ;; =&gt; ["ABC1" "DEF2"]

(s/def ::abc (s/coll-of (s/keys :req-un [::loca ::dets ::totalNumber ::pieceID])))
(s/conform ::abc [{:loca {:addr {:country "USA"}}
                   :dets {:some{:timestamp "2020-04-09T15:27:00"
                                :Url "<https://mywebpage.com/answer>"
                                :prod {:name "this product"}}}
                   :totalNumber 399
                   :pieceID ["ABC1" "DEF2" "GHI3"]}])

;; now ex is an map of a single instance)
ex
;; =&gt; {:abc [{:loca {:addr {:country "USA"}}, :dets {:some {:timestamp "2020-04-09T15:27:00", :Url "<https://mywebpage.com/answer>", :prod {:name "this product"}}}, :totalNumber 399, :pieceID ["ABC1" "DEF2" "GHI3"]}]}

(s/def ::an_instance (s/keys :req-un [::abc]))
(s/conform ::an_instance {:abc [{:loca {:addr {:country "USA"}}
                                 :dets {:some{:timestamp "2020-04-09T15:27:00"
                                              :Url "<https://mywebpage.com/answer>"
                                              :prod {:name "this product"}}}
                                 :totalNumber 399
                                 :pieceID ["ABC1" "DEF2" "GHI3"]}]})

(s/conform ::an_instance exampl)
(s/valid? ::an_instance exampl)
(s/explain-str ::an_instance exampl)

dev.4openID 2020-06-06T14:31:17.196700Z

strange

hadils 2020-06-06T14:35:28.196900Z

Thanks a lot @dev.4openid!!!!

hadils 2020-06-06T16:22:51.197500Z

@dev.4openid what do you do when you are trying to spec 2 different JSON outputs with same names -- some of the fields are identical and some are not, but have the same name?

hadils 2020-06-06T16:23:15.197700Z

Do I need to split them out into separate files?