clojure-spec

About: http://clojure.org/about/spec Guide: http://clojure.org/guides/spec API: https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html
borkdude 2020-11-03T12:39:11.161800Z

Is there any existing work combining clojure-spec conform + core.match or datalog? Like: I want ?x and ?y from (s/conform ...)? I could see this being useful for grasp

borkdude 2020-11-03T12:41:21.162Z

Maybe meander is a good match for this

2020-11-03T12:54:57.162200Z

take a look at mine matchete https://github.com/xapix-io/matchete if meander is not a good fit

borkdude 2020-11-03T12:58:48.162500Z

thanks. can you describe the difference between meander and yours?

2020-11-03T13:01:40.162700Z

• no macro • less power -> cleaner pattern syntax (this is just my opinion, btw) • much smaller codebase -> much easier to read through and explore

borkdude 2020-11-03T13:02:32.162900Z

:thumbsup:

borkdude 2020-11-03T14:34:46.163100Z

@delaguardo How do I express this meander usage in your lib?

(m/find (first conformed) {:clauses {:clause {:sym !interface} :clauses [{:sym !interface} ...]}} !interface)

borkdude 2020-11-03T14:35:01.163300Z

The output should be something like:

[clojure.lang.IDeref clojure.lang.IBlockingDeref clojure.lang.IPending java.util.concurrent.Future]

borkdude 2020-11-03T14:35:17.163500Z

And the input:

{:reify reify, :clauses {:clause {:sym clojure.lang.IDeref, :lists [(deref [_] (deref-future fut))]}, :clauses [{:sym clojure.lang.IBlockingDeref, :lists [(deref [_ timeout-ms timeout-val] (deref-future fut timeout-ms timeout-val))]} {:sym clojure.lang.IPending, :lists [(isRealized [_] (.isDone fut))]} {:sym java.util.concurrent.Future, :lists [(get [_] (.get fut)) (get [_ timeout unit] (.get fut timeout unit)) (isCancelled [_] (.isCancelled fut)) (isDone [_] (.isDone fut)) (cancel [_ interrupt?] (.cancel fut interrupt?))]}]}}

borkdude 2020-11-03T14:52:34.163700Z

Here's the full example: https://github.com/borkdude/grasp#meander

Jim Newton 2020-11-03T15:27:04.165Z

I'm starting looking at clojure.spec.alpha can someone explain the different between the s/valid? function and the s/spec? function? Their docstrings look pretty similar.

borkdude 2020-11-03T15:27:58.165500Z

@jimka.issy spec? is more like an instance check, valid? validates a value against a spec

borkdude 2020-11-03T15:28:36.165800Z

in fact, it is an instance check :)

Jim Newton 2020-11-03T15:33:40.167900Z

As I understand the normal use case is the user associates certain tags, normally starting with :: with some DSL object which tells how to check validity? Does spec? check whether the symbol has been associated with a spec object, or does it check whether this is such an object?

Jim Newton 2020-11-03T15:34:01.168400Z

How can I know whether a symbol such as ::xyzzy designates a spec or not?

borkdude 2020-11-03T15:34:44.168600Z

user=> (s/def ::foo int?)
:user/foo
user=> (s/spec? ::foo)
nil
user=> (s/spec? (s/get-spec ::foo))
#object[clojure.spec.alpha$spec_impl$reify__2059 0x4d48bd85 "clojure.spec.alpha$spec_impl$reify__2059@4d48bd85"]

borkdude 2020-11-03T15:35:30.169Z

So get-spec returns the spec for a keyword or a symbol (in case of fdef)

borkdude 2020-11-03T15:38:08.169400Z

@jimka.issy There is also s/spec:

user=> (s/spec ::foox)
Execution error at user/eval162 (REPL:1).
Unable to resolve spec: :user/foox
user=> (s/spec ::foo)
#object[clojure.spec.alpha$spec_impl$reify__2059 0x4d48bd85 "clojure.spec.alpha$spec_impl$reify__2059@4d48bd85"]

borkdude 2020-11-03T15:38:48.170Z

fwiw, I don't use these functions a lot, except maybe when I'm writing tools around spec. Not in normal daily spec usage

borkdude 2020-11-03T15:39:25.170900Z

s/def, s/valid? and s/conform are probably the most common ones I use

2020-11-03T15:49:04.171Z

almost the same for that particular case

(require '[matchete.core :as mc])

(def data
  '{:reify reify
    :clauses {:clause {:sym clojure.lang.IDeref
                       :lists [(deref [_] (deref-future fut))]}
              :clauses [{:sym clojure.lang.IBlockingDeref
                         :lists [(deref [_ timeout-ms timeout-val] (deref-future fut timeout-ms timeout-val))]}
                        {:sym clojure.lang.IPending
                         :lists [(isRealized [_] (.isDone fut))]}
                        {:sym java.util.concurrent.Future
                         :lists [(get [_] (.get fut)) (get [_ timeout unit] (.get fut timeout unit)) (isCancelled [_] (.isCancelled fut)) (isDone [_] (.isDone fut)) (cancel [_ interrupt?] (.cancel fut interrupt?))]}]}})

(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/scan {:sym '!interface})}})

(mc/matches pattern data)

borkdude 2020-11-03T15:57:28.171200Z

This returns:

({!interface [clojure.lang.IDeref clojure.lang.IBlockingDeref]} {!interface [clojure.lang.IDeref clojure.lang.IPending]} {!interface [clojure.lang.IDeref java.util.concurrent.Future]})

borkdude 2020-11-03T15:57:40.171400Z

Is there a way to get a list of only the symbols like with meander?

2020-11-03T16:00:01.171600Z

sorry, I was too fast copypasting ) instead of mc/scan please use mc/each

2020-11-03T16:00:43.172100Z

scan is for inspecting items in collection individually (in separate decision branches) each is for walking over a collection

2020-11-03T16:01:03.172600Z

(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/each {:sym '!interface})}})
this pattern should work

Jim Newton 2020-11-03T16:01:18.173Z

After something like the following:

(s/def ::big-even (s/and int? even? #(> % 1000)))
can I call some function on ::big-even to get back (s/and int? even? #(> % 1000) ?

borkdude 2020-11-03T16:01:27.173100Z

I get:

No such var: mc/each

2020-11-03T16:04:14.173300Z

you are using experimental 2.0.0 version check out 1.2.0

borkdude 2020-11-03T16:05:24.173500Z

no, I was using the version that is suggested in the README

$ clj  -Sdeps '{:deps {io.xapix/matchete {:mvn/version "1.1.0"}}}'
:)

2020-11-03T16:05:27.173700Z

argh, cljdoc badge got updated for master branch as well,

2020-11-03T16:06:19.173900Z

then this is my bad, I should update it there

2020-11-03T16:06:21.174100Z

sorry

borkdude 2020-11-03T16:06:34.174400Z

almost:

user=> (s/def ::big-even (s/and int? even? #(> % 1000)))
:user/big-even
user=> (s/form ::big-even)
(clojure.spec.alpha/and clojure.core/int? clojure.core/even? (clojure.core/fn [%] (clojure.core/> % 1000)))

borkdude 2020-11-03T16:07:11.174500Z

no problem!

alexmiller 2020-11-03T16:09:29.175400Z

s/describe returns a shorter form, useful for printing (but not fully resolved so not too useful as data)

Jim Newton 2020-11-03T16:10:38.176300Z

seems like s/form does what I want. But it's not certain.

alexmiller 2020-11-03T16:11:03.176700Z

research question - if you are using specs with aliases, do you tend to use one alias, a few, or many in any given namespace?

borkdude 2020-11-03T16:12:18.177600Z

what do you mean by an alias?

borkdude 2020-11-03T16:12:41.178Z

like (alias 'foo 'bar) and then ::bar/spec ?

borkdude 2020-11-03T16:13:05.178300Z

@alexmiller I think grasp could be used to find this out

2020-11-03T16:13:39.178700Z

one reason for me to use many aliases would be if many keys in a s/keys require a same generator. For one, we have a present string spec, with corresponding generator that we alias a lot

borkdude 2020-11-03T16:14:17.179200Z

hmm, never mind, grasp returns the fully qualified keyword, so you can't see if it was aliased or not

alexmiller 2020-11-03T16:15:21.179600Z

@borkdude yes, that's what I mean

alexmiller 2020-11-03T16:16:38.180200Z

https://grep.app/search?q=%28alias%20%27&filter[lang][0]=Clojure gets me some idea at least

alexmiller 2020-11-03T16:17:04.180800Z

but really wondering what people are doing in private code bases

borkdude 2020-11-03T16:17:31.181200Z

we usually don't use alias for this, since a lot of our specs are in CLJS. We create namespaces on disk for this

2020-11-03T16:17:34.181300Z

ah nevermind then, I interpreted this as an alias:

(s/def ::my-spec ...)
(s/def ::key-2 ::my-spec)
(s/def ::key-1 ::my-spec)
(s/def ::map (s/keys :req [::key-2 ::key-2]))

borkdude 2020-11-03T16:21:08.181700Z

@alexmiller typically we have this in our backend, using on disk namespaces:

(s/def ::system
  (s/merge ::dict/system
           ::adis/system
           ::annotator/system
           (s/keys :req [::database/db ::cache/cache])))

borkdude 2020-11-03T16:55:56.181900Z

Excellent, it works now!

user=> (def conformed (map #(s/conform ::reify %) matches))
#'user/conformed
(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/each {:sym '!interface})}})
#'user/pattern
user=> (mc/matches pattern (first conformed))
({!interface [clojure.lang.IDeref clojure.lang.IBlockingDeref clojure.lang.IPending java.util.concurrent.Future]})

borkdude 2020-11-03T17:05:18.182100Z

@delaguardo https://github.com/borkdude/grasp#matchete

borkdude 2020-11-03T17:18:59.182500Z

Nice, it seems your lib also works with babashka:

$ bb -cp "$(clojure -Spath)" -e "(require '[matchete.core :as mc]) (mc/matches '{:x ?x} {:x 1})"
({?x 1})

Jim Newton 2020-11-03T17:23:57.185Z

More spec questions. When I define ::big-even as in https://clojure.org/guides/spec.

(s/def ::big-even (s/and int? even? #(> % 1000)))
I'm guessing that the function #(> % 1000) is a closure in its current lexical context. Right? So later when I call s/form I don't get back that object, but some new text, which doesn't know the original lexical context. Right?
(s/form ::big-even)
 ==> (clojure.spec.alpha/and
 clojure.core/int?
 clojure.core/even?
 (clojure.core/fn [%] (clojure.core/> % 1000)))

borkdude 2020-11-03T17:26:38.185700Z

@jimka.issy The text #(...) is not representable as an s-expression, it's something built into the reader which expands to (fn [...] ...)

borkdude 2020-11-03T17:27:23.186100Z

What simply happens is that s/def is a macro which captures the input and by that time the s-expression has already expanded to the fn form

alexmiller 2020-11-03T17:36:14.188200Z

the spec function will capture the lexically scoped values at the point of definition (but the form will not - it will have the names)

alexmiller 2020-11-03T17:36:32.188500Z

user=> (let [x 100] (s/def ::foo (s/and int? #(= % x))))
:user/foo
user=> (s/valid? ::foo 100)
true
user=> (s/valid? ::foo 99)
false
user=> (s/form ::foo)
(clojure.spec.alpha/and clojure.core/int? (clojure.core/fn [%] (clojure.core/= % x)))

alexmiller 2020-11-03T17:36:54.188700Z

^^ form is broken there

alexmiller 2020-11-03T17:42:05.190400Z

this is all somewhat more consistent in spec 2 (namely, this will not work at all as the divide between symbolic specs and spec objects is clearer) but there is built-in support for creating parameterized specs (like the one above) if needed

Jim Newton 2020-11-04T08:51:46.193800Z

@alexmiller, thanks for the explanation. however, i'd like to get the actual function object which spec would use to validate with. For example given something like (s/and int? #= % x)), I can use int? to get the int? function, but I can't use the s-expression form of the closure to recuperate the closure. However, I'm pretty sure spec has the closure somewhere, otherwise s/valid? would not work.

Jim Newton 2020-11-04T08:58:52.194Z

What I'm trying to do is figure out which spec incantations can be expressed in a simple type system which I have implemented, called genus , normally abbreviated gns/. For example, a naïve first attempt is able to translation ::big-even as follows

clojure-rte.rte-core> (require '[clojure.spec.alpha :as s])
nil

clojure-rte.rte-core> (s/def ::big-even (s/and int? even? #(> % 1000)))
:clojure-rte.rte-core/big-even

clojure-rte.rte-core> (s/get-spec ::big-even)
#object[clojure.spec.alpha$and_spec_impl$reify__2183 0x61966631 "clojure.spec.alpha$and_spec_impl$reify__2183@61966631"]

clojure-rte.rte-core> (s/form ::big-even)
(clojure.spec.alpha/and
 clojure.core/int?
 clojure.core/even?
 (clojure.core/fn [%] (clojure.core/> % 1000)))
The important result being the next step:
clojure-rte.rte-core> (gns/canonicalize-type '(spec ::big-even))
(and
 (or Long Integer Short Byte)
 (satisfies clojure.core/even?)
 (spec (clojure.core/fn [%] (clojure.core/> % 1000))))
So genus can unwind the spec into types and predicates, if the predicates are symbolic. But what's left (clojure.core/fn [%] (clojure.core/> % 1000)) is not something which is useable.

Jim Newton 2020-11-04T09:00:26.194200Z

I could try to write a translator/compiler for expressions such as (clojure.core/fn [%] (clojure.core/> % 1000)) which try to convert them back to a closure, in the case that they don't have any free variables. But that work seems redundant since spec already has the closure in its internal data structure.

Jim Newton 2020-11-04T09:09:04.194600Z

I just realized something interesting. If I eval the list (clojure.core/fn [%] (clojure.core/> % 1000)) then I get a function object which seems to have the same semantics as the original function, in case there are no free variables.

clojure-rte.rte-core> ((eval '(clojure.core/fn [%] (clojure.core/> % 1000)))  12)
false
clojure-rte.rte-core> ((eval '(clojure.core/fn [%] (clojure.core/> % 1000)))  100000)
true
clojure-rte.rte-core> 
Perhaps that is good enough for my proof-of-concept.

Jim Newton 2020-11-04T09:10:56.194800Z

And if there's a free variable, eval throws an exception

clojure-rte.rte-core> (eval '(clojure.core/fn [%] (clojure.core/> x 1000)))
Syntax error compiling at (clojure-rte:localhost:51477(clj)*:45:51).
Unable to resolve symbol: x in this context

alexmiller 2020-11-04T14:11:53.274Z

Well eval is literally the function that takes a form and returns a function

alexmiller 2020-11-03T17:42:07.190700Z

user=> (s/defop narrow-int [x] (s/and int? #(= % x)))
#'user/narrow-int
user=> (s/def ::foo (narrow-int 100))
:user/foo
user=> (s/form ::foo)
(user/narrow-int 100)
user=> (s/valid? ::foo 100)
true
user=> (s/valid? ::foo 99)
false

2020-11-03T17:50:35.190800Z

🎆

2020-11-03T17:51:27.191Z

another pusher to polish documentation 😉