ring-swagger & compojure-api
rbarker 2018-08-15T04:28:55.000100Z

I’m getting errors using compojure-api and therefore ring-swagger with spec coercion enabled - the exception says that there is no spec defined for a keyword ‘Unable to resolve spec’ - however I am simply checking for the presence of a keyword in the response map. Any ideas?

ikitommi 2018-08-15T06:35:00.000100Z

@stathissideris we have multiple projects on reitit (and c-api), but nothing big in prod yet - the oldest projects have started last year, haven’t had any rough corners in a while. The routing core is quite stable and proven, ring-side has still some moving parts, and it’s not on par with c-api on all the features. I would say it’s alpha, but so is everything nowadays 😉

ikitommi 2018-08-15T06:35:20.000100Z

for the muuntaja: https://github.com/metosin/muuntaja/pull/77

ikitommi 2018-08-15T06:35:56.000100Z

@rbarker could you show some minimalistic code?

rbarker 2018-08-15T06:37:53.000100Z

(GET "foo" []
        :produces ["application/json"]
        :summary "foobar"
        :return (s/get-spec ::foo)
(s/def ::foo (s/keys [::bar]))

rbarker 2018-08-15T06:38:11.000200Z

This would cause an exception saying that spec ‘bar’ cannot be found

ikitommi 2018-08-15T06:39:34.000100Z

does :return ::foo work?

rbarker 2018-08-15T06:39:52.000100Z

I get the error when I try to load the swagger docs

ikitommi 2018-08-15T06:40:03.000100Z

is the ::bar defined?

rbarker 2018-08-15T06:40:08.000100Z


rbarker 2018-08-15T06:40:38.000100Z

only by creating a spec (s/def ::bar some?) do the swagger docs load

rbarker 2018-08-15T06:41:04.000100Z

my understanding is that if a spec is not defined then simply the presence of the key/value is asserted

ikitommi 2018-08-15T06:41:23.000100Z

ok, I thought it would fail too.

ikitommi 2018-08-15T06:42:34.000100Z

the code to walk over the specs is here: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/visitor.cljc

ikitommi 2018-08-15T06:43:40.000100Z

That should be changed so that if the key is not defined, return nil and handle that gracefully.

ikitommi 2018-08-15T06:44:00.000100Z

PR for that welcome.

Empperi 2018-08-15T10:38:17.000100Z

I think I found a bug in spec-tools or in compojure-api using it... Not sure though what is causing it currently

Empperi 2018-08-15T10:41:24.000100Z

It's a weird one. Everything was working fine until I added nil as a possible value to one of the spec definitions via (s/or ....). After that I get:

my-appl.api/fn             api.clj:   94
                               compojure.api.coercion/coerce-request!        coercion.clj:   45
              compojure.api.coercion.spec.SpecCoercion/coerce-request            spec.clj:  125
                                            clojure.spec.alpha/unform           alpha.clj:  157
                                         spec-tools.core.Spec/unform*           core.cljc:  233
                                            clojure.spec.alpha/unform           alpha.clj:  157
                     clojure.spec.alpha/regex-spec-impl/reify/unform*           alpha.clj: 1658
                                         clojure.spec.alpha/op-unform           alpha.clj: 1449
                                                  clojure.core/mapcat            core.clj: 2775 (repeats 2 times)
                                                   clojure.core/apply            core.clj:  652
                                                     clojure.core/seq            core.clj:  137
                                                  clojure.core/map/fn            core.clj: 2745
                                      clojure.spec.alpha/op-unform/fn           alpha.clj: 1449
                                         clojure.spec.alpha/op-unform           alpha.clj: 1446
                                            clojure.spec.alpha/unform           alpha.clj:  157
                       clojure.spec.alpha/map-spec-impl/reify/unform*           alpha.clj:  808
                                            clojure.spec.alpha/unform           alpha.clj:  157
                                         spec-tools.core.Spec/unform*           core.cljc:  233
                                            clojure.spec.alpha/unform           alpha.clj:  157
                          clojure.spec.alpha/every-impl/reify/unform*           alpha.clj: 1274
                          clojure.spec.alpha/tuple-impl/reify/unform*           alpha.clj:  976
                                            clojure.spec.alpha/unform           alpha.clj:  157
                                         spec-tools.core.Spec/unform*           core.cljc:  233
                                            clojure.spec.alpha/unform           alpha.clj:  157
                        clojure.spec.alpha/or-spec-impl/reify/unform*           alpha.clj: 1046
java.lang.UnsupportedOperationException: nth not supported on this type: Keyword

Empperi 2018-08-15T10:41:55.000100Z

This happens when a request comes in and compojure-api tries to coerce the data

Empperi 2018-08-15T10:43:04.000100Z

if I remove the compojure-api form the picture and try to validate the input data against the same schema definitions via (s/explain ...) it doesn't break like that but it does report bunch of problems which are due to the fact that coercions have not been ran

ikitommi 2018-08-15T10:43:54.000100Z

could you try using spec-tools.core/decode?

Empperi 2018-08-15T10:44:02.000100Z

good idea, one moment

Empperi 2018-08-15T10:45:08.000100Z

returns :clojure.spec.alpha/invalid but doesn't explode like that

ikitommi 2018-08-15T10:45:25.000100Z

btw, go vote up https://dev.clojure.org/jira/browse/CLJ-2251 to get the native coercion into clojure.spec. there is also CLJ-2116 but “not likely to be accepted”

Empperi 2018-08-15T10:47:44.000100Z

yeah, would be nice

ikitommi 2018-08-15T10:48:07.000100Z

hmm.. it seems that c-api is not using the st/decode yet, but https://github.com/metosin/compojure-api/blob/master/src/compojure/api/coercion/spec.clj#L119-L125 instead.

Empperi 2018-08-15T10:49:28.000100Z

calling conform doesn't break it either, let me replicate that logic and see if I get to duplicate the problem

ikitommi 2018-08-15T10:49:30.000100Z

the s/invalid? doesn’t work for some reason? shoudn’t be…

mgrbyte 2018-08-15T10:49:46.000100Z

think you need to add s/nilable around your spec

ikitommi 2018-08-15T10:51:04.000100Z

if you copy-paste the code i highlighted and run it locally with the data, does it blow up?

ikitommi 2018-08-15T10:52:03.000100Z

and yes, s/nilable is the way to go, but still shoudn’t blow up on error.

Empperi 2018-08-15T10:53:20.000100Z

weirdly no, it does not blow up

Empperi 2018-08-15T10:54:07.000100Z

but I do not pass transformer in my code snippet

Empperi 2018-08-15T10:54:34.000100Z

copy-pasting that stuff would require copying a lot of other stuff also

Empperi 2018-08-15T10:55:06.000100Z

but let me try if s/nilable would allow me to go past this problem at this point, that being said this bug should be duplicated with a minimal reproduction case

Empperi 2018-08-15T10:55:28.000100Z

this is way too complex as it is now to work as an example case plus I cannot put this anywhere publicly

Empperi 2018-08-15T10:56:39.000200Z

right, changing from s/or to s/nilable bypassed the problem

Empperi 2018-08-15T10:57:13.000100Z

which is doubly weird

Empperi 2018-08-15T11:04:40.000100Z

anyway, I'll try to reproduce this later with a more simple example. Thanks for the ideas which allowed me to bypass this problem this time

Empperi 2018-08-15T11:05:05.000100Z

and naturally if I do manage to reproduce it I'll post an issue to github

bja 2018-08-15T13:41:10.000100Z

is there a good way to turn muuntaja off for a request body?

bja 2018-08-15T13:46:35.000100Z

actually, this is the wrong question. I want to add a new format that is simply ignored by muuntaja

ikitommi 2018-08-15T13:46:41.000100Z

@bja just for somr routes? Sadly no. The middleware are applied in place, so you would need to disable it at api and re-mount the mw for some routez

bja 2018-08-15T13:47:28.000200Z

basically, I want to mount an application/csv that just slurps the request body (or passes it through), but I'd still like muuntaja to handle the response negotiation via Accept

ikitommi 2018-08-15T13:48:19.000100Z

It should work out by default?

bja 2018-08-15T13:48:26.000100Z

I thought (assoc-in muuntaja/default-options [:formats "application/csv"] {:decoder [slurp]} was close

bja 2018-08-15T13:48:59.000100Z

ooh, maybe I did something else that is trying to cause coercion

bja 2018-08-15T13:50:09.000100Z

worse, I feel like I figured this out with my old codebase (such that it was documented properly in csv), but I don't have access to said codebase anymore

bja 2018-08-15T14:09:03.000100Z

I suppose, alternately, can I add per-endpoint custom swagger documentation?

ikitommi 2018-08-15T14:10:51.000100Z

I think the slurp is not right. It should be a function returning the decoder.

bja 2018-08-15T14:10:52.000100Z

oh, found it here: https://github.com/metosin/compojure-api/wiki/Swagger-integration

ikitommi 2018-08-15T14:11:14.000100Z

yep, via :swagger

bja 2018-08-15T17:20:56.000100Z

is there a way to encode that the body should be a CSV string in Swagger?

bja 2018-08-15T17:21:42.000100Z

looking at the OpenAPI spec, it's not immediately obvious that this is supported for the body, although it seems to be an opt for individual parameters

ikitommi 2018-08-15T17:31:41.000100Z

OpenAPI has better options for body, don't think Swagger2 supports those. Setting`:consumes` might help the ui. OpenAPI3-support PRs welcome...

ikitommi 2018-08-15T17:32:23.000100Z

Spec-tools is quite close, schema-tools would require some refactoring.