expound

Charlie Briggs 2020-08-17T15:11:18.024100Z

I actually ran into the above error again using orchestra/expound so I guess it wasn’t entirely user error

Charlie Briggs 2020-08-17T15:11:18.024200Z

java.lang.IllegalArgumentException: No matching clause: :clojure.spec.alpha/nil
	at expound.alpha$show_spec_name.invokeStatic(alpha.cljc:168)
	at expound.alpha$show_spec_name.invoke(alpha.cljc:165)
	at expound.alpha$eval18213$fn__18214.invoke(alpha.cljc:314)
	at clojure.lang.MultiFn.invoke(MultiFn.java:261)
	at expound.alpha$format_err.invokeStatic(alpha.cljc:335)
	at expound.alpha$format_err.invoke(alpha.cljc:331)
	at expound.alpha$eval18493$fn__18494.invoke(alpha.cljc:802)
	at clojure.lang.MultiFn.invoke(MultiFn.java:261)
	at expound.alpha$print_explain_data$iter__18500__18504$fn__18505$fn__18506.invoke(alpha.cljc:850)
	at expound.alpha$print_explain_data$iter__18500__18504$fn__18505.invoke(alpha.cljc:848)
	at clojure.lang.LazySeq.sval(LazySeq.java:42)
	at clojure.lang.LazySeq.seq(LazySeq.java:51)
	at clojure.lang.RT.seq(RT.java:535)
	at clojure.core$seq__5402.invokeStatic(core.clj:137)
	at clojure.core$apply.invokeStatic(core.clj:660)
	at clojure.core$apply.invoke(core.clj:660)
	at expound.alpha$print_explain_data.invokeStatic(alpha.cljc:847)
	at expound.alpha$print_explain_data.invoke(alpha.cljc:833)
	at expound.alpha$printer_str.invokeStatic(alpha.cljc:988)
	at expound.alpha$printer_str.invoke(alpha.cljc:970)
	at expound.alpha$custom_printer$fn__18569.invoke(alpha.cljc:1028)
	at org_api.middleware$wrap_exceptions$fn__28170.invoke(middleware.clj:53)
	at org_api.rollbar$wrap_exceptions$fn__23420.invoke(rollbar.clj:19)
	at org_api.middleware$wrap_unexpected_exceptions$fn__28180.invoke(middleware.clj:81)
	at ring.middleware.json$wrap_json_response$fn__6667.invoke(json.clj:139)
	at ring.middleware.keyword_params$wrap_keyword_params$fn__5554.invoke(keyword_params.clj:53)
	at ring.middleware.params$wrap_params$fn__4136.invoke(params.clj:67)
	at ring.middleware.absolute_redirects$wrap_absolute_redirects$fn__6013.invoke(absolute_redirects.clj:47)
	at ring.middleware.content_type$wrap_content_type$fn__5961.invoke(content_type.clj:34)
	at ring.middleware.default_charset$wrap_default_charset$fn__5985.invoke(default_charset.clj:31)
	at ring.middleware.not_modified$wrap_not_modified$fn__5942.invoke(not_modified.clj:61)
	at org_api.logging$middleware$fn__17647.invoke(logging.clj:8)
	at org_api.middleware$wrap_no_cache_param$fn__28187.invoke(middleware.clj:105)
	at opentracing_clj.ring$wrap_opentracing$fn__16360.invoke(ring.clj:45)
	at org_api.context$wrap_bind_user_id$fn__16381.invoke(context.clj:20)
	at iapetos.collector.ring$run_instrumented.invokeStatic(ring.clj:127)
	at iapetos.collector.ring$run_instrumented.invoke(ring.clj:123)
	at iapetos.collector.ring$wrap_instrumentation$fn__16161.invoke(ring.clj:163)
	at compojure.core$pre_init$fn__3997$fn__4000.invoke(core.clj:338)
	at compojure.core$wrap_route_middleware$fn__3878.invoke(core.clj:127)
	at compojure.core$wrap_route_info$fn__3883.invoke(core.clj:137)
	at compojure.core$wrap_route_matches$fn__3887.invoke(core.clj:146)
	at compojure.core$routing$fn__3902.invoke(core.clj:185)
	at clojure.core$some.invokeStatic(core.clj:2701)
	at clojure.core$some.invoke(core.clj:2692)
	at compojure.core$routing.invokeStatic(core.clj:185)
	at compojure.core$routing.doInvoke(core.clj:182)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:660)
	at compojure.core$routes$fn__3906.invoke(core.clj:192)
	at compojure.core$routing$fn__3902.invoke(core.clj:185)
	at clojure.core$some.invokeStatic(core.clj:2701)
	at clojure.core$some.invoke(core.clj:2692)
	at compojure.core$routing.invokeStatic(core.clj:185)
	at compojure.core$routing.doInvoke(core.clj:182)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:660)
	at compojure.core$routes$fn__3906.invoke(core.clj:192)
	at compojure.core$make_context$handler__3974.invoke(core.clj:286)
	at compojure.core$make_context$fn__3978.invoke(core.clj:296)
	at compojure.core$routing$fn__3902.invoke(core.clj:185)
	at clojure.core$some.invokeStatic(core.clj:2701)
	at clojure.core$some.invoke(core.clj:2692)
	at compojure.core$routing.invokeStatic(core.clj:185)
	at compojure.core$routing.doInvoke(core.clj:182)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:660)
	at compojure.core$routes$fn__3906.invoke(core.clj:192)
	at compojure.core$routing$fn__3902.invoke(core.clj:185)
	at clojure.core$some.invokeStatic(core.clj:2701)
	at clojure.core$some.invoke(core.clj:2692)
	at compojure.core$routing.invokeStatic(core.clj:185)
	at compojure.core$routing.doInvoke(core.clj:182)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:660)
	at compojure.core$routes$fn__3906.invoke(core.clj:192)
	at compojure.core$wrap_routes$fn__4007.invoke(core.clj:351)
	at compojure.core$routing$fn__3902.invoke(core.clj:185)
	at clojure.core$some.invokeStatic(core.clj:2701)
	at clojure.core$some.invoke(core.clj:2692)
	at compojure.core$routing.invokeStatic(core.clj:185)
	at compojure.core$routing.doInvoke(core.clj:182)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:660)
	at compojure.core$routes$fn__3906.invoke(core.clj:192)
	at iapetos.collector.ring$wrap_metrics_expose$fn__16170.invoke(ring.clj:184)
	at clojure.lang.Var.invoke(Var.java:384)
	at ring.middleware.refresh$wrap_with_script$fn__4214.invoke(refresh.clj:80)
	at compojure.core$routing$fn__3902.invoke(core.clj:185)
	at clojure.core$some.invokeStatic(core.clj:2701)
	at clojure.core$some.invoke(core.clj:2692)
	at compojure.core$routing.invokeStatic(core.clj:185)
	at compojure.core$routing.doInvoke(core.clj:182)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$apply.invoke(core.clj:660)
	at compojure.core$routes$fn__3906.invoke(core.clj:192)
	at ring.middleware.params$wrap_params$fn__4136.invoke(params.clj:67)
	at ring.middleware.reload$wrap_reload$fn__1837.invoke(reload.clj:39)
	at ring.middleware.stacktrace$wrap_stacktrace_log$fn__1209.invoke(stacktrace.clj:26)
	at ring.middleware.stacktrace$wrap_stacktrace_web$fn__1275.invoke(stacktrace.clj:98)
	at ring.adapter.jetty$proxy_handler$fn__484.invoke(jetty.clj:27)
	at ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown Source)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:834)
The explain-data being:
{:clojure.spec.alpha/problems ({:path [:clojure.spec.alpha/nil], :pred nil?, :val {:email-domains [], :updated-at #inst "2020-08-17T14:57:04.174369000-00:00", :single-sign-on-service-url "REDACTED", :stage "DISABLED", :x509-public-certificate ".", :external-id "REDACTED", :id #uuid "2e6a3bea-eae5-4c47-8331-e0094c5bf106", :created-at #inst "2020-08-17T14:14:05.456667000-00:00", :organisation-ids []}, :via [], :in []} {:path [:clojure.spec.alpha/pred :enabled-idp :x509-public-certificate], :pred (clojure.core/complement org-api.spec.identity-provider/not-base-64?), :val ".", :via [:org-api.spec.identity-provider/enabled-identity-provider :org-api.spec.identity-provider/x509-public-certificate], :in [:x509-public-certificate]} {:path [:clojure.spec.alpha/pred :disabled-idp :x509-public-certificate], :pred (clojure.core/complement org-api.spec.identity-provider/not-base-64?), :val ".", :via [:org-api.spec.identity-provider/disabled-identity-provider :org-api.spec.identity-provider/x509-public-certificate], :in [:x509-public-certificate]}), :clojure.spec.alpha/spec #object[clojure.spec.alpha$nilable_impl$reify__2556 0xdebc6ea "clojure.spec.alpha$nilable_impl$reify__2556@debc6ea"], :clojure.spec.alpha/value {:email-domains [], :updated-at #inst "2020-08-17T14:57:04.174369000-00:00", :single-sign-on-service-url "REDACTED", :stage "DISABLED", :x509-public-certificate ".", :external-id "test", :id #uuid "2e6a3bea-eae5-4c47-8331-e0094c5bf106", :created-at #inst "2020-08-17T14:14:05.456667000-00:00", :organisation-ids []}, :clojure.spec.alpha/fn org-api.get-identity-provider/get-by-id, :clojure.spec.alpha/ret {:email-domains [], :updated-at #inst "2020-08-17T14:57:04.174369000-00:00", :single-sign-on-service-url "REDACTED", :stage "DISABLED", :x509-public-certificate ".", :external-id "REDACTED", :id #uuid "2e6a3bea-eae5-4c47-8331-e0094c5bf106", :created-at #inst "2020-08-17T14:14:05.456667000-00:00", :organisation-ids []}, :clojure.spec.alpha/failure :instrument, :orchestra.spec.test/caller {:file "identity_provider.clj", :line 238, :var-scope org-api.identity-provider/update!}}

Charlie Briggs 2020-08-17T15:13:57.025300Z

that’s when removing the function spec for expound.alpha/value-in-context

bbrinck 2020-08-17T15:15:10.026100Z

@charliebriggs Does this only occur when using orchestra?

bbrinck 2020-08-17T15:15:43.027500Z

(It could still be an Expound bug, I just am trying to figure out if there’s some untested interaction here)

Charlie Briggs 2020-08-17T15:16:04.027700Z

Particularly it doesn’t seem to like this spec:

(defn not-base-64?
  "Returns true if a string is not base64 encoded.

   This does not mean the string _is correctly_ base 64 encoded."
  [s]
  (try
    (decode-base64 s)
    false
    (catch Exception _
      true)))
(s/def ::x509-public-certificate (s/and (complement string/blank?)
                                        (complement not-base-64?)))
I’ll check with/without orchestra now, just need to remember how to instrument without it!

Charlie Briggs 2020-08-17T15:20:47.030100Z

ah, this particular failure is checking the :ret value which clojure.spec.test.alpha/instrument doesn’t do

bbrinck 2020-08-17T15:21:03.030200Z

It could certainly be an Expound bug related to s/and - IIRC, that has been the source of Expound bugs in the past due to how it transforms values.

bbrinck 2020-08-17T15:21:42.031300Z

OK, cool, that’s helpful. It may relate to ret checking or we may be able to reproduce this with normal instrument if we created a modified function

Charlie Briggs 2020-08-17T15:24:17.033200Z

so I tried moving the same spec to an args fdef call and the error handler with expound worked as expected when using stest/instrument

bbrinck 2020-08-17T15:24:39.033600Z

Ah, interesting!

bbrinck 2020-08-17T15:25:11.034600Z

I wonder if there’s just an issue with how Expound interprets the explain-data created by Orchestra. I will look into it tonight

bbrinck 2020-08-17T15:25:37.035800Z

The only other thing that might work as a workaround (although my confidence is low) is to build a new compound predicate that doesn’t use s/and

Charlie Briggs 2020-08-17T15:26:26.037600Z

[orchestra "2020.07.12-1"]
[expound "0.8.5"]
versions at the mo’ I’m relatively new to spec and an and predicate seemed to be the only way of making this work in a sane way, I’ve just wrapped the expound in another try catch for now to get uglier spec errors if that fails, so no worries. Thanks again for looking

Charlie Briggs 2020-08-17T15:26:58.038600Z

I can try without the and if it would help you to understand this tho’

bbrinck 2020-08-17T15:27:06.038900Z

E.g. (s/def ::x509-public-certificate (fn [x] (and (not (string/blank? X)) (not (not-base-64? x))))

bbrinck 2020-08-17T15:27:20.039400Z

(Not tested at all, probably not balanced parens since I’m writing in slack)

bbrinck 2020-08-17T15:28:31.040800Z

Basically s/and can sometimes do weird things with conforming values. The above version is certainly less precise and may give worse messages, but may work. Again, my confidence is low here.

bbrinck 2020-08-17T15:28:44.041300Z

You could name that fn to something like valid-base-64 or something.

Charlie Briggs 2020-08-17T15:29:17.041900Z

ah, we’re using phrase at the minute to set error messages for each predicate on that check so that wouldn’t be ideal

bbrinck 2020-08-17T15:29:36.042100Z

Ah, gotcha

bbrinck 2020-08-17T15:30:04.042700Z

OK, then in the meantime, wrapping expound in try/catch is a sensible workaround

Charlie Briggs 2020-08-17T15:32:50.043Z

I tried removing the s/and and it still fails in the same way, just for further context

1👍
bbrinck 2020-08-17T15:40:19.044600Z

@charliebriggs Do you mind creating an issue on http://github.com/bhb/expound ? Feel free to pretty much copy/paste this discussion

bbrinck 2020-08-17T15:40:34.045300Z

My concern is that slack discussions go away and with time and I don’t want to lose details

Charlie Briggs 2020-08-17T15:40:50.045700Z

sure. I was originally going to just plonk one there but thought this might be easier to troubleshoot up front, makes sense to backfill that now

bbrinck 2020-08-17T15:40:57.045900Z

Thanks!

bbrinck 2020-08-17T15:41:21.046700Z

Yeah, it’s a good idea to discuss on slack first, but I always want to capture slack discussion for posterity :)

Charlie Briggs 2020-08-17T15:47:05.046900Z

https://github.com/bhb/expound/issues/199

1💯