Clojurians Log v2

I use lein-test-refresh, and I want to have my functions instrumented during these tests, with expound turned on. My approach has been using profiles.clj:

{:user {:plugins      [[venantius/ultra "0.6.0"]
                       [com.jakemccrary/lein-test-refresh "0.24.1"]]
        :dependencies [[expound "0.7.2"]]
        :injections
        [(require '[expound.alpha :as expound]
                  '[clojure.tools.namespace.repl :refer :all]
                  'clojure.spec.alpha
                  'clojure.spec.test.alpha
                  'spell-spec.expound)
         (alter-var-root #'clojure.spec.alpha/*explain-out*
                         (constantly #(expound/custom-printer
                                       {:show-valid-values? true
                                        :theme              :figwheel-theme})))
         (refresh)
         (clojure.spec.test.alpha/instrument)]}}

@alex.joseph.whitt Unfortunately, I’m away from my personal computer right now, so I can’t test it. Can you create a test that prints out the value of ‘explain-out’? I wonder if you need to use ‘set!’ instead of alter-var-root?

I’m not sure exactly when injections run, but if they run before Clojure sets up the default bindings, including explain-out, I think they would be incompatible with how spec expects you to configure the printer :(

If they run after the default bindings, then using ‘set!’ would alter the binding whereas ‘alter-vat-root’ won’t work as you want.

https://github.com/bhb/expound/blob/master/doc/faq.md

Hmm, *explain-out* is seemingly correct

Testing clj-canopen.cob-test
"*explain-out*:" #object[expound.alpha$custom_printer$fn__2814 0x4af9217f "expound.alpha$
custom_printer$fn__2814@4af9217f"]                                                      

ERROR in (header-codec-tests) (alpha.clj:132)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: Call to #'vertiv-common.binary.codec.complex-bytes/
encode did not conform to spec.
{:clojure.spec.alpha/problems....                                          

Interesting. So if you just, say, call ‘s/explain’ in a test (when it should fail), presumably you’ll see expound output?

If you just call ‘instrument’ as part of your test (or in ‘:each’ fixture), does it work?

Yep to your first question:

(deftest header-codec-tests
  (prn "*explain-out*:" s/*explain-out*)
  (s/explain int? :hi)
...)

Testing clj-canopen.cob-test
"*explain-out*:" #object[expound.alpha$custom_printer$fn__2814 0x4af9217f "expound.alpha$
custom_printer$fn__2814@4af9217f"]                                                      
-- Spec failed --------------------

  :hi

should satisfy

  int?

No to your second question

Ah, so I wonder if the issue is that “lein” is just not processing errors using explain-out

My memory is that maybe newer versions of lein do this? But I’m not 100% sure

Basically old versions of Clojure used to put the entire error string in the exception. Newer versions don’t: they just include the raw data, and expect the environment to format it correctly.

For instance, if you did the same experiment of calling “instrument” in your tests and ran them with recent version of “CLJ”, I wonder if you’d see a different output

Well, I'm on Leiningen 2.9.1 (newest) and Clojure 1.10.0. I suppose I could try clojure CLI, although I've never used it before

Hm, maybe lein didn’t do that? I thought they did but I could be mistaken.

Oh I think also newer Clojure 1.10.1-beta3 may have changed error handling in non-REPL environments

You could try that as an experiment.

I tried 1.10.1-beta3... didn't hurt anything but it didn't change the behavior 😕

Out of curiosity, what's your workflow like? I'm not married to lein-test-refresh, but I've tried several different ways of auto-testing my code and it seems to be the most performant

I use lein-test-refresh as well, but IIRC, I just set up expound in a fixture. Seems to work for me, but I can’t pinpoint what’s different off the top of my head. My test setup is in the expound source. Another point of complexity is that orchestra does something different here than vanilla instrument

For now I'm just using the regular instrument

I use orchestra (which has different bugs as you found above!) but I may be relying on that behavior.

Oh I see what you mean

There’s a lot of complex interaction right now unfortunately

Sadly expound can only work within the confines of tools like lein and Clojure CLI

Yeah. What I wouldn't give for all the niceties we want baked into Clojure / spec / clojure.test natively. We have excellent libraries like expound and such but getting to a nice workflow is not a great user story.

Since ‘explain-out’ is set correctly, my guess is that lein is not formatting exceptions using ‘explain-out’. Off the top of my head, this seems like something that would need to be addressed in lein

Agree! It’s a big pain and not obvious how to get it set up (And in some cases, not quite possible to do what we want!)

Ah, I think "orchestra" was the magic word

Short of a lein fix, you may be able to wrap your tests in a fixture that catches exceptions and calls expound w the data, but I realize that’s not optimal

Nice!

Only problem seems to be that it prints out the clojure.spec.alpha/problems in addition to the expound-formatted report:

........
-------------------------
Detected 2 errors

{:clojure.spec.alpha/problems ......

I think I asked you about that last week or something but I can't see the history.

IIRC, that’s still related to how lein handles exceptions. Clojure CLI may have different behavior.

Lein is getting the exception and printing out the message and the data AIUI

Not sure if this is configurable in lein

Part of me wants to migrate to Clojure CLI, because I like things that smell "standard," but I'm afraid of losing everything lein buys me. I'm not sure if the simple/easy tradeoff is worth it yet.

I hear ya. I haven’t moved away from lein for that reason, but it is more painful right now since it delayed in copying behavior changes in ‘clj’

Not sure if CLJ CLI has a robust test refresh solution yet

I've been looking and not turning up anything

But anyway, I think our environments are fairly similar at this point. Do you also get the problems printed out after the expound report when using lein-test-refresh?

It's more scrolling but things are livable now at least

I’d have to check when I’m back on my non-work computer, but my guess is yes, I do

Mmk. Well, I can deal. This is already 10,000% better. Thank you again for your help! It's authors like you that keep my faith in Clojure and OSS itself alive.

That’s nice of you to say! Happy to help, and thanks for using expound!