testing

Testing tools, testing philosophy & methodology...
onetom 2020-03-23T15:06:56.009200Z

@seancorfield is this supposed to work in expectations.clojure.test?

(deftest submap-test
  (expect (more {:a 1} {:b 2})
          (in {:a 1 :b 2 :c 3})))
because I'm getting an error with both 1.2.1 & 8f46193b2726e1366e78d60f9baf62d9416d122a:
ERROR in (submap-test) (xxx.clj:18)
Uncaught exception, not in assertion.
expected: nil
  actual: java.lang.IllegalArgumentException: more should only be used inside expect
but more is clearly within expect. sounds like it's been evaluated in the wrong context, but I can't follow the expect macro; it's a bit too complicated for me

onetom 2020-03-23T15:08:11.010500Z

the reason im trying to do it this way, is that i was hoping to get a shorter error message when the test fails and i wouldn't need to rely on some nice diffing extension to clojure.test to pin-point the error

seancorfield 2020-03-23T15:17:23.010900Z

@onetom No, you can't use more/`in` like that.

seancorfield 2020-03-23T15:18:03.011600Z

more is intended to be used with a series of predicates that can be "expected" of the actual value.

seancorfield 2020-03-23T15:19:03.012400Z

You could (expect {:a 1 :b 2} (in (some-expression))) -- that would make sense.

onetom 2020-03-23T16:01:53.015Z

The "foobar" in this example (from the README) totally suggests that I can just use any of the basic "expected" value types:

(expecting "string behavior"
    (expect (more #"foo" "foobar" #(clojure.string/starts-with? % "f"))
            (str "f" "oobar"))
then the following line from the "Compatibility with Expectations" section of the README also suggests this:
(expect (more e1 e2 ...) actual-expr)
I looked at the source code:
(and (sequential? e) (= 'more (first e)))
      (let [es (mapv (fn [e] `(expect ~e ~a ~msg ~ex? ~e')) (rest e))]
        `(do ~@es))
and even that suggested me that it should work, since it just transforms the elements in the more form into multiple expect calls.

seancorfield 2020-03-23T16:03:23.015700Z

But in does its transformation first and that's what prevents the above from working.

seancorfield 2020-03-23T16:04:00.016500Z

I've taken a couple of runs at it now and I can't get all cases to work with any of the rewrites I've done so far.

onetom 2020-03-23T16:05:33.017900Z

Okay, thanks. I will try to internalize how it works 🙂 I'm just asking these questions because I will have to teach this to my colleagues eventually and I want to be ready if they ask me why is it like this.

seancorfield 2020-03-23T16:05:46.018300Z

Classic Expectations doesn't support that usage either, BTW:

failure in (NO_SOURCE_PATH:1) : user
(expect (more {:a 1} {:b 2}) (in {:a 1, :b 2, :c 3}))

           Using both 'in with a map and 'more is not supported.

onetom 2020-03-23T16:07:34.020100Z

To be honest, the reason I was thinking about splitting up the map testing is because Cursive doesn't show me a proper diff when I use expect. If I just use (is (= {expected map,,,} {actual map,,,})) , then I can get a nice diff, but that doesn't allow extra keys in the actual map... 😕

onetom 2020-03-23T16:08:19.020800Z

That's a nice error message though; would worth porting it over!

seancorfield 2020-03-23T16:09:22.022Z

Yeah, I've done quite a bit of work on (new) Expectations to make it work with Cursive as best I can, but there are several corner cases that are a bit harder to make work properly (and I don't use Cursive so it's mostly been guided by Colin himself on that).

onetom 2020-03-23T16:10:28.023600Z

I've actually tried to use kaocha's REPL support with cursive and it's not bad, just not very integrated with Cursive...

seancorfield 2020-03-23T16:10:45.024Z

Have you tried using Paul Stadig's Humane Test Output? That helps a lot in some cases (Expectations has built-in support for it -- you just need to add it to your classpath and Expectations enables it automatically).

onetom 2020-03-23T16:11:05.024300Z

sure, but it's not compatible with Cursive

seancorfield 2020-03-23T16:11:46.025200Z

Ah, because Cursive messes with the clojure.test reporting machinery I suspect?

onetom 2020-03-23T16:11:54.025500Z

Cursive even prints an error message that it has detected the presence of humane output

onetom 2020-03-23T16:12:26.026600Z

yes, it does, so it can put gutter icons next to failing tests and can underline the failing case, which i find really helpful

seancorfield 2020-03-23T16:12:50.027300Z

Leiningen also tries to "help" by injecting its own stuff into clojure.test's reporting machinery. It's very hard to do any sort of custom reporting and still work with both Cursive and Leinningen.

onetom 2020-03-23T16:14:29.028700Z

But my plan B is to learn CIDER slowly 🙂 I'm avoiding Leiningen. Too much extra complexity and I'm also very impatient when it comes to REPL startup time. Plus it's harder to teach, when I have to admit eventually that there is a new, official, leaner way to do most of what lein does...

seancorfield 2020-03-23T16:15:55.030100Z

One of the things I like about Atom/Chlorine is the minimal overhead of tooling. No nREPL, no messing with clojure.test reporting. Just a straight up Socket REPL in any Clojure process and you're off to the races. It even does auto-complete without needing dependencies in your code.

onetom 2020-03-23T16:16:49.030800Z

I use the execution interruption feature quite a lot though and that doesn't work with a socket REPL.

onetom 2020-03-23T16:17:35.031500Z

I also used some unicode function names, like • or ∆, which is also not supported over socket REPL

onetom 2020-03-23T16:19:27.032700Z

But because of your tweets, blog articles and slack messages, I did look into Atom and Chlorine, so thanks a lot for those efforts!

onetom 2020-03-23T16:20:39.033900Z

I'm quite open to any tooling, but I'm trying to build a team and the current ppl are quite insistent on vim or spacemacs, which made me even try ideavim + intellimacs with intellij 🙂

onetom 2020-03-23T16:21:08.034300Z

Are you using some solution to avoid accidentally printing too long data?

seancorfield 2020-03-23T17:00:47.035700Z

Chlorine does some stuff with unrepl under the hood to deal with that. And provide execution interruption. I don't find I run into either of those issues in my day-to-day work anyway.

👌 1