testing

Testing tools, testing philosophy & methodology...
2021-04-15T13:38:48.030700Z

@nbtheduke I like that idea, and was thinking about that in some form… when you do that do you create a bunch of helper functions in the test namespace that help build that arg context? Do you do a bunch of “builder” functions (in the TDD world terminology) to keep the assertions simple…

NoahTheDuke 2021-04-15T13:39:48.032100Z

depends on the purpose of the helper functions!

2021-04-15T13:41:14.033500Z

some of the things I have seen is the building of up the args over a couple of lines of the let bindings as well…

(let [a  ...
      b  ... a ...
      c  .... a .. b...
      b'  ... a ...
      c'  .... a .. b'...]
  (is (= (form c) (form c'))))

2021-04-15T13:42:15.034700Z

it sometimes feels a bit imperative with all the let bindings from some of the other functional oriented languages I have done too

NoahTheDuke 2021-04-15T13:42:15.034900Z

from that example, it looks like you're doing the same thing twice, right?

2021-04-15T13:42:37.035200Z

various kind of building the input args

2021-04-15T13:43:07.035800Z

and it is likely a existing codebase thing in some cases and not a Clojure thing

2021-04-15T13:44:42.037400Z

I think it is more of a imperative use of let bindings more than anything

NoahTheDuke 2021-04-15T13:44:52.037800Z

i think that's okay

NoahTheDuke 2021-04-15T13:45:06.038300Z

if you have that kind of set of changes, a threading macro can make it look "nicer"

NoahTheDuke 2021-04-15T13:45:19.038700Z

if you're willing, real code is more helpful

2021-04-15T13:45:32.039Z

but being new, it feels like that the let bindings make it harder to eval a form in a REPL

NoahTheDuke 2021-04-15T13:46:10.039400Z

yeah, it can

NoahTheDuke 2021-04-15T13:47:06.040300Z

writing a helper function that performs a set of transformations for you can be helpful, which is what I think you were suggesting above

NoahTheDuke 2021-04-15T13:48:16.042200Z

my codebase has a lot of legacy cruft from years of non-clojure devs working on it, so there are 3 "test helper" files that have various functions that perform set-up and tear down and manage that stuff, and slowly massaging it all into pure functions has helped a lot

craftybones 2021-04-15T13:48:35.042900Z

@steven.proctor - in your let forms, are b and c and c’ just transforms of the result you wish to check? Have you used are ? That can potentially mitigate a few problems too

👍 1
2021-04-15T13:50:00.044300Z

i saw are, and that looked like it might solve some needs, I have done “ad-hoc” things like are by eaching over things and running tests in a block in Ruby and JS before

craftybones 2021-04-15T13:50:38.045600Z

are is quite handy in certain cases where your output requires some transformation in order to compare, especially if your output is a map

craftybones 2021-04-15T13:53:41.048200Z

For instance, you return a response with some optional headers that you aren’t interested in checking. There, writing an expected object that contains all the optional headers might be tedious and using are would help it like so

(let [res (handler req)]
  (t/are [x y] (= y (res x))
    :status 200
    :content-type....

👍 1
craftybones 2021-04-15T13:54:20.048600Z

of course, you could also use select-keys and is

2021-04-15T14:00:20.049300Z

dummy’ed out structure of a test I did that feels off…

2021-04-15T14:00:22.049600Z

(test/deftest feels-hard-to-eval-in-repl-tests
  (let [common-args          {...}
        args-case1           {...}
        args-case2           {...}
        error-args-case      {...}
        result-case1         (fn-under-test args-case1 common-args)
        result-case2         (fn-under-test args-case2 common-args)
        expected-form1       [:expected :things :here]
        expected-form2       [:other :things :here]]
    (test/testing "testing case 1"
      (test/is (= expected-form1 result-case1)))
    (test/testing "testing case 2"
      (test/is (= expected-form2 result-case2)))
    (test/testing "checking better error messages"
      (test/is (thrown-with-msg? ExceptionInfo
                                 "A nice error message"
                                 (fn-under-test error-args-case common-args)))
      )
    ))

2021-04-15T14:01:39.051100Z

And I can see how moving the result-case expressions and expected-form in the test/is would help

craftybones 2021-04-15T14:02:38.051900Z

I like hard coding the expected values as an argument to is

👍 1
craftybones 2021-04-15T14:03:18.052800Z

I like hard coding arguments too 🙂

2021-04-15T14:03:21.053300Z

but if trying to eval some of the (fn-under-test args-case1 common-args) in the REPL to get details on what it is doing still seems a bit tricky

craftybones 2021-04-15T14:03:30.053700Z

That way, failures show what happened

2021-04-15T14:04:30.055400Z

and maybe it is a test and “eval form” don’t play as nice I hope;

craftybones 2021-04-15T14:04:58.055700Z

One place where the general DRY principle hurts is in tests. There is a tendency to pull out stuff like common args, but I like leaving them in there, even if they look bulky

craftybones 2021-04-15T14:05:58.057400Z

(deftest some-test
  (testing "Some case")
    (is (= (actual-fn arg-1 arg-2...) expected)))))

craftybones 2021-04-15T14:06:26.058300Z

I do use a let block occasionally, but try to avoid anything that isn’t contextual

2021-04-15T14:06:45.058500Z

Agree, in other languages I have found myself fighting the balance between needed setup and how much that setup distracts from what I am trying to test… 😄

2021-04-15T14:07:36.059600Z

any tests I have done before in Clojure has been all side/play stuff, so was’t too concerned about how will this be maintained…

2021-04-15T14:07:44.059900Z

day job in Clojure changes all that…

2021-04-15T14:07:45.060100Z

😉

craftybones 2021-04-15T14:08:09.060600Z

Sure. I think it is easiest to go from raw values to fns later. If you abstract early, then it gets harder to maintain later

craftybones 2021-04-15T14:08:20.060900Z

My biggest learning with tests is to be as explicit as possible

2021-04-15T14:09:10.062Z

yeah, Ruby and JS most recently, but even .NET before those, I tend to prefer any setup as local to the test as possible

2021-04-15T14:11:52.064300Z

and maybe some of it should be let bindings should prefer to be in a testing instead of at the top level of deftest

NoahTheDuke 2021-04-15T14:16:13.064900Z

test helper functions that i've written look like this:

(defn get-ice
  "Get installed ice protecting server by position. If no pos, get all ice on the server."
  ([state server]
   (get-in @state [:corp :servers server :ices]))
  ([state server pos]
   (get-in @state [:corp :servers server :ices pos])))

NoahTheDuke 2021-04-15T14:24:25.066500Z

not super meaningful by itself i now realize, but my intention is to show that a given helper function shouldn't be doing any business logic, just cutting out some of the typing so that you can more accurately demonstrate the desired function/effect

seancorfield 2021-04-15T14:26:13.067100Z

I think that if you’re finding yourself writing tests with that much setup, maybe you need to refactor your functions to be easier to test?

seancorfield 2021-04-15T14:26:56.067900Z

(I find REPL-Driven Development and Test-Driven Development tend to produce code that is simpler and easier to test)

twashing 2021-04-15T15:34:41.070300Z

I’m trying to wire into the cljs.test reporting system with a custom macro. I’m following the pattern in cljs.test/deftest: https://cljs.github.io/api/cljs.test/deftest https://github.com/clojure/clojurescript/blob/r1.10.773-2-g946348da/src/main/cljs/cljs/test.cljc#L230-L246 Copying and using deftest works just fine. But if I simply create my own test macro defspec-test, and return the results, I get the error Cannot read property 'test' of undefined. Anyone know what’s going on here? util.cljc

(defmacro deftest2 [name & body]
  (when cljs.analyzer/*load-tests*
    `(do
       (def ~(vary-meta name assoc :test `(fn [] ~@body))
         (fn [] (cljs.test/test-var (.-cljs$lang$var ~name))))
       (set! (.-cljs$lang$var ~name) (var ~name)))))

(defmacro defspec-test [name sym-or-syms]
  (when cljs.analyzer/*load-tests*
    `(do
       (def ~(vary-meta name assoc :test `(fn [] ~sym-or-syms))
         (fn [] (cljs.test/test-var (.-cljs$lang$var ~name))))
       (set! (.-cljs$lang$var ~name) (var ~name)))))
mytest.cljs
(deftest2 zoobar
  (t/is (= 1 1)))

(defspec-test coocoobar
  (t/is (= 1 1)))
Run results
Testing mytest

ERROR in (coocoobar) (TypeError:NaN:NaN)
Uncaught exception, not in assertion.
expected: nil
  actual: #object[TypeError TypeError: Cannot read property 'test' of undefined]

Ran 2 tests containing 2 assertions.
0 failures, 1 errors.

seancorfield 2021-04-15T15:47:35.070900Z

The arguments are different between those two macros @twashing

twashing 2021-04-15T15:48:50.072500Z

Correct. I need to invoke it differently. Does that affect the test system? deftest2 [name & body] vs defspec-test [name sym-or-syms]

seancorfield 2021-04-15T15:51:28.073900Z

Ah, I see you’re expanding the argument(s) differently too. I would suggest looking at both expansions using macroexpand — but I’m not sure how you’ll do that interactively with ClojureScript.

1
twashing 2021-04-15T16:01:57.074700Z

I can try the exact same implementation as in cljs/test.cljc, but still get the error. https://github.com/clojure/clojurescript/blob/r1.10.773-2-g946348da/src/main/cljs/cljs/test.cljc#L230-L246

(defmacro defspec-test [name & sym-or-syms]
  (when cljs.analyzer/*load-tests*
    `(do
       (def ~(vary-meta name assoc :test `(fn [] ~@sym-or-syms))
         (fn [] (cljs.test/test-var (.-cljs$lang$var ~name))))
       (set! (.-cljs$lang$var ~name) (var ~name)))))

twashing 2021-04-15T16:05:10.075400Z

As per your Q, in a comment block you can do this.

(macroexpand
    '(defspec-test coocoobar
       (t/is (= 1 1))))

=> (def coocoobar (clojure.core/fn [] (clojure.test/test-var (var coocoobar))))

seancorfield 2021-04-15T16:08:16.076Z

Sorry, I don’t do any ClojureScript — only Clojure — and this looks specific to cljs.

NoahTheDuke 2021-04-15T16:30:58.076100Z

woah really? do you do selmer/templating for all front end stuff?

twashing 2021-04-15T16:35:31.076300Z

@seancorfield No problems!

seancorfield 2021-04-15T16:47:04.076500Z

@nbtheduke Our main app — the dating app on 40+ sites — is React.js, not cljs. We have a couple of Clojure apps that do SSR with Selmer (our login server, our billing server).

👍 1
seancorfield 2021-04-15T16:50:34.076800Z

We explored cljs about seven years ago and decided it wasn’t ready for anything customer-facing at the time — the tooling was pretty bad and the language had a lot more differences from Clojure. When we started to build the new dating app (the old one was ColdFusion-based!), JS/React was really the only sane option for us. If we were starting over, maybe we’d use cljs. We are going to build a few new small apps where we are probably going to use cljs but the main app will stay JS for the foreseeable future.

NoahTheDuke 2021-04-15T17:22:31.077400Z

That makes total sense. Thanks for the reply!