Clojurians Log v2
testing
I’m setting up some integration tests for a Clojure app that’s using Lein and I’m curious how others typically approach this situation. (1) My app needs to run a shell script before and after the tests run (doesn’t matter whether I invoke the scripts inside or outside of the JVM), and I need to run some arbitrary setup function in the JVM once before any of my tests run. (2) I have three different types of tests: unit, integration tests that mock some things, and full integration tests which make real calls. I planned on using lein test-selectors for handling the different types of tests and I thought that would work. For the first requirement, I wrote individual Clojure namespaces for running the proper shell scripts and executing the aforementioned arbitrary setup function before running tests. However, I realize now that core.test doesn’t appear to provide an easy way to execute tests based on those selectors--that’s baked into Lein. I realize that I could hack that selector functionality into my own app, but this is significantly more work than I anticipated for something that I feel is rather common for large Clojure apps. What does your test setup look like? How would you deal with this situation?
At work we use deftest for testing pure functions and a custom macro for integration tests. It adds some metadata so it can be filtered by lein selectors. Tests run in parallel with https://github.com/weavejester/eftest . At CI pipeline we additionally use the “integration” selector to run tests in different jobs to parallelize and speed up the test suite. The macro also runs the test in a DB transaction that is rolled back at the end of the integration test.
(defmacro def-integration-test
[test-name & body]
`(deftest ~(vary-meta test-name assoc :integration true)
(clean (fn [] ~@body))))Interesting. So is clean a function that handles the DB transaction part?
Yes, it starts the transaction, runs the test and then does a rollback:
(defonce ^:dynamic *db*
;; create some connection here (e.g. jdbc / c3p0)
)
(defn clean
[f]
(jdbc/with-db-transaction [db *db*]
(jdbc/db-set-rollback-only! db)
(try
(f)
(catch SQLException e
(prn (.getNextException e))
(throw e)))))Also, I looked at using :once fixtures, but it appears that those only apply to the namespace level
Just found this in the Lein source code. Doesn’t sound like something I really want to be doing:
(def form-for-suppressing-unselected-tests
"A function that figures out which vars need to be suppressed based on the
given selectors, moves their :test metadata to :leiningen/skipped-test (so
that clojure.test won't think they are tests), runs the given function, and
then sets the metadata back."
Is there a use case for calling clojure.test/is outside the dynamic scope of a deftest?
The reason I’m asking is that I accidentally typed def instead of deftest the other day. It almost worked: the test ran when the namespace loaded and even printed its complains when it failed. But of course it couldn’t add the failure to the summary, and the overall test run succeeded.
I was thinking that this particular mistake could be prevented if clojure.test/inc-report-counter asserted *report-counters* instead of just doing nothing when it’s nil.
By binding *assert* to false while compiling or requireing the codebase
> Is there a use case for calling clojure.test/is outside the dynamic scope of a deftest?
sometimes I'm able to use is in quick repl experiments
I use it inside some fns that generate data (fixtures) from a spec. The fns allow you to pass in custom data for the maps but we sometimes mess it up and pass invalid data according to the spec. We have something like the following to warn us the data we passed is wrong:
(is (spec/valid? ::my-spec new-data)
(str (spec/explain-str ::my-spec new-data)))is output. I’m guessing there could be more use cases for it.@jivagoalves as an aside, it seems a good idea to wrap such an is in an assert, so that you can choose to disable such checks in production