Clojurians Log v2

I was trying to extract all the tests from the test-plan, which is a quite nested data structure

and this seems to work

(def plan (kr/test-plan))

(def all-test-names
  (->> plan
       :kaocha.test-plan/tests
       (mapcat :kaocha.test-plan/tests)
       (mapcat :kaocha.test-plan/tests)
       (map :kaocha.var/name)
       sort))

I wonder if there is an easier way though

There's kaocha.testable/test-seq, or you can use clojure.walk

ah yes better thanks.

seems like I was getting the same result at least

I posted an example of using clojure.walk recently https://github.com/lambdaisland/kaocha/blob/master/repl_sessions/strict_focus_meta_hook.clj

apart from the skip maybe

you are encourage to use hierarchy/leaf? to check if a testable is a an actual test as opposed to a grouping like a namespace or a test suite

different test types may have more or fewer than two levels of nesting

also :kaocha.var/name is specific to clojure.test, whereas all testables have a :kaocha.testable/id

ah nice thanks

well what I have to do is to actually split tests with the circleci tool

(def all-test-names-2
  (->> (kaocha.testable/test-seq plan)
       (map :kaocha.var/name)
       (remove nil?)))

(def split
  (-> (sh/sh "circleci tests split --split-by=timings"
             :in (clojure.string/join "\n" all-test-names-2))
      :out))

and then run them with kaocha

something like (kr/run "--focus" (first all-test-names-2)) for example

but yeah running all the ones returned by the split

but yeah I guess I can use the id as well

yeah so in this case you definitely want to use kaocha.testable/id, that's what you pass to focus. it just happens to correspond to the var name for clojure.test

ah yeah but don't think circleci would handle that

I'm not familiar with this circleci tool. What does it do?

it's https://circleci.com/docs/2.0/parallelism-faster-jobs/#splitting-by-timing-data

so it kind automatically split tests by timing, parallelising automatically

but still keeping only one job

we had that working for clojure.test, I just wanted to convert to kaocha

interesting, I'm guessing it gets the test information via junit.xml?

yeah probably

from the circleci agent from previous runs I guess

in which case it will have the testable ids

ah ok I'll try

I mean if you're only using clojure.test it comes down to the same thing

and then (kaocha.repl/run {:focus [id1 id2 id3]}) I think

I'm always confused about the arguments to pass to kr/run

finops.ci-tests> (kr/run {:focus smaller-subset})

0 tests, 0 assertions, 0 failures.
#:kaocha.result{:count 0, :pass 0, :error 0, :fail 0, :pending 0}
finops.ci-tests> (kr/run "--focus" (first smaller-subset))
[(...........................................................................................................................................................................................................................................................................................................................................................................................................................................

so it works fine if I pass things same args I would pass to the cli

but not with configs

hmmm that's surprising

but that "--focus" version only works by accident, kaocha.repl does not take command line options like that

kaocha.run takes testable-ids, and optionally a config map as last arg, so you can just do (apply kaocha.repl/run subset)

ahok thanks let's see if

(apply kr/run
         (conj (tests-to-run)
               {:plugin :kaocha.plugin/junit-xml
                :junit-xml-file "test-results/kaocha/results.xml"}))

maybe I can add an example in hte kr/run docs

it's not the first time I get confused even after looking at docs and code

that'd be great! doc improvements are always most welcome

in general a writeup of this parallel testing with circle would be very cool!

ah yeah as soon as I get it working

I can do that

Yeah I didn't even know there was this feature, but it's quite a good way to get some free parallelism without having to split yourself creating multiple jobs

still doesn't work but it's because of something I'm passing wrong to the circleci command, but well almost there

mm so splitting didn't work yet, but the other weird thing is

..............................................................................................................................................................................................................................................................................................................................................................................)]
1 tests, 2800 assertions, 0 failures.

so doing it like this seems to group all the tests into one

which is maybe expected I guess and maybe not a big deal

just seems a bit strange

also any idea how to pass the --reporter documentation config in that config map?

:reporter :documentation or :reporter "documentation" don't work

also I was trying to use leaf? but I find nothing if I do this for example

(defn fetch-tests
  [test-plan]
  (w/postwalk
   (fn [t]
     (when (kh/leaf? t) t))
   test-plan))

I think that's because of how postwalk works, you're removing the parents so the chidren are never reached

kaocha.repl/config is your friend to see what the results are of your config options

(:kaocha/reporter (kaocha.repl/config))
(:kaocha/reporter (kaocha.repl/config {:reporter kaocha.report/documentation}))