Clojurians Log v2
kaocha
Is there a way to make kaocha --print-test-plan emit edn?
It appears to print vars e.g. :kaocha.var/var #'foo.bar/baz as far as I can tell #' isn’t a valid edn tagged literal
Ok, so backing up a bit my direct use case is refactoring large test suites that take a long time to run. We have several test suites with various focus-meta / skip-meta’s defined, and I’d like to know what tests are going to be executed in each suite (or execution of the cli with focus opts etc), without actually running them all. So an abbreviated output of the plan, that showed the order of execution and the nesting of namespaces / test vars. Also it’s possible in complex suites to have overlapping tests, such an output would be useful to identify the overlap and potentially the cause
starter for 10 on a humanized output format that might be useful:
--- unit (clojure.test) ---------------------------
csv2rdf.metadata.properties-test ^:foo ^:bar
child-test ^:baz
set-table-group-parent-references-test
children-test
csv2rdf.metadata.inherited-test ^:foo ^:baz
null-value-test ^:bar
string
array
array with invalid values
invalid type
csv2rdf.http-test
parse-link-header-test
Valid header URI only
Valid header with parameters
Invalid URI
Malformedi.e. similar to the document output just missing testing blocks as printing those will mean executing the tests.
Could also be useful to augment ns’s and vars with their metadata keys… though we might want to elide standard clojure metadata e.g. ^:doc / ^:line etc.
(clojure.edn/read-string {:default tagged-literal} "{#'foo :bar}") raises:
Util.java: 221 clojure.lang.Util/runtimeException
EdnReader.java: 552 clojure.lang.EdnReader$DispatchReader/invoke
EdnReader.java: 757 clojure.lang.EdnReader/readDelimitedList
EdnReader.java: 680 clojure.lang.EdnReader$MapReader/invoke
EdnReader.java: 145 clojure.lang.EdnReader/read
EdnReader.java: 111 clojure.lang.EdnReader/read
EdnReader.java: 67 clojure.lang.EdnReader/readString
edn.clj: 46 clojure.edn/read-string
edn.clj: 37 clojure.edn/read-string
REPL: 15 mut.loader-test/eval98441
REPL: 15 mut.loader-test/eval98441
Compiler.java: 7177 clojure.lang.Compiler/eval
Compiler.java: 7132 clojure.lang.Compiler/eval
core.clj: 3214 clojure.core/eval
core.clj: 3210 clojure.core/eval
interruptible_eval.clj: 87 nrepl.middleware.interruptible-eval/evaluate/fn/fn
AFn.java: 152 clojure.lang.AFn/applyToHelper
AFn.java: 144 clojure.lang.AFn/applyTo
core.clj: 665 clojure.core/apply
core.clj: 1973 clojure.core/with-bindings*
core.clj: 1973 clojure.core/with-bindings*
RestFn.java: 425 clojure.lang.RestFn/invoke
interruptible_eval.clj: 87 nrepl.middleware.interruptible-eval/evaluate/fn
main.clj: 437 clojure.main/repl/read-eval-print/fn
main.clj: 437 clojure.main/repl/read-eval-print
main.clj: 458 clojure.main/repl/fn
main.clj: 458 clojure.main/repl
main.clj: 368 clojure.main/repl
RestFn.java: 137 clojure.lang.RestFn/applyTo
core.clj: 665 clojure.core/apply
core.clj: 660 clojure.core/apply
regrow.clj: 20 refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
RestFn.java: 1523 clojure.lang.RestFn/invoke
interruptible_eval.clj: 84 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 56 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 152 nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
AFn.java: 22 clojure.lang.AFn/run
session.clj: 202 nrepl.middleware.session/session-exec/main-loop/fn
session.clj: 201 nrepl.middleware.session/session-exec/main-loop
AFn.java: 22 clojure.lang.AFn/run
Thread.java: 834 java.lang.Thread/runYes IIRC the issue is that dispatch macros are essentially reader macros and only implementable by clojure/edn, i.e. in clojure and edn you can’t extend the reader itself; only tags. ' is a reader macro for vars… others include things like { for sets etc. AFAIK ' isn’t included in the edn subset of clj.
yeah the output of --print-test-plan isn't readable, making it unsuitable for its intended purpose: an interface that other tools/scripts can consume. Either we should add print handlers so things like vars/ns/atoms etc print as tagged literals, or we should do a pre-processing pass to filter them out or replace them with placeholders. Not sure if there's already an issue for this, if not feel free to create one. A PR for this would be more than welcome, perhaps integrating https://github.com/lambdaisland/data-printers . I'm quite partial to the tagged literals approach, my only concern with that is that Kaocha then modifies the global environment. But maybe we can only load the print handlers when calling --print-test-plan or --print-result. (`--print-config` is also in this list but is less likely to contain unprintable objects)
Thank you 🙏:skin-tone-2:
what are you planning to do with the plan you read from a file? how would u like those vars to be represented?
Well the plan is pretty noisy, so initially I was just wanting to post-process it with something like jet, just so I could understand it.
However after looking at it, I realised it seems quite recursive; so it’s awkward to handle in jet anyway.
Really I was hoping --print-plan or another option would let me figure out easily the test vars that are executed and their order (for a given focus etc), without actually running them.
Obviously the plan contains enough to figure that out, it’s just not easy to read it as a human.
i was trying to provide a dispatch fn for the single quote, but i failed too:
$ jet --from edn --edn-reader-opts "{:readers {(symbol \"'\") (fn [& args] args)}}" <<<"#'asd"
Exception in thread "main" clojure.lang.EdnReader$ReaderException: java.lang.RuntimeException: No dispatch macro for: '
...aaand what would u use the processed information for?
just looking at it, so u can learn more about how kaocha works?
trying to debug something, like the paths being traversed or see the effect of :focus-meta?
or maybe build some tooling on top, like show it in an emacs buffer?