Clojurians Log v2

Hey there, I am running into some issues with speculative. Here is a small example to reproduce the issue

(set! *print-level* 10)
  (set! *print-length* 10)
  (require '[clojure.tools.analyzer.jvm :as ana])
  (require '[clojure.tools.deps.alpha.util.maven :as mvn])
  (require '[clojure.tools.deps.alpha.repl :refer [add-lib]]) ;; using the add-lib branch of tools.deps.alpha
  (require 'speculative.instrument) ;; without loading speculative it works
  (add-lib 'edos {:mvn/version "0.1.0-SNAPSHOT"})
  (ana/analyze-ns 'edos.core)

(require 'speculative.instrument) 
(ana/analyze-ns 'speculative.instrument)
(remove-ns 'speculative.instrument)

> ;; without loading speculative it works what works?

what is going wrong?

@borkdude I am getting an ClassCastException

Execution error (ClassCastException) at speculative.core/fn (core.cljc:274).
class clojure.lang.LazySeq cannot be cast to class java.util.Map$Entry (clojure.lang.LazySeq is in unnamed module of loader 'app'; java.util.Map$Entry is in module java.base of loader 'bootstrap')

The last line of the first code snippet fails.

speculative.core/fn ... hmm, not sure what that is and this is at core.cljc 274: https://github.com/borkdude/speculative/blob/master/src/speculative/core.cljc#L274 I have no clue if I have to go by just that snippet. Maybe you can post the entire exception somewhere?

You can also try to call (speculative.instrument/unstrument) before removing the ns, maybe that helps

remove-ns just deletes an in-memory object, but maybe spec instrumentation breaks when you try to do that

also it seems you're using jdk11. do you get the same error with jdk8?

It's the spec for map

https://github.com/borkdude/speculative/blob/master/src/speculative/core.cljc#L302

One of the key access fails

can you put a println before that key access to see what is the value of args or ret?

output of having added (println args ret)

[:seqable {:f #object[clojure.tools.analyzer.utils$dissoc_env 0x24f3d208 clojure.tools.analyzer.utils$dissoc_env@24f3d208], :colls [[{:env {:locals {}, :ns edos.core, :file jar:file:/home/fv/.m2/repository/edos/edos/0.1.0-SNAPSHOT/edos-0.1.0-SNAPSHOT.jar!/edos/core.cljc, :end-column 33, :column 1, :line 156, :once false, :end-line 162, :context :ctx/expr}, :form events, :name events, :variadic? true, :op :binding, :arg-id 0, :local :arg}]]}] ({:form events, :name events, :variadic? true, :op :binding, :arg-id 0, :local :arg})

so it seems ret here is a lazy-seq instead of a map-entry...

no changes with respect to jdk8

error could be coming from this place: https://github.com/clojure/tools.analyzer/blob/d2c268a616d63b813be13fbfe1e786e41577b891/src/main/clojure/clojure/tools/analyzer.clj#L597

The edos lib I am analyzing depends on orchestra. Is this somehow incompatible? But more of a guess?

ehm, yeah could very well be. can you try with plain spec?

fwiw the specs in speculative are tested quite well, even using generative testing

user=> (instance? clojure.lang.MapEntry (s/conform (s/or :seqable seqable? :transducer any?) (map inc [1 2 3])))
true
user=> (instance? clojure.lang.MapEntry (s/conform (s/or :seqable seqable? :transducer any?) (map inc)))
true

@finn.volkel There was an issue about this exact problem here: https://github.com/jeaye/orchestra/issues/26

Ok. I am not familiar enough with spec. But why would this only surface if one adds speculative?

What do you mean by "trying with plain spec"?

because speculative has specs for clojure.core. if a library patches spec incorrectly, you'll get problems when you run core functions

what are you trying to accomplish? do you actually want to run with clojure.core specs instrumentation or do you just want to analyze code?

Just analyze code and learn something along the way, but I get your point that issue is somewhere else.

speculative.instrument is a "special" namespace that enables instrumentation when required.

so it would be better to not require that namespace if you don't want instrumentation

The main thing is I guess why speculative still shows up even if I used remove-ns

because by then the namespace has already been loaded

sorry my bad, it doesn't do that automatically, but only when you call i/instrument

I think your workflow is as follows: you load all specs from speculative (by requiring speculative.instrument) and orchestra instruments them. Now when you call a core function, you run into a bug with orchestra.

removing namespaces does not unstrument functions.

but maybe you can get rid of the problem by calling i/unstrument before doing anything else after loading speculative.instrument

(still not clear to me why you would load this in the first place)

I ran into the issue because I analysed speculativewith analyze-ns itself. I am just pulling jars with the clojars api, analyze the namespaces and then remove them again. I guess the easiest thing is to blacklist speculative in this case. I don't know though if the problem will show up again if some project depends on speculative .

thanks for the help anyway

@finn.volkel but if you're not actively instrumenting the clojure.core specs from speculative, how could you run into a bug with one of them?

I should maybe add that analyze-ns also evaluates the stuff in the namespace.

even then

does it execute snippets from comment sections?

or does orchestra have a hook that once you enable instrumentation and a new spec gets loaded it's automatically instrumented? something fishy there

The end of the stacktrace looks as follows:

[[clojure.core$key invokeStatic core.clj 1567]
  [clojure.core$key invoke core.clj 1567]
  [speculative.core$fn__106968 invokeStatic NO_SOURCE_FILE 102]
  [speculative.core$fn__106968 invoke NO_SOURCE_FILE 101]
  [clojure.spec.alpha$spec_impl$reify__2059 conform_STAR_ alpha.clj 923]
  [clojure.spec.alpha$conform invokeStatic alpha.clj 164]
  [clojure.spec.alpha$conform invoke alpha.clj 160]
  [orchestra.spec.test$spec_checking_fn$conform_BANG___391943 invoke test.clj 97]
  [orchestra.spec.test$spec_checking_fn$fn__391949 doInvoke test.clj 129]
  [clojure.lang.RestFn invoke RestFn.java 421]
  [clj_astminer.retrieve$analyze_from_clojar_map invokeStatic retrieve.clj 133]

yeah, so somehow something has instrumented the specs in speculative.

I don't think analyze-ns evaluates anything in the comments.

maybe disable orchestra 🙂

if you have a repro using vanilla clojure spec and speculative, an issue is welcome

which version of orchestra are you using?

well orchestra is just loaded via edos which uses [orchestra "2018.09.10-1"]

ah, that explains it then

it's an old version which still has that bug from the issue I linked to earlier

I see thanks