Clojurians Log v2

Following on from my earlier analyser problem, I’ve now updated my code to use some code cribbed from cljdoc, which does something very similar to what I was doing. It works fine for simple cases, but I can’t get it to work for a test case I’m working on. I’m trying to use a cljc file to define a macro which will generate some vars in cljs which Cursive can’t see via static analysis. However when I run the analyser over it, what I get back looks more like the clj side of things:

{:name stub-test,
 :publics ({:name normal-def, :type :var}
           {:name normal-defn, :arglists ([]), :type :var}
           {:name hidden-defmacro2, :arglists ([]), :doc "Doc", :type :macro}
           {:name gen-clj-stuff, :arglists ([]), :type :macro}
           {:name NormalProtocol, :type :protocol, :members ({:name normal-method, :arglists ([this]), :type :var})}
           {:name hidden-defmacro5, :arglists ([foo__2740__auto__]), :type :macro}
           {:name gen-stuff, :arglists ([]), :type :macro}
           {:name hidden-defmacro6, :arglists ([foo__2740__auto__] [foo__2740__auto__ bar__2741__auto__]), :type :macro}
           {:name normal-multimethod, :type :multimethod}
           {:name hidden-defmacro4, :arglists ([]), :doc "Moar doc", :type :macro}
           {:name normal-method, :arglists ([this]), :type :var}
           {:name hidden-defmacro3, :arglists ([]), :type :macro}
           {:name hidden-defmacro, :arglists ([]), :type :macro}
           {:name normal-macro, :arglists ([]), :type :macro})}

(defn- analyze-file [js-dependencies file]
  (let [state (cljs.env/default-compiler-env)
        faked-js-deps (fake-js-deps js-dependencies)]
    (swap! state update :js-dependency-index #(merge faked-js-deps %))
    (ana/no-warn
      ;; The 'with-core-cljs' wrapping function ensures the namespace 'cljs.core'
      ;; is available under the sub-call to 'analyze-file'.
      ;; <https://github.com/cljdoc/cljdoc/issues/261>
      (comp/with-core-cljs state nil #(ana/analyze-file state file nil)))
    state))

(defn- read-file [ns-name resource js-dependencies]
  (let [state (analyze-file js-dependencies resource)]
    (if-let [ns (ana/find-ns state ns-name)]
      (-&gt; ns
          (select-keys [:name :doc])
          (merge (-&gt; ns-name meta (select-keys [:author :deprecated :added])))
          (utils/remove-empties)
          (assoc :publics (read-publics state ns-name))))))

@cfleming a bit busy today will try take a look later if someone else doesn't get to it first

@cfleming I took a look yesterday and didn’t make any traction. But with your last edit, it is worth pointing out that defmacro creates a function var when compiled directly by the ClojureScript compiler (as opposed to when they are compiled by the Clojure compiler). This is how the self hosted compiler can compile and make use of macros.

IIRC that function has meta on it or something else that marks it as a macro implementation

I can't remember there's a probably a reason for this?

(do
  (def x (transient {}))
  (dotimes [n 32]
    (conj! x [n :foo]))
  (println (persistent! x)))

@alexmiller this only prints out 8 kv pairs

You discarding the result of conj! right; that's the problem

oh blerg

@mfikes Oh, interesting, thanks - I didn’t know that (in fact, my knowledge of cljs macros in general is pretty limited).

However I would have expected the defmacros not to have even been seen by cljs at all since they’re behind a reader conditional.

The fact that compiling a macro in ClojureScript makes a function var could be thought of as bit of an implementation detail.

@dnolen If you do get a chance to look at this at some point, I have a repro repo here: https://github.com/cursive-ide/cljs-analyzer-repro

@cfleming the backlog is really too disjointed for me to understand the issue here

@dnolen Ok, I’ll update the README on that repo

there's also just way too many thing in that repro

and it's not clear what the problem or goal is?

@cfleming before looking at anything let's just backtrack and talk about it for a second

so I understand the bigger picture

Sure.

so you want to analyze a file and macros are appearing in the analysis when you don't expect it?

So this is related to the stub generation in Cursive, where I generate function stubs for vars which are dynamically generated at runtime. Cursive can’t see these via static analysis.

sure I get that, but if we can remove Cursive the discussion we can just focus on the fundamental issue

Ok.

so is my previous statement correct?

trying to analyze some file (irrelevant how it was created, generated or not)

and it has macros and those are appearing in analysis?

The issue is that when analysing two example files (helix.dom and my test cljc ns), I’m not getting back the results that I would expect. When I analyse this file: https://github.com/cursive-ide/cljs-analyzer-repro/blob/master/src/stub_test.cljc, the ns publics contain macro forms which I did not expect (although mfikes explained why this might be happening, to support self-hosting). It also does not contain vars which I did expect, and I don’t understand why.

so let's break apart this first thing that you've said

what should and should not appear because I don't understand that at all from looking at the file

line 4 the comment says the follows defs shouldn't appear

I understand that the analysis might contain macro vars even for cljs, as mfikes explained. But as far as I can tell they should have been completely hidden from cljs by the reader conditionals.

but that I don't understand at all

you're still trying to get ahead of me 🙂

line 6 will definitely appear via analysis

Sorry, that is my mistake - that is a comment for what I would expect to see in Cursive which I left in for the repro. It’s not relevant here. I’ll remove that.

great - I just want to see something that make sense wrt. ClojureScript

then hopefully you can sort out the Cursive part once we have an understanding

Absolutely.

ok this is what I think will happen

4-8, only 6 will not appear