Clojurians Log v2

Hi All - I have a question about the implementation of LazySeq, actually several questions, but I'll stick with the main one first: Since the lazy-seq macro expands into a LazySeq deftype (which wraps the body in a thunk) at macro-expand time, it is not until runtime afaict when first or rest is called on the LazySeq that the the body is evaluated. However, when it is evaluated, the recursive nature of the body results in another call to lazy-seq, but this is after compile / macro-expansion time, right? How does that work?

Calling first or rest results in a call to seq on itself, which finally calls .sval. Inside the .sval call (and this is now runtime) the thunk bound to the deftype member fn is called resulting in a new call to the lazy-seq macro. It seems there is some interleaving of macroexpansion and runtime, or I'm misunderstanding the implementation.

Take my comment with a big grain of salt, since I am much less familiar with ClojureScript than Clojure/Java, but at least in Clojure/Java when you do a defn containing one or more uses of macros, those macros are expanded while compiling the entire defn form, not at run time.

i.e. at run time, there should be no remaining occurrences of lazy-seq or any other macro call.

I agree w/ your second statement, not sure how the first related though

What's confusing me is that at compiletime, (lazy-seq some-body) gets macroexpanded into the LazySeq deftype

Part of your original statement was "resulting in a new call to the lazy-seq macro". I do not think that is true.

e.g. (new cljs.core/LazySeq nil (fn [] ~@body) nil nil))`

doh.. (new cljs.core/LazySeq nil (fn [] ~@body) nil nil))

that's the macroexpansion right there

so that's what you have at runtime (i think), right?

and when you call any ISeq function on that, there is another call to lazy-seq that occurs, which appears to be at runtime

Every occurrence of a lazy-seq macro should be replaced by an expression like its body at compile time, yes, and what you pasted above is what I see in ClojureScript source code as its body.

That body contains no occurrences of the lazy-seq macro, though, so how would lazy-seq be involved at run time?

the body typically would return a recurisve call to a procedure which calls lazy-seq

for example: `(defn my-iterate [f x] (lazy-seq (cons x (my-iterate f (f x)))))`

A procedure which, at compile time, had all of its occurrences of lazy-seq replaced with the body of lazy-seq, too.

You could write my-iterate without using lazy-seq at all, just using the body of lazy-seq yourself, and you should get the same effect as if you used lazy-seq

Unless I have messed something up in my copy and paste (possible), you could write my-iterate like this:

(defn my-iterate [f x]
  (new cljs.core/LazySeq
       nil (fn [] (cons x (my-iterate f (f x))))
       nil nil))

and that is what the compiler sees and actually compiles, after it has done macro invocation

OK yes i'm with you

so now we agree macroexpansion is over, and ew have a lazyseq type

well.. LazySeq

right. Nothing named lazy-seq remains after macro expansion

oen of the fields of that LazySeq is fn which is bound to that body

bound to (fn [] (cons x (my-iterate f (f x))))

yes

or slightly more precisely, the function that expression evaluates to.

now we call first, rest, or anything else... and it evals that body

it calls the function

it calls that already-compiled body.

yes

which returns (cons x (my-iterate f (f x)))

and my-iterate expands into... a call to lazy-seq

there is no lazy-seq remaining anywhere at this point.

my-iterate has been compiled into javascript at this point. its not expanded

there is the function that looks like what I pasted above.

ok, what happens when (my-iterate f (f x))) is evaluated

It creates and returns a new LazySeq object.

ahhhh

ok I think I get what you're saying now

(after first evaluating the (f x) part of the expression)

right

I think this is a nuance of macroexpansion i've not encountered...

Think of them as a code-writing convenience, and they are all gone by the time "the real compiler" sees the ClojureScript code.

not exactly accurate, since macro expansion is part of the compiler, but perhaps useful in thinking of the distinction of macro expansion time.

I understand the concept of macroexpansion time and the fact that all macro calls are replaced with their expansions by the time runtime comes around

this is odd because at runtime there is a call that uses some already expanded code.. ors omething

need to think on this a bit

the way I have been thinking about it is that any macros the compiler can see, in source text, get replaced... and then you're done and on to runtime

I completely understand if that plus recursion can throw one for a mental loop, though.

but that's not quite the case there

here*

It seems to me to be the case here. Macros are expanded at compile time, the post-macroexpanded code is compiled to JavaScript, and then you are completely in the JavaScript runtime.

The post-macroexpanded code is a function that returns a new object. One field of that object happens to be a reference to an already-compiled function, in this case an anonymous one that happens to call the outer function.

I've got it

I don't know why I was so confused

the second call to my-iterate was already macroexpanded at that time

Wow that was a bit of a headsplosion for me

thanks for taking the time to patiently explain it

Cut yourself a little slack -- I think it has that effect on lots of people the first time they understand it (and many just use it without trying to pick it apart).

there is always the draw over time to try and pick apart clojure/script source for me... hah

It reminds me of the saying "In order to understand recursion, one must first understand recursion"