Clojurians Log v2

is it a bad idea to use the clojure compiler options in my code, something like so:

(if (:direct-linking *compiler-options*)
  handler
  #(apply handler  %&))

@dominicm Is this to circumvent direct linking? In that case you can make your var ^:redef or ^:dynamic if it should be a dynamic var.

It is also possible that some parts of the Clojure code were compiled with direct linking enabled, and other parts were compiled with it disabled, and I am not sure whether and expression like that, even if it does appear to work, would actually "remember" which code was compiled how. clojure.core that most people use is compiled with direct linking enabled, and unless someone enables it, other Clojure code is compiled with it off.

@borkdude the broader context of this would be use with jetty or anywhere you pass a function for later execution. Making your ring functions reloadable without a reset.

The above still applies to your situation I think

I think reitit's docs have a solution for you https://cljdoc.org/d/metosin/reitit/0.5.10/doc/advanced/dev-workflow#an-easy-fix

@ben.sless yes, I'm wondering if direct linking could be used as a proxy for switching between those two routers, yes.

@borkdude I've not heard of redef, so I need to check that out.

@dominicm It's documented here: https://clojure.org/reference/compilation#directlinking

You can probably even get away with making that metadata dynamic based on dev or prod

Okay, not after redef @borkdude. I'm not trying to bypass direct linking. I want to allow redefining a function in development, but not in production. But I'm really working around the fact I'm passing a function value around, rather than a var.

Then why did you bring up direct linking?

#(foo 1 2 3) will see redefinitions of foo, but (partial foo ...) won't

I was thinking that direct linking would be an appropriate flag to use to switch between the two.

@dominicm I'm curious if you've measured the overhead of using the Var in production vs a function value? We tend to just leave the Var references in place for handlers in our apps, but we do have direct linking enabled when we AOT compile for uberjars (so, in production, we can't redef functions on the fly in general anyway).

@seancorfield I have, https://github.com/SevereOverfl0w/direct-performance But the difficulties slip in cases like the reitit one where compiling a router is an expensive operation.

@dominicm That seems to indicate that indirect use is faster than direct use 👀 and neither of those tests use a Var reference?

"the difference appears insignificant."

And the ability to hotfix a var in production might outweigh the small perf hit :P

that is a quote from the repo readme

I understood that

but I should also point out, that the code in the repo is not testing direct linking, and is not testing using a var reference directly

so you should be very careful about using that to justify making decisions about direct linking and var reference usage

my understanding is directly linking is mostly a win around avoiding the volatile field which vars use to hold data

reading and writing a volatile is, relative to a lot of user code, very cheap

but I believe it limits the amount of inlining the jvm jit can do

so in a hot loop, direct linking can be a huge performance win, because it lets the jit do more inlining, apply more optimizations, get more opportunities to inline, and so on

for these cases you could maybe turn on direct linking temporarily, compile the function with hot loops and then disable it again

but that would get confusing if you patch a var that's used in the body of such a function

and elsewhere

you could program against this by de-refing the vars outside the loop and then using the bindings in the loop as well

(let [x @#'inc] (loop [] (x 2)))

or even without @#' this works already

but then you would get a lookup in the lexical context. I'm not sure how expensive that is, maybe similar to a var deref?

user=> (defn foo [x] (inc x))
#'user/foo
user=> (time (dotimes [i 10000000000] (foo 1)))
"Elapsed time: 6210.834273 msecs"
nil
user=> (time (let [foo foo] (dotimes [i 10000000000] (foo 1))))
"Elapsed time: 2476.871089 msecs"
nil

@borkdude I am not sure this is a conversation for #clojure-dev

sorry

I mean, I don't know, it just seems like a tutorial about how clojure compiles references to different kinds of names would be better somewhere like #clojure

I think of #clojure-dev to be more about the development of clojure, where that kind of thing can usually be assumed. But I don't know that I've seen a statement of purpose for #clojure-dev , so I may just be wrong

I was exploring a local variation of direct linking but I probably could have done that in private and not while thinking out loud. You're right.

Direct linking also directly affects bytecode size and thus load time because it does need to load or store vars in fields

Ah of course, should have remembered that, the impact to load times is why we have it turned on at work

I have also found it decreases GC pressure