Clojurians Log v2

I’ve just seen this WARNING on clojure 1.10.0 on jdk 12: > WARNING: An illegal reflective access operation has occurred >WARNING: Illegal reflective access by clojure.lang.InjectedInvoker/0x0000000801182440 (file:/Users/rick/.m2/repository/org/clojure/clojure/1.10.0/clojure-1.10.0.jar) to method com.sun.org.apache.xerces.internal.jaxp.datatype.XMLGregorianCalendarImpl.getYear() > WARNING: Please consider reporting this to the maintainers of clojure.lang.InjectedInvoker/0x0000000801182440 > WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations > WARNING: All illegal access operations will be denied in a future release I suspect it’s a known issue

I see that all the time, and also suspect it's a known issue 🙂

I'd suspect that this isn't actually an issue in Clojure but rather some library relying on reflection and the JDK blaming the wrong thing for it

yes

the reflection happens in Clojure (at the behest of code doing reflection)

InjectedInvoker doesn't appear in the clojure source at all, curiously

the question is - who is calling getYear on a calendar impl?

what's the policy change here anyhow? what kind of reflection is no longer allowed?

https://clojure.org/guides/faq#illegal_access

@rickmoynihan use --illegal-access=debug and it will tell you where it's happening

it will dump a stacktrace

thanks alex & ghadi

well the stack trace may just be the clojure compiler, which would not tell you

depends whether it's compile-time or run-time reflective determination being made

is that just the distinction between (def foobar (.getYear coolCalendar)) and (defn jhomason [] (.getYear coolCalendar)), or something more subtle?

ok thanks that gives me a decent stacktrace… thankfully it also looks like it’s in one of my libraries, so I can take a look at it

yes, that, I believe

coolthx

@rickmoynihan public library?

yeah

https://github.com/Swirrl/grafter

@alexmiller the compiler should be free to reflect without warning, no? it's only usage/invocation that incurs the illegal access warning

iirc it uses the javax xml calendar for some type coercions — so I guess this is unsurprising

boooooooo

it has to read xsd datatypes so it needs to do something like that

ah

we coerce them to java.time; but have to read XSD representations off the wire so to speak

will shave this yak another day

sounds like you have an actually legit usage of javax.xml for date formatting

perhaps the world's first 😂

@ghadi when you say the compiler should be free to reflect, do you mean the compiler should be free to generate reflection code?

I mean the compiler should be free to enumerate a class's methods, even inaccessible ones. If it generates bytecode that refers to an inaccessible method, that's where the warning will happen -- in runtime, not in the compiler

yeah, right

enumeration doesn't ever warn

so (def foo-bar (.getYear coolCalendar)) might warn/throw, but the stack would at least point to the line in question despite having lots of compiler frames in it, I think

meaning --illegal-access=debug is a reliable method to get accurate traces

so I'm not sure what Mister Miller was worried about

yeah it’s a really gross API… but RDF borrows its data types from xsd - so needs must.

so that example is interesting because it's runtime reflective - as you say it will have extra frames in it -- but I don't think there will be any compiler frames

the main job grafter does is coerce that gunk into proper values

the compiler will just give up and emit reflective code

but the code runs at compile-time

oh yeah - true

I retract my worry

the only emoji completion for phew is :christopher_walken:

I'm running a nested java.jdbc query, trying to process both result sets lazily in parallel, and am not getting the head-holding-OOM I expected, and I'm curious to understand why the simplest version of the structure is:

(jdbc/query ...
 {:result-set-fn
  (fn [xs]
   (jdbc/query ...
    {:result-set-fn
     (fn [ys] (reduce ... (map vector xs ys)))}))})

the only way I can imagine this is working anyhow is if we have this sequence of events 1. .invoke called on the inner function 2. inner function essentially passes xs to reduce 3. jvm notices that the instance variable xs won't be used again in the invoke method 4. AND the jvm notices that the instance of the inner function isn't going to be used again either 5. and so concludes that the head of xs is unreachable

is this really happening because it sounds too fancy to be true

I dunno maybe it isn't

I guess it just never occurred to me that the jvm could tell a function was being called for the last time

Your description above sounds like what I understand escape analysis to be. I’ve no idea if it’s happening; but escape analysis is something the JVM does; and it is something it’s getting better at. So what you say sounds plausible to me.

I think it is entirely possible it is just locals clearing, but you would want to look at the bytecode to be sure

the clojure compiler can't know the function is only going to be called once

if it might be called again, you have to hold onto the head, just in case