Clojurians Log v2

Wow, I don't have enough evidence collected to confirm this hypothesis, but I might be seeing a case of JVM's JIT optimizing the method clojure.lang.Util.hasheq differently in two scenarios: (scenario 1) 99.9% of the calls to that method were with type Long (scenario 2) large fraction of calls were with type long, but also a large fraction with type PersistentHashSet. In scenario 2, the clojure.lang.Util.hasheq calls on type long are much much slower.

If that hypothesis is true, then I am awestruck with the extra levels of complexity of predicting the performance of code running on the JVM.

without knowing more that sounds like could be difference between monomorphic and bimorphic inlining

which version of java are you on?

These measurements are with OpenJDK 11.0.3 installed via apt-get on Ubuntu 18.04 Linux. I don't have clj-async-profiler measurements on macOS running Oracle/Apple JDK 1.8.0_192 but am seeing similar overall elapsed times of the two scenarios there.

If you're really interested, check out Aleksey Shipilev's blogs like https://shipilev.net/jvm/anatomy-quarks/16-megamorphic-virtual-calls/

you can pass jvm options to see what the jvm is doing around a lot of these kinds of things

Thanks for the link. I will at least start down the rabbit hole, but not sure how far I want to go just yet.

it's normal for bimorphic or megamorphic things to not inline as well so what you said above does not seem surprising to me

Clojure protocol call-site caches have the same kind of potential issue - last choice is cached, so monomorphic case will perform much better

Over the years I've worked with Clojure, I've often found that I've wanted to work with data -- hash maps -- but also work with Java objects that have setters. An example I'm working with right now is HikariCP and there's a Clojure wrapper that supports Clojure hash maps and turns them into .setFooBar calls for :foo-bar members. I want to do this in general and it feels like this ought to be part of core. Do we already have something that I'm just missing? Is this something that might be considered for core?

Java.data has something like that

'k thx...

I've written macros in limited domains to do this kind of thing in the past, as always depends how generic and/or customizable you want to make it

@hiredman It's close. :fooBar is needed instead of :foo-bar but it's close enough that I can use it for what I'm trying to do. Thanks.

well, that's the customizability - can overload the multimethod to fix

It's perfect for what I need, to be honest.

Turns out that org.clojure/java.data does exactly what I need for next.jdbc to use c3p0 and HikariCP so I can bake that into the library with minimal effort.

My performance problem sanity is restored -- I should have been looking for a simpler explanation, which was that clojure.lang.Util/hasheq was being called so many more times in my 10x worse performance scenario, about 10x more times, in fact. Duh. Application code that was the cause found and fixed.

Cool :)

Calling things more times is usually slower

It was not the cause of any problems for me, but was curious if you thought there may be any interest in a CLJ JIRA performance-related ticket on the equiv() methods I found that lack an identical fast path?

sure, if you happened to find a path where that's not used and makes a demonstrable difference

I can add a scenario to the ticket where it can show up -- using maps or sets as keys in array-map's, and the maps/sets you look up or assoc on are identical to the ones added earlier.

also, the equiv() methods that lack those identical fast paths are probably over typical inlining code size limits, I would guess, but can investigate that further, later.

there's a few in the patch for https://clojure.atlassian.net/browse/CLJ-1792

Hacky, crappy test

seems to show in profiling

(I would write something better for an “official” Jira thing…)

on the long case - with array-map, I see the all the action being in the equiv() call

Regarding method/function call counts on the JVM, I know that JVMs often make decisions on which methods to JIT based upon call counts, at least as one of the deciding factors. Do Flight Recorder and/or other perf tools make those call counts visible, without having to modify the code?

playing with NaN

(= ##NaN ##NaN)
=> false
(not= ##NaN ##NaN)
=> false
(apply = [##NaN ##NaN])
=> false
(apply not= [##NaN ##NaN])
=> true

Yes

The jvm itself can do some of that