clojure-dev

Issues: https://clojure.atlassian.net/browse/CLJ | Guide: https://insideclojure.org/2015/05/01/contributing-clojure/
souenzzo 2019-07-19T01:29:36.001100Z

After read tons of clojure source, I still can't understand this behavior.

(= ##NaN ##NaN)
=> false
(not= ##NaN ##NaN)
=> false
(apply = [##NaN ##NaN])
=> false
(apply not= [##NaN ##NaN])
=> true

2019-07-19T01:36:19.001700Z

You are curious what in the Clojure/Java implementation causes that to happen?

2019-07-19T01:42:38.002600Z

Hmm, now that you ask, I don't know if I have noticed that (not= ##NaN ##NaN) returns false before. That is pretty weird.

alexmiller 2019-07-19T01:43:39.003100Z

it's the weird thing about ##NaN that causes all the problems :)

alexmiller 2019-07-19T01:44:37.003600Z

##NaN is identical but not equals, which is very rare in Clojure

alexmiller 2019-07-19T01:44:47.003800Z

user=> (identical? ##NaN ##NaN)
true
user=> (= ##NaN ##NaN)
false

2019-07-19T01:46:05.004500Z

But does not= make sense to you? I am curious enough to dig for a few minutes and find out, but ... weird

2019-07-19T01:47:07.005400Z

It is probably switching between primitive double vs. Object equiv methods in clojure.lang.Util there, is my best guess before experiments confirm

alexmiller 2019-07-19T01:47:26.005700Z

does seem weird that the not= and apply not= are different

alexmiller 2019-07-19T01:48:13.006100Z

user=> (apply not= [##NaN ##NaN])
true
user=> (apply not= ##NaN [##NaN])
true
user=> (apply not= ##NaN ##NaN [])
false

alexmiller 2019-07-19T01:48:49.006300Z

maybe something weird with the rest seq

alexmiller 2019-07-19T01:50:30.006900Z

or inlining? = is inlined, not= is not

2019-07-19T01:50:35.007100Z

user=> (#'= ##NaN ##NaN)
true
user=> (= ##NaN ##NaN)
equiv(double NaN, double NaN) returning false
false

2019-07-19T01:50:56.007600Z

That equiv(...) ... line is extra debug print in my local Clojure Java code.

alexmiller 2019-07-19T01:51:18.007900Z

if you go through the var, probably not getting the inlining

alexmiller 2019-07-19T01:52:54.008200Z

prob need to compare the compiled bytecode

2019-07-19T01:53:45.009Z

I can make a table for Clojure 1.10.1 of the cases and stick them up on a page somewhere, for the morbidly curious

alexmiller 2019-07-19T01:53:46.009100Z

oh, maybe primitive long vs boxed long?

alexmiller 2019-07-19T01:54:12.010Z

the inlined will probably be primitives whereas non-inlined is boxed

2019-07-19T01:54:16.010200Z

I think (#'= ##NaN ##NaN) and (not= ##NaN ##NaN) go through equiv(Object, Object) method

alexmiller 2019-07-19T01:54:20.010400Z

same for apply

2019-07-19T01:55:23.010800Z

Obviously the person creating such a page is among the morbid 🙂

alexmiller 2019-07-19T01:56:22.011100Z

or sorry, doubles

alexmiller 2019-07-19T01:58:13.011500Z

thankfully jshell to the rescue...

alexmiller 2019-07-19T01:58:18.011700Z

jshell> Double.NaN == Double.NaN
$2 ==> false

jshell> Double.valueOf(Double.NaN).equals(Double.valueOf(Double.NaN))
$3 ==> true

alexmiller 2019-07-19T01:58:34.012Z

prims are false, boxed are true

alexmiller 2019-07-19T02:00:33.013100Z

pretty likely there is some place that's not handled right

2019-07-19T02:01:29.013600Z

I think the equality guide's recommendation of avoiding NaN's is unlikely to change as a result of any of this 🙂

alexmiller 2019-07-19T02:02:49.013800Z

same issue as -0.0 and 0.0 iirc

alexmiller 2019-07-19T02:03:28.014Z

jshell> -0.0 == 0.0
$5 ==> true

jshell> Double.valueOf(-0.0).equals(Double.valueOf(0.0))
$6 ==> false

alexmiller 2019-07-19T02:03:33.014200Z

thx java

alexmiller 2019-07-19T02:04:21.014600Z

to be fair, both of these are documented in Double.equals() - https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Double.html#equals(java.lang.Object)

alexmiller 2019-07-19T02:04:52.014900Z

"This definition allows hash tables to operate properly." orly?

2019-07-19T17:35:45.022900Z

does negative zero equal positive zero in some other context?

2019-07-19T17:55:29.023800Z

I think Java == on primitive doubles is defined to return true in that case?

2019-07-19T17:55:41.024Z

because IEEE 754 requires it, perhaps?

2019-07-19T17:56:28.024200Z

Here is reference to what Java equals and == do: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Double.html#equals(java.lang.Object)

2019-07-19T17:57:16.024400Z

And here is a mention in the Java language spec that says IEEE 754 requires +0.0 == -0.0 to return true: https://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.21.1

2019-07-19T18:15:11.024800Z

clojure.core/= and clojure.core/== return true when comparing +0.0 and -0.0 in the latest Clojure (that was a change in the last few years as a response to a filed issue)

2019-07-19T18:19:15.025Z

Not having done so, I have the impression that reading the IEEE 754 spec, and rationale if one exists, would lead one to believe that there were many heated discussions during its creation process.

alexmiller 2019-07-19T18:21:44.025200Z

IEEE 754 is trying to solve many use cases, some where +/- 0 is meaningfully, some where it's not. unsurprisingly, there are tradeoffs. :)

2019-07-19T18:23:18.025400Z

Oh, I'm immensely glad the standard was created. I can easily imagine the frustrated state of numerical software developers before it existed, trying to port software.

alexmiller 2019-07-19T18:23:36.025600Z

https://en.wikipedia.org/wiki/Signed_zero

2019-07-19T18:25:47.026Z

This is a gem I haven't seen before: https://en.wikipedia.org/wiki/IEEE_754#Total-ordering_predicate

alexmiller 2019-07-19T18:26:23.026300Z

as I'm sure you are both well aware, most things are weirder when examined closely :)

2019-07-19T18:30:29.026500Z

Recently read the first chapter of the book "Microbe Hunters" describing early days of Leeuwenhoek making the best microscopes of his day and spending most of his life looking at all sorts of things. Excellent book so far.

alexmiller 2019-07-19T18:31:17.026700Z

was he the guy that stabbed himself in the eye with a needle to figure out how light worked? or was that someone else

alexmiller 2019-07-19T18:32:00.026900Z

that was Newton, nvm

alexmiller 2019-07-19T18:32:27.027100Z

also, don't ever google that

2019-07-19T18:40:28.027300Z

That is a level of scientific curiosity I don't think I will personally reach.

2019-07-19T23:09:50.028800Z

it's also unclear to me what theories you rule out after doing that

alexmiller 2019-07-19T02:05:12.015100Z

just tease us

alexmiller 2019-07-19T02:05:43.015400Z

anyhow, ticket welcome

2019-07-19T02:06:41.016300Z

I guess I can create a ticket at least listing the current behavior and the root causes. Personally I'm not expecting high priority bugs out of those results, but that hasn't stopped me from creating tickets before 🙂

2019-07-19T02:16:23.017200Z

I think this should go into a Github repository named Batman -- hat tip to Gary Bernhardt's video https://www.destroyallsoftware.com/talks/wat

2019-07-19T03:03:14.017700Z

what the?

2019-07-19T03:03:15.017800Z

(not= ##NaN
      ##NaN)
=> false
(not= Double/NaN
      Double/NaN)
=> true

2019-07-19T03:20:20.018600Z

Thanks for the examples. I will be adding them to my list of expressions to explain their results.

2019-07-19T03:20:30.018900Z

will send link to the results here when they are published.

2019-07-19T07:21:23.020100Z

OK, that might not have been the best way to spend that many hours of my life, but here are the results of a bunch of Clojure expressions involving NaN, most with explanations for why they return the result they do, based upon adding some debug print statements to Clojure 1.10.1 source code and then evaluating the expressions: https://github.com/jafingerhut/batman

💯 1
2019-07-19T07:21:52.020700Z

Sample output of running the command in the doc directory, in case you just want to look through them: https://github.com/jafingerhut/batman/blob/master/doc/ubuntu-18.04.2-jdk-openjdk-11.0.3-clojure-1.10.1.txt

2019-07-19T15:52:51.022500Z

@alexmiller It is not clear to me from that fun collection of results about = and not= on various flavors of NaN, called in different ways, ought to do. Consistently return false when comparing NaN's for =, and consistently return true when comparing them for not=? I guess that could be a semi-reasonable desire.

alexmiller 2019-07-19T17:30:13.022800Z

Yes

2019-07-19T17:54:51.023700Z

OK, PR created for the few NaN weird cases found: https://clojure.atlassian.net/browse/CLJ-2526

alexmiller 2019-07-19T18:08:01.024700Z

thx

2019-07-19T20:57:47.028700Z

OK, also created a ticket with the potential speed improvement for identical keys in array-maps: https://clojure.atlassian.net/browse/CLJ-2527 The improvement could affect anything that uses EquivPred -- I will do a quick check of the code to see whether it is used for anything other than array-map key lookup.