Clojurians Log v2

I've been pretty deep in this rabbit whole now, and looked at the code and all as well. And I think @andy.fingerhut is correct actually. Nil is not the empty list in Clojure, that's a departure explained here: https://clojure.org/reference/lazy but it used to be. And the code that defaults conj on nil to an empty list is 14 years old, I think it predates the switch to lazy-seq even, and it was written when conj actually took the coll last same as cons: (conj 1 []). At this point, I think its just what it is, 14+ years ago Rich just chose to default (conj nil 1) to use a PersistentList, and that's that. And much more recently, for transducers he chose to default the init for conj to vector.

My guess is that closer to Clojure's inception, Common Lisp idioms were often adopted, so big emphasis on lists and thus prepended elements when building them up, as well as pervasive nil-punning, thus treating empty list as nil and thus nil as empty list. So the original implementation of Clojure seem to behave consistently to that. Later, Rich started to realize that those CL idioms might not be ideal, and slowly started to break away from some of the CL idioms. The first big change was with making nil and empty list no longer the same thing. This is why sequences now don't return nil when they are empty, they return an empty construct of some sort instead, either an empty list, an empty sequence, or something of that sort. Now a seq is still either at least one element or nil. But a sequence can be empty, and a list can be empty. Even later, I think Rich started to realize that lists maybe just aren't even a very useful collection for user programs (even if useful for modeling code). So when implementing collections, vectors were used over lists for defaults.

Only Rich could corroborate this, but that's my best guess right now.

In practice that does seem to mean that collection functions when passed nil as the coll seem to default to treating it like a list, while sequence functions when passed nil seem to default to sequences, and transducers when not passed an init coll seem to default to vector.

In that sense, things are consistent, kind of like @p-himik was saying:

(conj nil 1) ;> (1)
(into nil [1]) ;> (1)
(conj) ;> []
(into) ;> []

(pop '()) ;> java.lang.IllegalStateException: Can't pop empty list
(pop nil) ;> nil
(empty '()) ;> ()
(empty nil) ;> nil

The default for no-arg conj to return [] is as old as (conj nil x) returning (), I believe. What evidence do you have that it dates to when transducers were added @didibus ?

If you haven't watched this talk, or read the transcript, I would recommend it, if you are interested in how early some of these decisions were made (the talk doesn't explain all decisions, just some). https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureIntroForLispProgrammers.md. It might help eliminate some of the guessing.

Hum, the code blame attributes it to the "transducer wip" commit. Maybe it was there before somehow as well, I didn't go further back in the history.

Yup, if I go further back in history, conj does not have a 0-arity, only the 2-ary.

This is the source of conj before the transducers commit:

(fn ^:static conj 
        ([coll x] (. clojure.lang.RT (conj coll x)))
        ([coll x & xs]
         (if xs
           (recur (conj coll x) (first xs) (next xs))
           (conj coll x)))))

how likely is a collision on (hash (fn []))?

I see it now. Yes, 2014, "transducers wip" commit.

And the RT conj impl is the one that has the nil handling code:

static public IPersistentCollection conj(IPersistentCollection coll, Object x){
	if(coll == null)
		return new PersistentList(x);
	return coll.cons(x);
}

That has been there since 1.0 and probably since beginning.

Yup, since September 2007 it seems, before that it used to be:

static public IPersistentCollection conj(Object x, IPersistentCollection y) {
	if(y == null)
		return new PersistentList(x);
	return y.cons(x);
}

Which is interesting, the coll came as the second arg

Are you interested academically, or are you planning on any part of your system relying upon that likelihood of collision? Depending on your goals, I'd recommend trying to avoid relying upon that likelihood of collision.

Oh interesting, I think Rich copy/pasted the code for cons, and renamed it conj. And then, in the next commit he changed the impl, that's why at first it has the arg order of cons.

I built a system on it being unique and now it looks like it collides pretty frequently in practice

Relying on hash codes being unique is almost always a bad idea.

Unless you are talking about UUID's or something, with so many bits that are uniformly distributed that you can prove miniscule probabilities of collision.

With 32 bits of hash result, if they were uniformly randomly distributed, the so-called birthday paradox predicts about a 50% chance of at least one collision if you get up to 2^16 = 65,536 of them.

I have no reason to think that hash on Clojure functions is uniformly randomly distributed, though.

user=> (count (into #{} (take 10000 (repeatedly #(hash (fn [] ))))))
9988

yeah hmm i'll switch to a UUID of some sort

There are 2 collisions already.

These hashes are not cryptographically secure

collisions are frequent, and the output is 32-bit

Actually, I wonder if that's why conj defaults to emptylist, since cons did, which made sense, since cons constructs a list, and the code was copy/pasted from it and already had that implementation, so I think that's what happened 😛

always assume hashCode returns 1 unconditionally

but map lookups are done via magic

Promise is definitely thread safe

But both of the implementations I'm seeing here don't seem to be thread safe

But it's not clear what you mean by thread safe exactly. Like your code can have two callers race for who gets to set the value first

And that's true with deliver on a promise as well

Like I think you're trying to only run real-compute once?

commenting out wrap-csrf gives a response of 500

I expect a response of 2xx

thanks. The 500 was due to an error in the handler

now it works

Oh wow, I missed that ClojureCLR now supports running on .Net Core, that's a big achievement. Seems everything works except AOT, and its up with Clojure 1.10 in featureset. https://groups.google.com/g/clojure-clr/c/MdxEbo0znAw

yup... i'm happy that there's an effort on this.. i tried the repl, it works fine so far... hope the rest (compilation, etc) will follow... definetly watching this space :-)

Ya, its been a while since I've used the .net ecosystem, but blazor and xamarin seem nice, and compilation to web assembly. So if ClojureCLR can get that working it could be interesting.

yup... i'm learning F# on the side also... in fact, i have both fsharp and clojure slack channels opened.. hehe.. i hope both communities interact more and share... there's bolero for F# (to do blazor stuff).. maybe clojureclr could leverage that as well

Would you say this can be a normal scenario?

I don't think I have a leak because all my app's objects/classes get eventually GCed. But I'm not sure if it's intended usage to have so many classloaders around

are you using nrepl?

yes

can't remember the details but i think i remember that nrepl executions can result in new classloader objects. if you can compare against an instance using a socket server or just no repl at all you might see a large difference

Thanks for the pointer! It lead me to https://github.com/nrepl/nrepl/issues/8 which looks related, as I'm using [org.clojure/tools.nrepl "0.2.13"] which predates that issue (that dep is outdated, I know)

I am a bit confused to why I cannot use var in this function: (defn fname [f] (-> f var meta :name)) I understand the reader macro would not work (since it would be quoting the literal symbol in that ns?) and guess that it is because var is a special form that it fails, but why? Is the effect the same as if I would have used the reader macro?

var is a special form that works more like a macro than a function. (var f) also tries to look up f, not the value of f

you probably want something like ns-resolve or resolve

although that will only work if you have the actual symbol at some point

ns-resolve is very weird, but for whatever reason people always seem to grab that first rather than the simpler resolve

I have a vector of functions (a middleware stack) and I want to make a simple tapper that tap>s the context after each middleware, function name included, but maybe they have to be quoted to be resolvable?

I remember that someone was able to obtain a reasonable name from a function using a snippet they found in clojure's spec source code, but I can't seem to find it

yeah, resolve needs a symbol, clojure doesn't really encourage function / symbol cross translation

maybe it was:

(defn- fn-sym [^Object f]
  (let [[_ f-ns f-n] (re-matches #"(.*)\$(.*?)(__[0-9]+)?" (.. f getClass getName))]
    ;; check for anonymous function
    (when (not= "fn" f-n)
      (symbol (clojure.lang.Compiler/demunge f-ns) (clojure.lang.Compiler/demunge f-n)))))

IMHO you can kind of make that work, but there's likely a much simpler solution that doesn't require turning functions into symbols

eg. providing the var rather than the function in the place where the middleware is added

agreed. it wouldn't be my first choice.

There is a hack like the above in the clojure.spec.alpha ns as well

oh it's literally that code ;)

yup, that's where I found it, https://github.com/clojure/spec.alpha/blob/ad06cdc7407c11990c7e93206133fb14eb62cacf/src/main/clojure/clojure/spec/alpha.clj#L131

I should have linked to it earlier, but I found it using jump to source in my editor before I could find it on github

@love.lagerkvist You can also attach metadata to clojure functions, or keep a map of fn obj -> name

oh, that a hack worthy of its name. @borkdude I guess that would be the saner choice, considering the number of functions is quite small. There is also (de-)munge I guess, not sure about the characteristics of that, but it sorta does what I want.

(require '[clojure.main :as m])

((fn [f] (-> f class str m/demunge)) util/qualified-keyed-ns-publics)
; => "class motform.bestå.util/qualified-keyed-ns-publics"

the fact that "munge" is an ugly word is intentional

I like recommending dirty hacks and appealing to authority at the same time 😈

the authorities are working under design constraints that probably don't apply to your project

if you only knew the power of the dark side