Clojurians Log v2

Got a more thorough version of the above working on an actual SharedArrayBuffer, in conjunction with the model. Still has an obvious bug that should be pretty easy to fix, but I've got a few tests for it now. But at the moment I'm exploring the idea of building versions of CLJS's persistent collections backed by this SAB.

But that brings up the problem of reference counting. Persistent objects on a SAB will share structure, so mutating one portion of the structure will only add the minimal difference to the SAB, but now we don't want to delete the old references, as this is no longer full copy-on-write.

So we need some kind of reference counting to know which workers currently have references to particular nodes in structures. However, how can we know where a user has passed the reference around to?

I'm not absolutely sure yet, but there's a new primitive in JS called weak references, in the form of WeakSet and WeakMap.

So, I'm thinking, every time a node is dereferenced off the SAB, we increment a ref-counter on that node and then we add that node's reference to a WeakMap with a ref-id-object (`#js {:id "ref-id"}`) as a key and the reference as a value - also adding the ref-id to a ref-id-set. Then a worker can do a cleanup phase by going through each reference in the ref-id-set and checking if the WeakMap still .has it. If not, nothing on this worker is still holding on to the reference, so we then decrement the ref-counter for that node on the SAB, indicating that this worker no longer has a reference to it. If the result of the dec equals 0, free the node from the SAB.

This way we're essentially leaning on the JS garbage collector to do our reference counting for us.

The downside of this approach (not doing full copy-on-write, with immediate auto-garbage-collection) is that there's no JS garbage collector in the WASM world yet and it's not clear that a JS collector will ever be there. The full copy-on-write algorithm is so simple that it could be implemented in fairly tractable WebAssembly and we could have JS and WASM processes operating on the same shared objects fairly seamlessly.

So, huh? JS is too slow to iterate over this matrix? Well, just do it in WASM, over the same exact matrix, and it'll get done faster. Then you can just dereference it on the JS side when it's done.

Though, I suppose if you're willing to sacrifice some semantic parity between the JS and WASM sides, the WASM side could be made to interoperate with a particular node in a transient context, without too much advanced machinery on the WASM side.

I'd have to learn more about the transient machinery to be sure though