Clojurians Log v2

@whilo two questions. 1. Should the first byte of the meta also contain the version? 2. Why don't we store the version konserve version in the meta?

One more question @whilo in string stores (ser/string-serializer) do we store the version as U+0001 or "1"

@alekcz360 The version is describing the layout of the binary object and therefore should be attached to each object. It is also independent of the serializer, because we need to determine whether we need to migrate an object before we want to read it. So it should always be a binary byte, I think. (Unless you see problems.)

@whilo lemme just make sure I understand.

;in a store where meta and data can be separated but stored atomically
{:data <version-as-byte + serialized-data>
 :meta <serialized-meta>}

;in a store that requires all data in one location
{:data <version-as-byte + meta-length + serialized-meta + serialized-data>}

I would prepend the version-as-byte also to the metadata.

get-meta only touches it and it needs to know how to read it.

Awesome. I'll be done with carmine and clutch tomorrow.

Very good, you are faster than we are 🙂. We will hopefully be done with the filestore (migration from our old formats is currently finished) and leveldb stores until Monday. I hope we can put out a konserve beta with the garbage collector bits next week and start some experiments for online GC on Datahike after that.