Clojurians Log v2

anyone had any luck with quickly serializing/deserializing datascript dbs with lots of datoms?

@isak What have you tried? Have you seen datascript-transit? https://github.com/tonsky/datascript-transit

Or do you need to incremental updates to the serialization/deserialization?

@petterik I've tried that one (datascript-transit - seems to optimize for size rather than speed), fressian, and just got nippy to work with it. But not happy with the performance yet - takes several seconds to serialize just 30 mb of data. Curious if other people have looked into this and found something they are happy with.

running in which environment, for reference?

clojure on windows

What I did is to serialize everything, not just :eavt datoms, because I think it is too slow to have to sort many times when loading the data

how does datascript store the indices? are they sorted arrays or some other structure? it seems like a bulk-loading API would be useful.

It just uses this structure: https://github.com/tonsky/datascript/blob/master/src/datascript/btset.cljc#L5-L17 The constructors for these are public, so I just used those