datahike

https://datahike.io/, Join the conversation at https://discord.com/invite/kEBzMvb, history for this channel is available at https://clojurians.zulipchat.com/#narrow/stream/180378-slack-archive/topic/datahike
whilo 2020-05-14T07:00:32.164800Z

@alekcz360 There is no minimum speed requirement, but it would be nice for users to know some performance metrics of course. Something we should provide in konserve itself including plotting.

whilo 2020-05-14T07:08:09.167900Z

The versioning is to be able to do online migration, i.e. when you decide to change the storage format you can detect that before you read an old value and trigger a migration. This is an internal design decision, but it has proven necessary, since otherwise you need to manually detect the version which might require more IO. If you prefer to do offline migration, e.g. to not violate SLAs on key access, you can just read all keys.

whilo 2020-05-14T07:09:22.168600Z

For the file store we currently encode the version in the first byte.

alekcz 2020-05-14T09:56:17.168700Z

I'm thinking something needs to be in the test suite so that if someone refactors something they aren't going to impact application performance downstream or in datahike itself

alekcz 2020-05-14T19:24:35.169100Z

I can't seem to find it in the filestore.clj. What line does it happen on?

whilo 2020-05-14T19:36:28.169600Z

I think it is not pushed yet. Ferdi is working on it again on Friday, hopefully we can get it pushed until next week, we will see.

whilo 2020-05-14T19:39:39.169800Z

Konserve can be used without Datahike in different scenarios, some stores, such as S3, might be slow by their very nature, yet still valuable.

alekcz 2020-05-14T20:31:25.170Z

Cool cool. Thanks.