Clojurians Log v2
datascript
Anyone else crying out for a persisting Datascript? I'm guessing IndexedDB is the sensible (if intimidating) API to build on and living with async queries would mean keeping a full history to provide "point in time" queries.
I'd also love it to persist in React Native which doesn't seem to provide IndexedDB.
@olivergeorge what direction are you leaning? In my exploration - AsyncStorage seems well supported but has a size limit. FS & SQLite have less support but fewer limits.
For now I’m using SQLlite.
AsyncStorage limits are solvable now by the way.
It doesn’t suit me because pulling everything into memory and deserializing is slow. Also the variety of access patterns I need makes it tricky to break things up into convenient documents.
SQL queries are friction but solvable
It’s just a shame that I can’t use the same techniques on the web app version of the same application
(holding large datasets in memory on mobile devices seems like a design flaw but I've not tested the impact on performance)
Is it not what datahike is about?
sqlite seems popular on mobile
@olivergeorge I don’t see much to be done tbh. I think that you could easily serialize the datascript db and put it in indexeddb or localstorage (depending on size / etc.)
but I think the flow would be to then pull out the whole db and deserialize to do queries
@olivergeorge This is exactly what Datahike is about. Only thing missing right now is a durable mode on the front end. Means rewriting the query engine to be async (since no blocking in JS), which is some work, but I believe the team is queueing that up soon.
@lilactown You can do what you are suggesting if you don't have too much data, but that way you still have to hold things in memory. What's being worked on would not be limited in this way.
it’s a tradeoff, but I’m struggling to think of a time when have had so much data that I would rather pay the performance cost of going to disk
I’m sure for some niche cases that’s true, but the majority of webapps have nowhere near those kinds of requirements, which is why I think the datomic-esque “load the db in memory and query it” makes more sense
you could probably do something similar to datomic where you cleverly store the EAVT values in indexeddb to allow an efficient as-of without duplicating data
my hunch is just that you don’t want to hit indexeddb for every query. I’m interested to know if I’m wrong!
Perhaps stupid question: can I browse a datascript database with REBL?
Open PR asks for this: https://github.com/tonsky/datascript/issues/313
Yeah it’s problematically slow to load given the data I have on mobile. Also holding everything in memory on mobile is a questionable.