Clojurians Log v2

awesome, thank you

I see how the 10mb limit is a problem though...

firestore has a 1mb document limit even so it wouldn't be better for that setup

did you consider/try doing something more granular than a full db serialization?

like a conversion layer into firebase documents

We've considered doing something like making each entity a key in firebase and caching the entities on the client so we only have to load the entities that have changed but we would still have to deserialize the entities from local storage and transact them into a db, which I haven't tested but I think takes longer than deserializing a whole db and calling conn-from-db. Maybe this approach would be faster on slow connections, but I'm not sure. In other words I haven't figured out an architecture that would 100% work better than storing the whole db. And if the goal is to just have a db larger than 10mb, we can use firebase storage api to store the db, since we don't actually need the functionality of realtime database for the datascript db store.

gotcha

thanks for being so responsive and providing such a great sample

No problem, I've been wanting to write this up for awhile now 🙂

I'll see if I can get my head around it and maybe use something similar

btw I find roamresearch quite interesting! I've had something in the same problem space in my head for a while

so it's really cool to see how roamresearch and other tools are trying to improve note taking

worth mentioning that the Pixel phones (maybe others, not sure) have two cool things that help with the data input: you can talk to the recorder app and it'll transcribe, and you can take pictures of notes and it'll try to OCR the text, handwriting included

both of these work offline

Glad to see other people interested in this space as well! thats pretty cool, we've thought of integrating with http://otter.ai (really great transcribing app), didn't know phones come with things like that (I have an old galaxy phone haha)

I'm curious what your looking to use datascript for

similar thing as you guys have, but instead of focusing on the research field I was focusing on software engineers that wanted to do deep work

was looking at supplementing deep work with graph-based note taking with full-text search

a big driver was to have a persistent context between work sessions

and I wanted something firebase/datascript based because of the implicit transaction history

I think allowing users to replay how the last work session actually happened in the system would help get people back in the right context and mindset

It's awesome to meet someone with such a cross section of interests as ours 🙂. We originally wanted to use the transaction history to replay actions on a database (both to replicate actions that led to a bug and to show the user), but it hasn't really worked in practice with our setup because the db/ids can get out of sync between clients (we mostly have this solved now by using a unique identifier for every entity instead of the db/id, but now the logs are all messed up and can't be replayed)

yeah log changes often break replays in a lot of systems 😞