data-science

Data science, data analysis, and machine learning in Clojure https://scicloj.github.io/pages/chat_streams/ for additional discussions
chrisn 2020-08-12T11:50:11.459200Z

We have a new post up exploring memory mapping and the new Apache Arrow data format via Clojure and https://github.com/techascent/tech.ml.dataset. A few times in my career I have used memory mapping and found it both simpler and faster than stream-based IO. Using it we can 'load' datasets far larger than physical RAM or load only 1 column/row out of many in a dataset without loading the rest. We hope you enjoy this simple demonstration! https://techascent.com/blog/memory-mapping-arrow.html

👍 12
💯 1
🚀 2
2020-08-12T11:55:33.459700Z

Thanks!