data-science

Data science, data analysis, and machine learning in Clojure https://scicloj.github.io/pages/chat_streams/ for additional discussions
2020-06-08T18:45:58.301300Z

@nick.romer You can also take a look at semantic-csv, which gives you some nice utilities for working with large csv files lazily, and composes with clojure.data.csv and friends. https://github.com/metasoarous/semantic-csv

2020-06-08T18:46:30.302Z

I agree though that if you can fit in memory, using <http://tech.ml|tech.ml>.dataset is likely the way to go

niveauverleih 2020-06-08T20:42:19.306500Z

@vlaaad @chris441 @metasoarous Thank you all! For the time being I was able to load some interesting columns like this: (defn reducer [ac row] (conj ac (map #(nth (first (csv/read-csv row)) %) [16 3 17 18]))) (def master (with-open [rdr (io/reader data-local)] (reduce reducer [] (line-seq rdr)))) I will have a look at tech-ml-dataset.

👍 1