data-science

Data science, data analysis, and machine learning in Clojure https://scicloj.github.io/pages/chat_streams/ for additional discussions
niveauverleih 2020-12-06T14:22:50.112400Z

I am reading the tablecloth instructions https://scicloj.github.io/tablecloth/index.html I don't understand the use of :pre together with % in the following snippet: ... If you want to select :V2 values which are lower than or equal mean in grouped dataset you have to precalculate it using :pre. (-> DS (api/group-by :V4) (api/select-rows (fn [row] (<= (:V2 row) (:mean row))) {:pre {:mean #(tech.v3.datatype.functional/mean (% :V2))}}) (api/ungroup)) ... I thought :pre could only be used to validate input?

genmeblog 2020-12-06T16:39:40.116Z

@nick.romer :pre is a short name of precalculate or preprocess. In this example it injects precalculated mean of the :V2 column from given group to the row map. The true is I wanted to avoid this way of doing things but couldn't figure out how to achieve it without adding artificial column for each group.

niveauverleih 2020-12-06T17:11:47.118300Z

I think this is a good way of proceeding. However I couldn't find any documentation that suggested you could use the output of :pre . How did you know this would work?

niveauverleih 2020-12-06T17:56:59.119100Z

Actually, can't you use "let" for this?

genmeblog 2020-12-06T19:36:11.120200Z

For regular dataset - yes, you can use let. But for grouped dataset, you have to calculate mean for every subdataset.