Hi - my first question to this channel, hope you all are doing well. I am processing some CSV input - fairly large file with four types of records in them. All records are grouped with a header line and they are separated with blank (empty) lines. I have come up with the following attempt to use Specter for the transformation:
(let [data (->>
;; my data comes in as text from a csv file
(csv/parse-csv (slurp "some-file-name.csv")
;; group by number of elements and remove group of empty rows
(dissoc (group-by count csv) 1)
;; remove trailing empty cells within records
(setval [MAP-VALS ALL LAST empty?] NONE)
;; prepare fields that will become map keys
(transform [MAP-VALS FIRST ALL] keify)
;; turn the groups into maps
(transform [MAP-VALS] sc/mappify)
;; lastly, read our data fields
(transform [MAP-VALS ALL ALL] read-field)
)]
(def a-records (get data 6))
(def b-records (get data 7))
(def c-records (get data 11))
(def d-records (get data 15)))
While this works for my immediate needs, I find myself navigating to MAP-VALS over and over again, thinking this can probably be done in a more elegant way. Is there a way in Specter to group the four transformations (setval + 3 transforms) into a sub-select of some sort? Should I even care about this type of optimization?@marcus165 There is multi-transform with multi-path to group setval and transform together.
(multi-transform [MAP-VALS (multi-path [ALL LAST empty? (terminal-val NONE)]
[FIRST ALL (terminal keify)]
[(terminal sc/mappify)]
[ALL ALL (terminal read-file)])])
awesome - thank you @drowsy!