Clojurians Log v2

what is the more elegant way to extract a value from a nested data structure and then apply a transform to it?

@ben.mumford620 uploaded a file: https://clojurians.slack.com/files/U8QHZL09E/FB4KLMB40/-.txt

my use case is that i have lots of different types of input data structures and i want to normalise them so that i can compose and compare them

so what i'm doing is for each input object creating a new output object where each field is calculated using a select then a transform (so i guess question 2 is does this make specter a good fit for my use case as i'm not modifying a pre-existing data structure)

@ben.mumford620 you can use view or transformed at the end of the path to the select operation

Would spectre be the 'natural/goto' thing for taking a nested data structure and a 'key' and replace any occurance of it with a value? Where 'key' would most probably be literally a keyword (or possibly string)?

@jsa-aerial yes, that's a bread and butter use case

walker is the lazy way to do that, but if your data has any structure to it whatsoever it's better to make a precise path

much higher performance + much less bug prone

@nathanmarz OK, sounds promising. The data will have quite a lot of structure and the 'paths' (there will typically be several such 'keys') will basically not be known ahead of time. So, this basically involves 'search' and replace. Why would non precise path and walker tend to be more 'bug prone'??

walker will descend everywhere including places you don't expect, like records, map keys, etc.

Actually, ALL almost looks like exactly what is needed here. Except it returns [k v] for map elements

here's a variation of walker that doesn't descend into map keys / key/value pairs

(def my-walker
  (recursive-path [afn] p
    (cond-path (pred afn) STAY
               map? [MAP-VALS p]
               coll? [ALL p]
               )))

if your data is truly completely unstructured, then walker is more appropriate

though even then, it's better to make a variation that doesn't navigate to key/value pairs

e.g.

(def my-walker
  (recursive-path [afn] p
    (cond-path (pred afn) STAY
               map? [(multi-path MAP-VALS MAP-KEYS) p]
               coll? [ALL p]
               )))

I think we have a semantic issue here - you are using 'structured' in a special way - not just to indicate a lot of structure in the data

by structure I mean that the organization of the data is precise and well-understood

Something like this {:a ::some-key ....} can occur and should become {:a <the-value-for-::some-key> ...}

It's precise, it is just not fully known up front. And these 'keys' can occur in all sorts of places. Basically, these structures will be "parameterized" vega-lite specifications.

if the paths to what you care about aren't known in advance, then I would call it unstructured

if you know it will never occur in a map key, then it's best your path reflect that

or if you know any other constraints, those should be included in the path

OK, that is fine - now I know what you mean by that, it would seem walker is the way to go

Yeah, those sorts of constraints and assumptions cannot be relied upon in this sort of scenario.

There are constraints and requirements in vega-lite data specs, but here the idea is to use a basic layout as a template which will have these 'keys' replaced by values for specific cases and these 'keys' could be standing in for both VL keys and/or values.

Hmmmm ALL maybe could still work if it is just called recursively on any [k v] pair returned

is it generally frowned upon to use setval inside of a transform-fn in a transform call?