Clojurians Log v2

@nathanmarz apropos our conversation about warnings under clojure 1.9.0, a new version of midje has been released with an updated (direct) dep on specter:

[midje "1.9.1" :scope "test"]
   [com.rpl/specter "1.0.4" :scope "test" :exclusions [[org.clojure/clojure] [org.clojure/clojurescript]]]
     [riddley "0.1.12" :scope "test"]

@mbjarland great

Hi. Don't know if this is the right forum, I have a newbie specter question.

I wanted to filter out elements not matching the path for a transform call.

It's easier to explain in code. The following (specter/transform [specter/ALL (specter/must :a) (specter/must :b)] str [{:a {:b 10}} {:a {:z 42}}]) returns [{:a {:b "10"}} {:a {:z 42}}]

It called the transformation function for an element matching the path and returned the rest unchanged.

I'm wondering if is there anything I can do to make a similar call return [{:a {:b "10"}}] instead?

(filtering out the non-matching {:a {:b 10}} element)

@rafael if you want to do a select and a transform in the same operation, you probably want to do select to get only the value you want returned and then use the view selector to make your transformation (https://github.com/nathanmarz/specter/wiki/List-of-Navigators#view)

i.e. how I think of it: transform is for modifying in place a piece of a structure; select is for pulling out specific information I want from a structure; and select with the view selector is for doing both selecting only a piece and applying a transformation

Thanks @tanzoniteblack. I'm trying it out now

Naively applying view to the example above gets me:

(specter/select [specter/ALL (specter/must :a) (specter/must :b) (specter/view str)] [{:a {:b 10}} {:a {:z 42}}])

=> ["10"]

I wonder if is there a way to get [{:a {:b "10"}}] in the result.

https://github.com/nathanmarz/specter/wiki/List-of-Navigators#transformed might do that?

It would perfectly fine to chain a call to select and a another to transform, or whatever, but even then I'm having a hard time coming up with the right incantation 🙂

not actually sure off the top of my head, sorry

No worries, thanks for the help. I'm checking out transformed now

(specter/select [(specter/filterer (specter/must :a) (specter/must :b)) 
                                            specter/ALL
                                            (specter/transformed [:a :b] str)]
                                           [{:a {:b 10}} {:a {:z 42}}])

filterer removes all items that don't have :b key inside the value at the :a key, and then for each item in that list, we're going to reach into the path :a then :b and run the function str

probably not more efficient then the manual way of doing it? But no idea

The output looks right. I'm trying to digest it before applying to my real use case (which deals with a structure a bit more complicated than the example). Efficiency is not super important for my use case, I'm reaching for specter for the terseness in dealing with a deeply nested structure, so it should be alright

@tanzoniteblack you're looking for not-selected?

(setval [ALL (not-selected? (must :a :b))] NONE data)

With not-selected and setval the complete example would then be:

(->> data
       (S/setval [S/ALL (S/not-selected? (S/must :a :b))] S/NONE)
       (S/transform [S/ALL (S/must :a :b)] str))

yea, that would work

for second part you don't need must anymore since they're guaranteed to be there by first part

Cool, now off to apply it to the real use case 🙂

Hi again. When applying to my actual use case things turned out to be more complicated. There are multiple S/ALL where we want to remove non-matching elements. Again, a code example is probably easier to explain:

(def data [{:a [{:b 10}]} {:a [{:b 20} {:z 42}]}])

  (->> data
       (S/setval [S/ALL (S/not-selected? (S/must :a) S/ALL (S/must :b))] S/NONE)
       (S/transform [S/ALL :a S/ALL :b] str))

=> [{:a [{:b "10"}]} {:a [{:b "20"} {:z 42, :b ""}]}]

(it's a similar example to above, but with one extra nested sequence)

I could get the output I wanted with an extra setval:

(->> data
       (S/setval [S/ALL (S/not-selected? (S/must :a) S/ALL (S/must :b))] S/NONE)
       (S/setval [S/ALL (S/must :a) S/ALL (S/not-selected? (S/must :b))] S/NONE)
       (S/transform [S/ALL :a S/ALL :b] str))

This works, but I was wondering if there is any way of reducing the duplication

@rafael (setval [ALL :a ALL (transformed (must :b) str) (not-selected? (must :b))] NONE data)

do you also want to remove top level maps that don't have any inner maps under :a with :b?

Ideally yeah, If data is (def data [{:a [{:b 10}]} {:a [{:b 20} {:z 42}]} {:a [{:z 42}]}]) then I also need to remove the last {:a [{:z 42}]}

(trying to understand your suggestion above... thank's for taking the time to help)

@rafael this does the trick:

(defdynamicnav with-matching [path]
  (if-path path
    path
    (terminal-val NONE)
    ))

(transform
  [ALL
   (with-matching (must :a))
   ALL
   (with-matching (must :b))
   ]
  str
  data)

Wow

even better:

(defdynamicnav with-matching [path]
  (if-path path
    (multi-path path (if-path path STOP (terminal-val NONE)))
    (terminal-val NONE)
    ))

(defdynamicnav with-non-empty [path]
  (multi-path path (if-path empty? (terminal-val NONE)))
  )

(transform
  [ALL
   (with-matching (must :a))
   (with-non-empty ALL)
   (with-matching (must :b))
   ]
  str
  data)

with [{:a [{:b 10}]} {:a [{:q 20} {:z 42}]}] that removes the second map completely

you can read about dynamic navs here: https://github.com/nathanmarz/specter/wiki/Specter's-inline-caching-implementation

they're similar to macros

Looks like I have a lot of reading to do. Thanks

actually this is a more flexible implementation I think:

(defdynamicnav with-matching [path]
  (if-path path
    (multi-path path (if-path path STOP (terminal-val NONE)))
    (terminal-val NONE)
    ))

(defn ^:direct-nav ensure-pred [pred-fn]
  (path (stay-then-continue (if-path (pred pred-fn) STOP (terminal-val NONE)))))

(transform
  [ALL
   (with-matching (must :a))
   (ensure-pred #(-> % empty? not))
   ALL
   (with-matching (must :b))
   ]
  str
  data)

I may add some variant of these into specter

I think this is the simplest solution:

(defdynamicnav ensure-matching* [path]
  (multi-path path (if-path path STOP (terminal-val NONE))))

(def ensure-matching (eachnav ensure-matching*))
    
(transform
  [ALL
   (ensure-matching (must :a) ALL (must :b))
   ]
  str
  data)