Clojurians Log v2

Can someone help me understand why (setval (filterer even?) [:example] (range 0 10)) => (:example 1 3 5 7 9) whereas (setval (filterer even?) [] (range 0 10)) => (nil 1 3 5 7 9), yet (setval (filterer even?) [NONE] (range 0 10)) => (1 3 5 7 9)

being able to resize a subselection is really beautiful, but this special case of resizing it to zero seems a bit odd, and I dont quite understand why it happens

I would expect replacing a sequence with an empty sequence would remove all elements, rather than leaving a nil in the first matching spot

@npetryk looks like a bug

try (setval (filterer even?) nil (range 0 10))

fixed for next version: https://github.com/nathanmarz/specter/commit/7b646ca566bb3bdd362d43a348c2fe00457438b2

Hello, I again have a nested map in the form:

{... ...
 ’A {:type :double}
 :variables {... ...
             ’A {:type :int}
			 ’C {:type :double}
             :variables {... ...
			             ’B {:type :double
						     :formula “A + C”}
						 ... ...
			             }
		     ... ...}}

I also recursively traverse it and apply transformations to variables and other nodes (the map is an AST of a DSL)
						 

In ‘process-vars’ I have access only to the current node which works for most transformations I need. Now if in that DSL I would like to provide variable scope/visibility rules like in Java for example, where the localness of a variable is directly mapped to the depth of it’s definition over the :variables subbranches, I would need to have access not only to var, but to the root of the datastructure AFTER it has been processed in a previous step. That means I can’t just pass ast as a second parameter, because it would have the original state only before any transformation through Specter. collect I think would also not help me in that situation.

So 1. how one can pass not only the currently selected path element, but the whole structure as well in the already modified state on each invocation of the transform function? 2. if I would like that in the context of where variable B is definied, I would like to have access to variable A in the ast, then I should be able to get to the upper and recursively to all further upper levels up to the root from the current node transformed to access the variables defined there. I know how to do it in Clojure without specter, but how with Specter? (for example to pull the types of variables A and C for calculating B) 3. outside of Specter I have a lot of variable and other dependency analysis (essentially expressions/filters over topological sorts of the variables), but is there an idiomatic way to do it in Specter? Any hints would be very helpful

@plamen don't understand what you're asking about getting root of data structure after it's been processed

it would help if you simplified your examples to get at the heart of your questions

Hello Nathan, what I mean is that in the transform function I have as parameter only the current node in the tree as it gets recursed. I would like to pass in addition to it the whole structure so that I can walk it inside the transform function, but not in it’s inital state, but in it’s current (probably already processed state from previous steps in the recursion - my question is probably related to not knowing when exactly the transformation function is applied to the nodes - does first a walk happen in which paths get constructed, then the list is processed, or is it just as in the recursion - step by step depth first, or post-walk or…?) I hope this makes more sense.

(not sure how to make the question simpler as the map with nested :variables keys consisting of lists of other such maps is the simplest one I can think of related to it. This is why I make the analogy with an abstract syntax tree/depth in the tree and variable scope/visibility as in languages like Java/C#)

(it is actually not an AST, but more of a types tree)

so, in the (defn process-var-list [ast] (transform [NODES-VARIABLES :variables MAP-VALS] (fn [var] ; do something to var ) ast))

if I rewrite it to

(defn process-var [var] ;do something and return the transformed var node) (defn process-var-list [ast] (transform [NODES-VARIABLES :variables MAP-VALS] process-var ast))

what I mean is that in my understanding to achieve what I ask for in 1., passing ’ast as an additional argument to ’process-var would pass the inital state of ’ast before the recursion and possible transformation inside it happened?

what I need is that during the recursion, during the invocation from Specter of the function ’process-var, inside ’process-var I get as parameter the ’ast as processed up to there. (which is possible only in pre-walk, but not post-walk, but this is also the point - I don’t know how to control or know when the invocation of ’process-var actually happens)

(sorry - the “possibility” in the “pre-/post-walk” part of the last sentence is wrong)

Does that make more sense?

or in other words which is probably more visual - how to transform

{... ... ’A {:type :double} :variables {... ... ’A {:type :int} ’C {:type :double} :variables {... ... ’B {:type :double :formula “A + C”} ... ... } ... ...}}

into

{... ... ’A {:type :double} ’D {:type :string} :variables {... ... ’A {:type :int} ’C {:type :double} :variables {... ... ’B {:type :double :uses [[:variables ’A] [:variables ’B]] :sees [’D [:variables ’A] [:variables ’B]] :formula “A + C”} ... ... } ... ...}}

where the vectors in :uses and :sees are either a path in a map as in ’get-in or may be even a Specter path if that’s possible easy.

(this is more about 2. and 3.)

about question 1. another example

so when you process var 1 and then var 2, you want the var 2 transform fn to get the updated ast from processing var 1?

yes

exactly

as in

initial tree

{... ... ’A {:type :double} :variables {... ... ’A {} ’C {} :variables {... ... ’B {:type :double :formula “A + C”} ... ... } ... ...}}

when transforming ’A I can add {:type :int} to it

I do exactly that with my graph navigators

and in the next recursion step when I process ’B, the :type values of ’A and ’C are already visible to the invocation of the transform function

my navigators don't navigate directly to a node, but to a pair of [graph node-id]

I know how to do it outside of Specter, but specter gives me much more concise way

yes

so when I do TOPSORT, it navigates to [graph node-id] in topological order, with each step having the updates form previous steps

I believe you can do something similar for your use case with zippers

look at com.rpl.specter.zipper

in my normal clj code I also on each step have something like [root-node local-path-in-the-recursion]

exactlyt

-t

https://github.com/nathanmarz/specter/wiki/Using-Specter-With-Zippers

I even did’t tought about zippers, as in normal Clj I achieve it without them. Thank you for the big hint!

zippers are an advanced form of navigation

so when I do TOPSORT, it navigates to [graph node-id] in topological order, with each step having the updates form previous step - was exactly what I was looking it

it=at

more indirect but can do things you can't with regular navigators

at the cost of increased overhead

will read

yes - the overhead is not that important for me as it will be used in a compilation phase, where not the compilation, but the runtime is important

while the topological order is important for me, because without it - no chance for correct code generation. but needed to ask first about step 1 and then to think how to overlay the order on top of it in Specter.

reading the doc it looks to me that I have to solve couple of things (and rewire the brain a bit for using zippers)

1. I need to create a specter/zip constructor for nested maps and vectors which can be used from Specter

2. not sure if I read it correctly from the examples, but all S/transform invoke a function with the node itself, but not with the zipper, so how would I navigate back to an upper level if the transform function (in the examples - inc) receives only the current value?

what I mean - from the examples the line (S/transform [SZ/VECTOR-ZIP SZ/DOWN SZ/NODE] inc data) invokes ’inc only with the value of the current node, but how would one consult upper nodes from inside ’inc?

3. not sure yet how to do the recursion as the zip paths are to specific ‘coordinates’. I could using normal clojure recursion where on each node I construct a new zipper and walk/transform from there. Is this the right line of tought?

yea, on second thought zippers won't be enough for what you want to do

you need a custom navigator that understands the data structure as a whole

to do something like the TOPSORT I described

for reference, my TOPSORT code looks like this:

(defn- multi-anchor-path-select [graph getter next-fn]
  (doseqres NONE [anchor (getter graph)]
    (next-fn [graph anchor])))


(defn- multi-anchor-path-transform [graph getter next-fn]
  (reduce (fn [graph node-id]
            (-> [graph node-id] next-fn first))
          graph
          (getter graph)))

(defnav multi-anchor-path [getter]
  (select* [this graph next-fn]
    (multi-anchor-path-select graph getter next-fn))
  (transform* [this graph next-fn]
    (multi-anchor-path-transform graph getter next-fn)))

(def TOPSORT (multi-anchor-path topsort))

this is built on top of loom https://github.com/aysylu/loom

Thank you for the direction, Nathan! I tought it is something trivial 🙂 Now I need to rethink

And for taking time again for my questions!