Clojurians Log v2

I’m playing with Meander. It looks like a useful tool to transform arbitrary data. So now I’m trying to transform this JSON data with a deeply nested structure. There are many optional keys in the map. I wonder what’s the best way to describe this The following transforms the data recursively, but I don’t how to solve the optional keys. With & I can catch the rest, but I probably need to use this to capture optional keys

(require '[meander.strategy.epsilon :as m*])

(def rewrite-child
  (m*/rewrite 
    {"title".      !node-title
     "create-time" !create-time
     "children" !node-children

     & !others 
     }
    {:create/email !create-email
     :create/time !create-time
     :node/title !node-title
     :node/children !node-children
     
     & 
     })

(def rewrite-children
      (m*/bottom-up
       (m*/attempt rewrite-child)))    

{"create-time" 1582030610649,
 "title" "<https://en.wikipedia.org/wiki/Consciousness>",
 "children" [{"create-time" 1582030610649,
              "string" "<https://en.wikipedia.org/wiki/Consciousness>"}]}
=&gt; 
{:create/time 1582030610649,
 :node/title "<https://en.wikipedia.org/wiki/Consciousness>",
 :node/children [{:create/time 1582030610649,
                  :node/string "<https://en.wikipedia.org/wiki/Consciousness>"}]}

{:create/time 1582030610649,
 :node/title "<https://en.wikipedia.org/wiki/Consciousness>",
 :node/children [{:create/time 1582030610649,
                  :node/title nil
                  :node/childeren nil}]} 

I'll have to do some thinking about how I'd do that. Are there certain keys that are optional and you don't want them if they aren't there? Or all the keys?

Yah more detail would be helpful. I can also jump on later today and lend a hand. 👍

Thanks! I’m trying to import a json file. The data structure in the file has an a recursive structure of variable depth. The first level has slightly different keys than the child levels. And there are some special childs with others fields. I guess I could write multiple rewrite rules. I’m wondering what the best way would be to compose that. I’ll add a snippet of the import file to give you an idea

This is the file. Eventually I’m trying to put it in a Datomic database by flattening the structure, but my first concern is mapping the right fields

So if I could combine at least 3 rewrite rules with attempt and bottom-up it could work I guess. Maybe attempt is composable? I’ll give it a try

I’m try rewrite with multiple rules now, one for each map type. But it seems the first one is always chosen. I had the understanding that ? would require the match to be non-nil. I’m probably missing something. Will try more

(m/some ?x) is what you want to use for requiring something to be non-nil. Will take a look at this later tonight. Sorry can't get to it earlier.

np, thank you!

Ok maybe multiple rewrite rules did work

Now I’m wondering if I can do transformation on values. E.g. I would like to change timestamps to an instant. I could do a postwalk over the data, but maybe this is something Meander can do as well, saving another data iteration