Clojurians Log v2

How would you tackle this transform

(m/rewrite  {:Data
               {:FirstName "Bob"
                :CommunicationChannels [{:ChannelType {:Name "EMail"}
                                         :Content "<mailto:bob@no-mail.com|bob@no-mail.com>"}
                                        {:ChannelType {:Name "Phone"}
                                         :Content "5555-0000"}
                                        {:ChannelType {:Name "EMail"}
                                         :Content "<mailto:bob@other-no-mail.com|bob@other-no-mail.com>"}]}}
              {:Data
               {:FirstName ?first-name
                :CommunicationChannels [{:ChannelType {:Name !comm-channel-type}
                                         :Content !comm-channel-content} ...]}}
              {:first_name ?first-name
               :CommunicationChannels [{!comm-channel-type !comm-channel-content} ...]})

{:first_name "Bob"
 :email "<mailto:bob@no-mail.com|bob@no-mail.com>"
 :email_2 "<mailto:bob@other-no-mail.com|bob@other-no-mail.com>"
 :phone "5555-0000"
 :phone_2 nil}

Is it just up to two emails and phones or n emails and phones?

That is the annoying part. I am shaping it to a flat structure which goes into SQL and CSV. Perhaps a solution which has up to 2 and one that can have variable would be good.

This is what exists in the db actually.

phone_number,
   phone_number_2,
   mobile_number,
   mobile_number_2,
   email,
   email_2,

At present I’m not 100% sure where placement of nil values will occur. Process is. Folder full of JSON files -> Meander to get values of interest -> Turn into single or multiple CSV’s from that JSON -> Generate db tables from CSVs -> Query them with SQl and perform secondary transform with Meander if necessary.

That makes it a lot easier. I won't be able to write up a solution right now. There will be a tiny complication with map ordering. But other than that I think it shouldn't be two bad. My recommendation for a first stab at it is to use or to capture the different types in different memory variables.

So first Meander step is flattening JSON files and splitting up the data so it fits with the rest of our CSV etl workflow.

@jimmy no rush 😄

I have a lot of other very nested JSON files so I am hoping when I see a solution it enlightens me as to how I can do the rest.

@grounded_sage

(m/rewrite {:Data
            {:FirstName "Bob"
             :CommunicationChannels [{:ChannelType {:Name "EMail"}
                                      :Content "<mailto:bob@no-mail.com|bob@no-mail.com>"}
                                     {:ChannelType {:Name "Phone"}
                                      :Content "5555-0000"}
                                     {:ChannelType {:Name "EMail"}
                                      :Content "<mailto:bob@other-no-mail.com|bob@other-no-mail.com>"}]}}
  {:Data
   {:FirstName ?first-name
    :CommunicationChannels [(m/or
                             {:ChannelType {:Name "EMail"}
                              :Content !emails}
                             {:ChannelType {:Name "Phone"}
                              :Content !phones}) ...]}}
  {&amp; [[:first_name ?first-name]
      [:email !emails]
      [:email_2 !emails]
      [:phone !phones]
      [:phone_2 !phones]]})

Hi! A little bit of an odd example (it’s been minimized from a much larger example). Isn’t m/app called from the action part of a match?

(defn ->h [v]
  (case v
    1 {:info "some info"}
    2 {:extra "something else"}))

(defn foo [data]
  (m/match
    data

    {:a ?a
     :b ?b}

    {:foo
     (m/app str ?a
       merge
       {}
       (m/app ->h ?a)
       (m/app ->h ?b))}))

(foo {:a 1 :b 2})

;; expected
;; {:foo {:info "some info" :extra "something else"}}

The use case is that I have a value in the pattern part (like ?a and ?b) that I need to merge (after some transformation) in the action.

Never mind, solved it with m/rewrite!

And then realized that it could be used with m/match using just merge, mot m/app.

Yeah for match the right hand side is just normal clojure code. With rewrite it is a substitution.

Just wanted to share something fun from progress being made on the zeta branch showing a properly greedy * and the concept of the fold (which is still a work in progress).

(solve '[1 8 2 -5 3 4]
       (mz/with [%min (mz/fold *min 0 clojure.core/min)
                 %max (mz/fold *max 0 clojure.core/max)]
         [(mz/* (mz/and %min %max)) . !zs ...]))
;; =>
({#meander.runtime.zeta/fold-variable [*min ,,,] -5,
  #meander.runtime.zeta/fold-variable [*max ,,,] 8,
  #meander.runtime.zeta/memory-variable !zs []})

*min is -5, *max is 8, and !zs is empty because the greedy star ate its lunch. 🙂