Clojurians Log v2

Would this be idomatic "Meander"? I have a vector of maps read in by the ultra-csv package, i just want to filter by the country column:

Sorry, did actually include the code:

(defn find-records-for-country2 [records country]
  (m/search records
            (m/scan {:Country ~country :as ?country})

            ?country))

Can you do aggregates similar to this Clojure code (which is probably not great as I am a new to Clojure). Again, I have a vector of maps in this example read in from a csv:

;;group-by example
  (def groupby-example (map
                       (fn [[grp-key values]]
                         {:group grp-key
                          :sum (reduce + (map :Sales values))
                          :max (reduce max (map :COGS values))})
                       (group-by :Country csv-seq)))

This looks fine to me. 👍

Meander doesn’t have anything for doing aggregates as conveniently as say group-by. In fact, we tend to recommend group-by because, well, its just fine. Of course, you could pull it off with Meander but it wouldn’t be as clean as what you have here. I am, however, interested in the idea of having some kind of aggregate type thing but I’m not sure what it would look like.

How would one start with web scraping in Meander. I was planning on using Jsoup and following along the lessons in Purely Functional TV. Though I’m wondering now if I just collect the html using clj-http as I am already using this in my code. Convert to hiccup then use Meander to fetch the information.

I can’t seem to follow the web scraping with Meander. This is to small of a sample. https://github.com/noprompt/meander/blob/epsilon/doc/cookbook.md#webscrape-html This is way over the top for a beginner. https://github.com/noprompt/meander/blob/epsilon/examples/hiccup.clj This returns nothing.

(m/search html-in-edn
   (m/$ [:main {:class "maincontents"}
         . _ ...
         [:h2 _ ?var]])
   [?var])

(m/search cal-edn
   (m/$ [:h2 _ ?var])
   [?var])

@grounded_sage does your main tag end with an h2? If not you need to add & _ afterwards to make it match.

And yeah the hiccup parser is supposed to be a very advanced example to show what meander is capable of.

Returns nothing.

[:main {:class "maincontents"}
                  . _ ...
                  [:h2 _ ?var]
                  & _ ]

What is your hiccup?

I thought my intuition was building on it but I’m uncertain now

If you give the input I'm sure we can find the problem.

(def google-edn (as-hiccup (parse (slurp google-url))))
  (m/search google-edn
            (m/$ [:html {:lang "de"}
                  . _ ... 
                  [:title _ ?title]
                  & _])
            ?title)

Gives me nothing

(m/search google-edn
            (m/$ [:title _ ?title])
            ?title)

(m/search google-edn
            (m/$ [:html {:lang ?lang} & _])
           ?lang)

Guessing at the structure. Google has its title inside the head tag right? If so you'd need to model that or use another $

That one says the the title is a direct child of HTML.

Ah that’s my error

(m/search google-edn
          (m/$ [:html {:lang ?lang}
                . _ ...
                [:head . _ ... [:title _ ?title] . _ ...]
                & _])
          [?lang ?title])

Awesome. Hopefully that makes sense.

Slowly getting there 🙂

It is a definitely a learning curve. But these are good important steps to go through.

Both (m/$ …) and (m/scan) works in that nesting. As well.

Yep, so $ will let you traverse deeper. And scan will let you look for things at the same level.

So this runs. But I’d like to trim down those repeating items in the (m/scan..) and also figure out how to get the commented pieces to work.

(m/search events-edn
            (m/$ [:main
                  . _ ...
                  (m/$ [_ {:class "cal-listitem"}
                        . _ ...

                        (m/scan [_ {:class "cal_date_show"}
                                 . _ ...
                                 [_ {:class "cal_date_date"} ?date]
                                 . _ ...
                                 [_ {:class "cal_date_time"} ?time]
                                 . _ ...
                                 [_ {:class "calinfo"} ?info]
                                 . _ ...
                                 & _])
                        ;. _ ... 
                        #_(m/$ [_ {:class "listinfo"}
                                . _ ...
                                [_ {:class "thetitles"} ?titles]
                                & _])

                        & _])
                  & _])
            
           [?date ?time ?info])

[:div {:class "cal_date_show"} 
  [:div {:class "cal_date_day"} _]      ; This could simply be _ignored ??
  [:div {:class "cal_date_date"} ?date]
  [:div {:class "cal_date_time"} ?time]
  [:div {:class "calinfo"} ?info]]