Clojurians Log v2

I'm working through the meander examples at https://cljdoc.org/d/meander/epsilon/0.0.402/doc/understand-meander-s-pattern-matching-macros#rewrite I'd like to change the example a bit to return something like clojure.core's group-by. For example, I have a collection

[{:name    "entity1"
               :vals  [{:value 1} {:value 2}]}
              {:name "entity1"
               :vals [{:value 3} {:value 4} {:value 5}]}]

[{:name "entity1" :vals [1 2 3 4 5]}

(m/rewrite [{:name    "entity1"
               :vals  [{:value 1} {:value 2}]}
              {:name "entity1"
               :vals [{:value 3} {:value 4} {:value 5}]}]
             [{:name    !name
               :vals [{:value !values} ...]} ...]
             [{:name   !name
               :value  [!values ...]} ...])

@mark340 at the moment we typically recommend group-by 🙂

We are working on something which will help with reduction kinds of problems like group-by but in a robust and general way.

You can do reductions with rewrite though, depending on the task, it might not be what you want.

(let [es [{:name "entity1"
           :vals  [{:value 1} {:value 2}]}
          {:name"entity1"
           :vals [{:value 3} {:value 4} {:value 5}]}]]
  (me/rewrite [{} es]
    [?state []]
    ?state

    [{?name [!values ...] & ?state} [{:name ?name :vals [{:value !values} ...]} & ?rest]]
    (me/cata [{?name [!values ...] & ?state} ?rest])

    [?state [{:name ?name :vals [{:value !values} ...]} & ?rest]]
    (me/cata [{?name [!values ...] & ?state} ?rest]))
  ;; Semantically equivalent too
  (reduce
   (fn [state e]
     (me/rewrite [state e]
       [{?name [!values ...] & ?state} {:name ?name :vals [{:value !values} ...]}]
       {?name [!values ...] & ?state}

       [?state {:name ?name :vals [{:value !values} ...]}]
       {?name [!values ...] & ?state}))
   {}
   es))
;; => (let [es [{:name "entity1"
           :vals  [{:value 1} {:value 2}]}
          {:name"entity1"
           :vals [{:value 3} {:value 4} {:value 5}]}]]
  (me/rewrite [{} es]
    [?state []]
    ?state

    [{?name [!values ...] & ?state} [{:name ?name :vals [{:value !values} ...]} & ?rest]]
    (me/cata [{?name [!values ...] & ?state} ?rest])

    [?state [{:name ?name :vals [{:value !values} ...]} & ?rest]]
    (me/cata [{?name [!values ...] & ?state} ?rest]))
  ;; Semantically equivalent too
  (reduce
   (fn [state e]
     (me/rewrite [state e]
       [{?name [!values ...] & ?state} {:name ?name :vals [{:value !values} ...]}]
       {?name [!values ...] & ?state}

       [?state {:name ?name :vals [{:value !values} ...]}]
       {?name [!values ...] & ?state}))
   {}
   es))
;; =>
{"entity1" [1 2 3 4 5]}

This, for example, is gross.

You know, I think you told me that before but I forgot. It seems such a natural thing to do in meander. Looking forward to zeta 🙂

Thanks. I want things to go more quickly but life has been more demanding than usual. 😅

For everyone. Stay safe; stay healthy!

https://github.com/noprompt/meander/pull/120

☝️ I’ll leave this open for the next day or so in case anyone wants to say something.

Hey, I’ve started using Meander more and more and it’s great! I have started getting some weird compiler exceptions however when running tests using lein kaocha --watch that occurs when reloading namespaces containing Meander code. /t

The exception is clojure.lang.Compiler$CompilerException: Syntax error macroexpanding m/rewrites at (...).

Caused by: clojure.lang.ExceptionInfo: Call to #'meander.syntax.epsilon/resolve-expander did not conform to spec:
epsilon.cljc:308
-- Spec failed --------------------
Return value
  meander.epsilon/eval13971/expander--auto--
should satisfy
  (fn
   [%]
   (or (nil? %) (sequential? %)))
-------------------------
Detected 1 error
{:clojure.spec.alpha/problems [{:path [:ret], :pred (clojure.core/fn [%] (clojure.core/or (clojure.core/nil? %) (clojure.core/sequential? %))), :val #object[meander.epsilon$eval13971$expander__9593__auto____13972 0x7abb82d6 "meander.epsilon$eval13971$expander__9593__auto____13972@7abb82d6"], :via [], :in []}], :clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__2509 0x1a0fd147 "clojure.spec.alpha$regex_spec_impl$reify__2509@1a0fd147"], :clojure.spec.alpha/value #object[meander.epsilon$eval13971$expander__9593__auto____13972 0x7abb82d6 "meander.epsilon$eval13971$expander__9593__auto____13972@7abb82d6"], :clojure.spec.alpha/ret #object[meander.epsilon$eval13971$expander__9593__auto____13972 0x7abb82d6 "meander.epsilon$eval13971$expander__9593__auto____13972@7abb82d6"], :clojure.spec.alpha/failure :instrument, :orchestra.spec.test/caller {:file "epsilon.cljc", :line 308, :var-scope meander.syntax.epsilon/expand-form}}

This is a minimal (I think) case where CompilerException is thrown

(meander/match nil
  (meander/seqable)
  nil)

What are your thoughts around pluggable optimization strategies? I have a set of documents that, right now, are stored in plain Clojure sequences and yields plenty good enough performance. I have a somewhat complex meander pattern that encodes the business logic to find the right data within those documents. Over time, however, I expect the set of documents to grow perhaps to the point where sequence performance is not good enough. Much like adding an index in an RDBMS, I'd love to swap out my sequences of data for something else but not change the declarative search strategy encoding in Meander. In practice, the next performance step would be storing the data in Datascript. I can almost imagine providing Meander some hints to access the data using index seqs. Thoughts?

I can definitely look into this. I'm guessing these tests are instrumenting all specs?

I just tried recreating this. I instrumented every thing and tried running the code above and it ran with no issues. Do you have a project that is open with the issue? Can you share your deps? Any information for recreating would be great.

We don't have any plans right now for going that route. I do think having indexes and looking up by thing is a good thing to do if you have large datasets and are looking for performance. But I'd personally recommend just making those indexes and passing them to your match. In my view, that would be the same as giving us a hint. Just give us the actual index and make your pattern match on that index.

Thank you for checking it out :) Yes, the tests are instrumented using https://github.com/jeaye/orchestra. Sorry for leaving out details, I’ll try my best to recreate it in a fresh, open project. I’ll get back to you

Orchestra would explain the difference.

Hopefully have some time today to try that out and track down the issue.

I’ve recreated the issue in a new project now. I can share it on github

I was able to do it with orchestra. Thanks, that was the missing piece

Awesome! I also uploaded my project here incase it’s still relevant https://github.com/magnusdk/meander-orchestra-kaocha-issue

Found the issue. Should have a fix today. There is nothing actually broken going on just a bad spec.

That is fantastic! 🦜 Thank you for your time 🙂 This is great

Just FYI, we’re probably not going to be using spec going forward. Its been a pain for a number of users and myself.