Clojurians Log v2

I was thinking a little about the proposed fold operator in zeta, something doesn't feel right about the mutable variable

Shouldn't the process of folding be an implementation detail that's hidden from the user? I can't think of any case where the intermediate values of the accumulator would want to be referred /outside/ of the fold

It seems more natural for the (fold ...) form to match the entire sequence to be reduced, and bind its argument as a general pattern to the result of the reduction

something like:

(m/match [1 8 9 -1 10 30 3]
  (m/fold ?result 0 clojure.core/min)
  ?result)
;; => -1

if anything, there could be a 5-arity of the operator (fold result init acc elem fn-return) where everything except fn-return is a pattern

{:total-score ?total
 :bonus ?bonus
 :games (m/fold [?total ?min-score] ?bonus
          [*t *min] {:score *s} [(+ *t *s) (min *min *s)])}

just throwing out ideas here, I have no clue what the implementation dififculties are like 😅

Hi, I’m trying to decide if meander is the right tool to use. I’ve never used it before, so I have some trouble deciding on its applicability. I have a sequence of maps like this:

(def data
  [{:a 1 :b 0 :c 0 :score 0.0123}
   {:a 0 :b 1 :c 0 :score 0.0123}
   {:a 1 :b 0 :c 1 :score 0.0123} ...])

(defn has-kvs?
  [m subset]
  (let [create-predicate (fn [[k v]] (fn [x] (v (get x k))))
        pred (apply every-pred (map create-predicate subset))]
    (pred m)))

(let [preds {"Pred 1" #(or (has-kvs? % {:a pos?
                                        :b zero?
                                        :c zero?})
                           (has-kvs? % {:b zero?
                                        :c zero?}))}

            "Pred2" #(or (has-kvs? % {:a zero?
                                      :b pos?})
                         (has-kvs? % {:a zero?
                                      :c pos?}))]

    (for [[title pred] preds]
      {:title title :values (->> data
                                 (filter pred)
                                 (map :score))}))

@qythium On zeta they are not that hard and, in fact, there is already a working implementation of the fold idea. Having the match happen at the “end” of the match presents a bit of a challenge but its an interesting suggestion. I’ll think about it. Another form I had considered was

(fold *var init clauses ,,,)

[current input] output

The “main idea” for zeta is to have the LHS and RHS be duals of each other in everyway. All of the primitives like and, or, let, etc. will work on the RHS.

We can give it a shot. Given your input data what are you thinking you would like your output to look like?

Hm, maybe something like:

[{:title "group name 1" :scores [0.0123 0.0123]}
 {:title "group name 2" :scores [0.0123 0.0123]}]

Does that make sense?

The first thing I might suggest would be to write something which matches your records and does the bucketing. Maybe something like this

(match x
  {:a 1 :b _ :c _ :score ?score}
  ["one-in-column-a" ?score]

  {:a _ :b 1 :c _ :score ?score}
  ["one-in-column-b" ?score]

  ,,,)

interesting

will try it out

@jlmr Here’s something that connects more with the problem you described

(defn record-buckets [record]
  (m/search record
    (m/or {:a (m/pred pos?) :b 0 :c 0}
          {:b 0 :c 0})
    ["Pred 1" record]

    (m/or {:a 0 :b (m/pred pos?)}
          {:a 0 :c (m/pred pos?)})
    ["Pred 2" record]))

(mapcat record-buckets [{:a 1 :b 0 :c 0 :score 0.0123}
                        {:a 0 :b 1 :c 0 :score 0.0123}
                        {:a 1 :b 0 :c 1 :score 0.0123}])
;; =>
(["Pred 1" {:a 1, :b 0, :c 0, :score 0.0123}]
 ["Pred 1" {:a 1, :b 0, :c 0, :score 0.0123}]
 ["Pred 2" {:a 0, :b 1, :c 0, :score 0.0123}])

search works to find all the possible solutions that match your input. If you want only the first match you would switch to m/find and use map instead of mapcat.

I went back to look at the given example :

(m/find [1 8 9 -1 10 30 3]
  (m/with [%min (m/fold *min 0 clojure.core/min)]
    [%min ...])
   *min)
;; => -1

It's still quite unintuitive how that's supposed to work... maybe I need to set up a zeta scratchpad to try things out

Yes. fold is a primitive which, in essence, manages the binding. Theres a couple of smaller pieces missing on zeta but once they are there you could just them to derive logic and memory variables from the same structure.

The name fold might also be inappropriate.

That might be partly it, based on the name I was expecting a pattern which stood in place of a collection to be folded

Also my category theory knowledge is a little shaky but does it somehow correspond to catamorphisms on the LHS and anamorphisms on the RHS?

Yes, but I’ll be honest I don’t have enough CT skills to go in depth on it. Though I can say there’s this cool thing: http://conal.net/talks/folds-and-unfolds.pdf 🙂

I should have some time soon to fill in some of the meander.zeta namespace for folks to play around with.

All of the work that is being done on zeta is happening in dev/meander/ and src/meander/runtime

Its worth mentioning that Clojure’s reduce has a very specific implementation: its driven by seqable things only.

But the heart of reduce is the actual reducing function.

@qythium If you decide to play with zeta let me know if you have questions. 👍

I definitely think the current fold we are playing around with is a bit lower level than people will need to actually use. Being able to match on the result of a fold with a logic variable definitely makes sense. I think the fold we currently have is the primitive that will power that sort of thing. For example, we could actually implement memory variables in terms of fold, the really are just syntatic sugar for a fold with empty vector and conj.

Really the mutable variable in fold doesn't even need to be exposed. (mutable is really a misnomer for this imo)

that's great to hear 🙂 I wonder if the current cata operator is somewhat of a misnomer too

Cata on the lhs I think is cata. Cata in rhs is really recur. (Had it backwards originally)

Any hints on how to use zeta in its current state? I checked out the branch and started a repl clj -A:dev but the namespaces are doing weird things when loaded

overwriting the compiled/*.clj files and throwing "unmatched delimiter" errors

lots of advanced macrology going on 😵

@qythium Don’t know your setup. But I just pulled down the latest. Loaded up cider with the -A:dev and evaluated the meander.dev.zeta namespace

Everything worked for me doing a simple (rewrite 1 ?x ?x)

There is definitely a lot of weird stuff going on. But I will say, so far working with the meander.zeta compiler is so much better. I am going to have a blog post coming out soon sketching the idea. Not full details, but showing how to make your own meander lite compiler using meander.

That's really strange, it works in a plain clj repl in the terminal but not when I use Cider

it's alright, I'll figure it out later - probably something to do with my Emacs config

aha, it was due to my *print-length* settings, the compiled output got truncated

Thats good to know. That means we should probably rebind it in the defmodule source.

defmodule is basically meander.epsilon/rewrite where the result of its macro expansion is rewritten to use the zeta runtime and the source is dumped to a file where it is defned.

The subst compilation emits to code which uses only the runtime, match compilation is somewhere in between.

If something doesn’t work its probably not implemented.

The last time I was in there I was starting to work on m/string and that ultimately caused me to realize that we’re missing greedy star/plus.

This setup definitely seems strange. But there are lots of reasons we went down this route. One being that meander.zeta will eventually be bootstrapped so our optimization effort will yield both faster generated code and a faster compiler at the same time.

Its also, like, 10,000 times easier to work on the parser.

Its been a breeze to work on the parser and, really, the other components too.

Well, for me anyway.

A reason for that is due to not having to manage a bunch of functions or deal with the hassle of doing things manually with Clojure. Using recursive rewrite rules is just so much simpler and easier to me because I only have to think of shapes. Pretty much the value proposition of rewrite.

To put it in perspective, I spent several weekends writing the zeta parser in Clojure before dumping it because it was frustrating to change and debug. I rewrote it with rewrite and had a working, bug free implementation in less than a day, really about a few hours.

Now, because Jimmy and I have been the only ones to be working on zeta there isn’t much in the way of commentary explaining how things got to the point they are at on zeta, however, I am (and I’m sure Jimmy is too) very happy to discuss and collaborate on any of it. 🙂

One of the biggest advantages to this approach, in general — again, to me — is the very minimal amount of scope. In day-to-day programming in most every language theres just this deluge of scope.