Clojurians Log v2

I haven't used Meander for a while now (deemed too experimental for work use), but wouldn't the "unspecified keys" proposal conflict with the use of ?logic vars as lookup keys in a map?

(let [db {:user-db {"Alice" {:id 123}
                    "Bob"   {:id 124}}
          :info-db {122 {:favorite-food "nachos"}
                    123 {:favorite-food "noodles"}
                    124 {:favorite-food "nutella"}}}]
  (m/match db
    ;; query: what is Alice's favorite food?
    {:user-db {"Alice" {:id ?id}}
     :info-db {?id {:favorite-food ?food}}}
    ?food))
;; => "noodles"

can't construct a version with !memory vars at the moment, but it sounds bad for the semantics to be different depending of the type of var

> deemed too experimental for work use Ha! 🙂

I see ticket #130 more or less as an invitation to think about how we can improve the visual parts of map/set patterns for the reasons mentioned in that ticket.

One thing we could do is provide a way to hook into the parser via protocols.

Clojure doesn’t respect the namespace aliases of tagged literals though which makes doing something like

#my.custom.MapPattern {!k !v}

The other possibility is making syntax extensions a bit more interesting.

There’s also the road of flags.

FWIW I’d happily accept the different behavior based on type of var to regain the structure of my expressions 🙂 I have tried to think of ways to explicitly differentiate and unfortunately they all bloat the structure a lot. I really do think it boils down to treating memory variables differently in maps and sets… and that there is significant value in doing so.

I think there is significant value in achieving what you’re after in #130, however, the culprit is the search space the container represents not the variable.

I think it has been mentioned before but would you agree with

{& ([!k !v] ...)}

I’d rather just stick with map-of or {& (m/sequable than make that change!!! 😛 FWIW I think that violates the structure rules

Can you explain more what “the culprit is the search space the container represents not the variable” means?

Maybe an example 🙂

I think you mean… what is does this mean? {:a 1, !k !v} To which I say it should logically match {:a 1, :c 3} and that !k should be bound as [:a :c] order is arbitrary but matches !v [1 3]

Notably if you wanted to you can also write {:a 1, & {!k !v}} and instead have !k/!v only match the remaining map conveniently

{p1 p2}

{:a 1 :b 2}

([:a 1] [:b 2])

The search space for a map is actually a bit more complex than this but I’m choosing the simplest case for illustration.

Why is this important? Well if you want the variables to bind differently, you need to change the search space or change the pattern.

The syntax extensions exist to change patterns which can affect how search spaces are yield.

However, visually they are not satisfactory.

We don’t have complete control over how to view a structure in a given context beyond wrapping it in a (symbol p) kind of pattern where symbol has been defined as a syntax extension.

What would be nice is to be able to say something like

(notation {!k !v} (m/map-of !k !v))

So instead of making the interpretation of an arbitrary pattern context sensitive with respect to the container it happens to appear in you say instead that here we say that a pattern which looks like this should be replace with a pattern that looks like this.

So, for this example, you could write

{:a 1 & {!keys !vals}}

{:a 1 & (map/of !keys !vals)}

@noprompt that sounds like a pretty dangerous proposal..! I wouldn't trust myself with such large amount of control over the semantics of an expression

currently I can look up where each Meander operator is defined and what it does, with arbitrary notation transforms I won't even be able to tell at a glance what's being transformed and where it's defined

I’m not really able to follow the “search space” argument; maybe you can simplify for me … observing that maps are sets of key-values … lets just talk about sets, and not use any pre-existing variables like ?k or !k: let’s just talk about #{-k} a magical new thing that will allow me to match a set and substitute its values, is this impossible for some reason? I’m explicitly not including find in this description.

so this -k is bound to all the values in the set?

I know what you are going to say… oh but -k could have matched any element! Greed is evil! To which I reply this is why I need greed.

I’m trying to be abstract and just specify that I’d like

(m/rewrite #{1 2 3}
  #{-k}
  #{-k})
;;=> #{1 2 3}

(m/rewrite #{1 2 3}
  (m/and #{} ?xs)
  ?xs)

(m/rewrite #{1 2 3}
  #{-k}
  #{(m/app inc -k)})
;;=> #{2 3 4}

Greed is certainly not evil.

What is evil is a that a pattern could have different semantics in different contexts: precisely your point about {& ([!k !v] …)} .

can you give an example of how #{-k} would have different meanings?

No because it only has one meaning.

lol

🙂

I’m so confused 😕 sorry if I’m dense

The search space yielded by the container determines what values it’s contents will be matched against.

Tim, you are not dense. 🙂

I’ve known you for, what, something like 6 years or something? Dense isn’t even on the map. ❤️

Is it that #{?k} and #{-k} have different meanings and that’s bad?

And, fwiw, these conversations are really important.

It is that, in general, for all patterns p the semantics of #{p} do not change.

gotcha

This is why I was lamenting the fact that tagged literals could be so bulky.

^-k #{} <-- is this ok?

But I wouldn’t want to get in the way of something like #mine #{!k} where perhaps #mine maps to

(defn mine [value]
  (reify
    m.protocols/IExpandSyntax
    (-expand-syntax [this]
      ,,,)))

is here a more direct solution here where I write my own custom tag that translates {!k !v} to (map-of !k !v) and stop complaining?

That would be the notation idea. I’ve been on the fence about it because, like @qythium pointed out, it’s crazy powerful and could be bad. That being said @jimmy has somewhat convinced me that isn’t necessarily a good reason to dismiss the idea on the grounds that we can’t stop people from obfuscation. defsyntax can easily obfuscate. Going the route of notation should involve a commitment to tooling that enables makes it easier to expand patterns as you would a macro.

yeah, the last time I played around with defsyntax I found myself really wishing for a way to macroexpand sub-patterns

not sure if that's already possible or meaningful

It is meaningful.

that's a relief to hear!

One really nice property I like about Meander is that all the special vars and things are drop-in replacements for literal matches

there's probably a better way of putting this but it makes a lot of sense in a easy refactoring / referential transparency mindset

so having !k !v be context dependent would complect the whole situation IMO

Currently, I’m a huge bottle neck with respect to the project. I’ve got work related requirements and my kids at home 24/7 (two of which are being home schooled now). What I really want to figure out is how to get others involved in a maximally collaborative but minimally pressured way.

I have a model for zeta that I think is mostly sound in terms of the primitives and assumptions. The interpreted model has explainability built in like spec and it works both for matching and substitution. But I need others to think about it too.

Compilation can be built on top of that model which can fall back to run time interpretation if needed.

https://gist.github.com/noprompt/387c7aa2b02b2de1330687a9bef0f469#file-two-clj-L679-L707

(let [?x (logic-cell)
      x-> (lifo-cell)
      x<- (fifo-cell)
      pattern (pair (&& (in [1 2 3]) x<- x->)
                    (&& (in [4 5 6]) x<- x->))
      extract-bindings (fn [result]
                         (let [bindings (get result :bindings-out)]
                           {'x-> (get bindings x->)
                            'x<- (get bindings x<-)}))]
  [;; Query the pair [2 6] against the pattern and pull out the
   ;; passing bindings.
   (map extract-bindings
        (filter pass? (query pattern [2 6] {})))
   ;; Attempt to generate 20 solutions and pull out the passing value
   ;; and binding.s
   (map (juxt :value extract-bindings)
        (filter pass? (take 30 (yield pattern {}))))])
;; =>
#_
[({x-> (6 2), x<- [2 6]})
 ([[1 4] {x-> (4 1), x<- [1 4]}]
  [[1 5] {x-> (5 1), x<- [1 5]}]
  [[1 6] {x-> (6 1), x<- [1 6]}]
  [[2 4] {x-> (4 2), x<- [2 4]}]
  [[2 5] {x-> (5 2), x<- [2 5]}]
  [[2 6] {x-> (6 2), x<- [2 6]}])]

“Two” is what I call the underlying framework. It thinks of variables similar to how Clojure thinks of them in that names point to things but the things are cells which specify how their content is accumulated (fold) and it is dispersed (unfolded).

At a more general level, patterns of which variables are subset, are objects which define what it means to be queried and also what it means for them to yield their instances.

IOW, patterns represent elements of sets.

query asks if a value is a member of those sets and produces bindings. yield asks if a value can be produced given bindings and if so produce those values.

This is deliberately not unification.

@qythium  could you expand on what “all the special vars and things are drop-in replacements for literal matches” means?

I think of the "base case" for patterns as matching on a pattern made entirely of literals

(m/match [1 2 [3]] [1 2 [3]] :ok)

(m/match [1 2 [3]] [1 2 ?x] ?x)

:thumbsup:

I don't know if that's always true eg. cata seems to be context dependent, but I find it a useful mental model

@noprompt That was quite a bit to digest but really interesting! So fifo-cells are Meander's memory variables?

cata isn’t context sensitive in the sense that its semantics are the same wherever it appears. It is context sensitive in the sense of what the target value is recursively matched against in terms of the system it appears within.

Yep. I didn’t chuck in the use of ?x there but it works as you’d expect.

Keep in mind this is a model for interpretation. This means, at a minimum, we have a non-macro version of search/yield.

There is some other interesting stuff going on in there too if you’ll notice the use of SplittableRandom and seed.

The idea here is not only to allow for concepts of matching elements from infinite sets but also producing them.

This is interesting for something like (sum ?x 3) where the query means find all ?x + 3 = TARGET and yield means all values that give you a value which can match ?x + 3 and, if you want, the bindings that makes that true.

The streams of data produced by the way include failures unlike previous versions.

This means that it’s possible to have a model that allows one to explain failures in either case, and, importantly, prevent divergence.

There’s also a rough sketch of group-by-cell in there.

This sounds a lot like unification, from the vague familiarity I have with both areas

ah scratch that, my thinking is too muddled for that to be a coherent question

I'd love to dive into this in my spare time too and contribute! It's just a little daunting how high-level the concepts seem to be from scanning through the Meander codebase

Also just throwing out a random thought I had before - seeing as how Meander's vars use different sigils that are compiled into some underlying representation, could this be made into an extendable notation?

I need write down what I wrote down here on that file, eh? 🙂

I think the sigils are valuable for the macro version for at least things like logic variables.

(defsigil © 
  "docstring" 
  [sym <other necessary args>] 
  <implementation, compiler/interpreter hooks etc.>) 

then you can go around using ©x in patterns with your own user defined semantics

Yeah. This is something I’ve been thinking about also.

I think the smallest thing would be declaring variables somehow

(m/declare [$x ([m/unbound 0] 0 _ m/unbound)]
  ,,,)

Or maybe [$x ~rewrite-rules] to keep plain functions in the mix.

Damn you covid-19!

Oh, before I forget, I should plug #asami as a project to keep an eye on. 🙂

I am on the same team as the original author and am gradually becoming a contributor.

I’m really looking forward to the logic-cell stuff as it sounds like a general solution for aggregation/reduce!

I’ve attached the “macro solution” of replacing {!k !v} with {& (map-of !k !v)} to issue #130 based on the suggestions here mainly to record it… I can go ahead and start using the macro in my projects and learn the hard way where it will bite me later.

actually I think this is already broken: #{^& ?rest}

It is?

I’m not sure notation is needed at all… as a user I can write a macro that translates my patterns

I mean maybe it solves some other problem 🙂

and that’s great.

We could also expose an ast transformation level if we really wanted to. Not saying we should, but it would solve the problem and also let us experiment with optimizations in an extendable way.

(let [s #{1 2 3}]
  (m/match s
    #{^& ?rest}
    ?rest))
;; => #{1 3 2}

(let [s #{1 2 3}]
  (m/find s
    #{^& ?rest}
    ?rest))
;; => #{1 3 2}

(let [s #{1 2 3}]
  (m/search s
    #{^& ?rest}
    ?rest))
;; => (#{1 3 2})

(let [s #{1 2 3}]
  (m/rewrite s
    #{^& ?rest}
    #{4 5 6 ^& ?rest}))
;; => #{1 4 6 3 2 5}