Clojurians Log v2

Ok, I should figure out what I need to do for namespaced elements for rewrite-cljc. (It feels like I might have said this a year ago 🙂). Right now I am trying to guess if a correct sexpr on an auto-resolved namespaced keywords and maps would have real value to rewrite-cljc users. For reference, I am calling ::mykeyword1 current-namespaced and ::alias/mykeyword2 alias-namespaced.

@vemv you had some https://github.com/lread/rewrite-cljc-playground/issues/7#issuecomment-667743070. I’m wondering if your issue has more to do with the awkwardness of working with the node than any issue specific issue with sexpr.

I guess the most important things to consider are: 1) is such a node distinguished from other rewrite-cljc maps/keywords -- can you tell if it's current-namespaced, or alias-namespaced by looking at some property? btw. I don't think rewrite-cljc can determine properly if the a in ::a/foo is an alias or full namespace name 2) can it round-trip properly, yielding the same string

Thanks for the reply @borkdude. I was thinking of maybe adorning the node with a namespaced-type which would be :literal , :alias or :current.

For ::a/foo you are saying a could be a namespace name or the namespace alias, right?

I'm just trying this out and perhaps it's always an alias

$ clj
Clojure 1.10.1
user=> (require '[clojure.string])
nil
user=> ::clojure.string/foo
Syntax error reading source at (REPL:3:0).
Invalid token: ::clojure.string/foo
user=> (require '[clojure.string :as str])
nil
user=> ::str/foo
:clojure.string/foo

so in that case it would be a good name

What is an example of a map with :literal namespaced-type?

Oh that’s my terminology sorry, I did not find a single definitive terms so made some up. An example would be :my-prefix/mykeyword3.

In clj-kondo I stored it this way: https://github.com/borkdude/clj-kondo/blob/2fb61754456e6d35f136b340e8aa40064699c907/parser/clj_kondo/impl/rewrite_clj/parser/namespaced_map.clj#L28

So the namespace and a boolean indicating if it's an alias

:foo/bar -> foo, false ::foo/bar -> foo, true :bar -> nil, false

I looked your version. It was helpful. Do you ever sexpr on, for example ::bar in clj-kondo?

I guess I have some weird keyword for representing the current namespace: :__current-ns__

since symbols and keywords are separate namespaces you can do that

let me try sexpr and see what happens

user=> (p/parse-string "#:foo{:a 1}")
<namespaced-map: #:foo{:a1}>
user=> (p/parse-string "#::foo{:a 1}")
<namespaced-map: #::foo{:a1}>

I guess sexpr is not a valid operation on these nodes since it will depend on your aliases

For your 2nd point on round-tripping I think that won’t be challenging and is not an sexpr concern.

in your runtime

I solved this problem in edamame by allowing the user to provide a function to resolve the alias

since you won't break the API of rewrite-clj, you could consider a dynamic var for the same purpose, or extra args in sexpr

it's only problematic with aliases. I think blindly resolving it to the runtime's aliases is not good

Yeah… rewrite-clj current has some partial support in for alias-namespaced maps where it does a (ns-aliases *ns*)…

Which implies that the code being parsed has been loaded.

which also implies a coupling between the tool parsing the code and the code being parsed, which does not make sense at all.

I’m not a big fan of the current approach.

I noticed that cljfmt actually parses the ns declaration to get aliases and considered that as an option.

I think that logic belongs somewhere else

you should just be able to parse #::foo{:a 1} with rewrite-cljc and provide an option how to resolve the aliases

before you know it you're writing a clojure interpreter. consider:

(ns foo (:require [clojure.string :as str]))
(in-ns 'bar)
#::str{:a 1}

I was thinking that rewrite-cljc might provide a rudimentary parser/resolver, but you could also plug in something else. But you are thinking such a thing does not belong in rewrite-cljc?

I can probably come up with a number of other examples where this will break down

Yes, that’s the rabbit hole I am struggling with.

I would do it in the reverse: start with the pluggable option, don't go down the rabbit hole until people actually want this to be part of the library

Yeah, that’s good advice.

The option before that could be: don't support sexpr for these nodes.

Can people implement sexpr themselves via some protocol?

(sexpr nsm {:resolve {'str 'clojure.string}})

Yeah, that was one consideration, but I think that would break some rewrite-clj fns. I think I’ll have to return a technically incorrect value for sexpr for auto-resolved namespace elements. This result could be improved by a plugin.

In edamame:

(parse-string "[::foo ::str/foo]" {:auto-resolve '{:current user str clojure.string}})
;;=> [:user/foo :clojure.string/foo]

I'm using this in sci where I already know the aliases because of handling the previous forms

Yes, at a minimum I’ll provide some mechanism for people to provide current namespace and aliases via hand-coding…. if that’s what you mean.

hence where I'm getting the idea from that an interpreter and a parser/rewrite-tool are separate things

yes

I’ve always been half-annoyed that rewrite-clj includes sexpr at all.

well, it's convenient, but I think in 99% of cases I call it to determine the type of thing

I can see that it makes usage more convenient though… but in some cases, like this one, well.. it is not..

what you said!

so it would have been cool if rewrite-clj supported something like (type nmp) -> :namespaced-map

There’s (tag nmp) -> :namespaced-map

But tag granularity is not likely what you might expect.

yeah, tag often returns token which can be a lot of things

Exactly

Anyway, I greatly appreciate the chat. I’ve been stuck in a bit of analysis-paralysis and I think you thawed me out. :simple_smile:

:)

Is adding (type node) worthy of a git issue? I haven’t given it any real thought…

Oh maybe it is similar to: https://github.com/lread/rewrite-cljc-playground/issues/5

I think so yes. It would save me writing code like this: https://github.com/borkdude/clj-kondo/blob/2fb61754456e6d35f136b340e8aa40064699c907/src/clj_kondo/impl/types.clj#L213

Ok will update the issue.

It doesn't have to be in the initial release I think, since it's not part of the original

Agreed! The only reason it would make it into the initial alpha release is if I found a real need to procrastinate on a must-have alpha feature/fix :simple_smile:.

I think parcera doesn't have sexpr right?

I haven’t dug deep into parcera, but I think you are right. Parcera seems simpler in many ways.

i am not sure about the no sexpr bit -- i don't remember well enough what sexpr does atm (just woke up). but parcera does have quite decent successful round-tripping afaict (have tested against large number of samples).

btw, wrt terminology (naming of constructs), i think there might be some benefit in examining what various grammars are doing. if nothing else it might make talking about some things easier.

for example, it may just be my ignorance, but in the parcera grammar i encountered the term "macro keyword": https://github.com/carocad/parcera/blob/master/src/Clojure.g4#L157 i might have mentioned this before, but i have a version of the tree-sitter grammar that tries to merge some ideas from parcera and the naming is part of that. i don't know if anyone is up to making a glossary of terms, but in the matter of grammars, it seems like it could help reduce confusion (e.g. some people say "discard" while others say "ignore"). i also started saying "metadatee" to refer to the thing the metadata is supposed to apply to.

@sogaiu sexpr transforms a parsed node into a real clojure thing

ah thanks -- i don't think parcera does that.

In the other direction rewrite-clj also supports coercion form real clojure things to rewrite-clj nodes.

For terms I’m kind of stuck with what I’ve got for rewrite-cljc. But where I have wiggle room, for Clojure elements, I am going to try to use what the Clojure core team uses. For example, I just asked on #clojure about namespaced element naming and got a response form Alex Miller.

cool you got a response! i think it totally makes sense to use what the core team uses -- but afaict, they don't have terms for everything.