tree-sitter gotcha: although it looks like the rules: { ... }
part of the grammar looks like a js object, order matters for the very first key-value pair. getting this wrong can (will?) cause chaos.
re: ignore markers and tag markers -- conceptually, i've been thinking of them as follows (hopefully correctly, but who knows?). for tags:
#my-outer-tag #my-inner-tag 'a
#my-outer-tag is "waiting" for (the function associated with) #my-inner-tag to have done something with 'a.
alternatively:
#my-outer-tag (#my-inner-tag ('a))
for discards, what i've currently got in the tree-sitter grammar (adapted from Tavistock's) is:
discard_form: $ =>
seq($._discard,
optional($.discard_form),
$._non_discard),
here $._discard
is basically just #_
so for:
#_ #_ 1 2
the inner #_
causes the 1 to be ignored, and the outer #_
doesn't see the #_ 1
and ends up ignoring the 2.With the edit it makes sense. 😃
he he -- sorry for the confusion 🙂
So what happens with something like:
#my-outer-tag #_ #my-inner-tag 'a 'b
my guess: #my-inner-tag 'a is "swallowed" by #_ so #my-outer-tag's associated reader function(?) is passed 'b for processing
here is an example:
user=> #uuid #_ #inst "1985-04-12" "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"
#uuid "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"
Cool. That's what I guessed.
sadly, the current grammar i have doesn't appear to handle it though
Don't try this:
#uuid #_ #uuid "84098d2d-2ed5-4de5-bd4a-c4b967f9e468" #uuid "8ce4afa3-1599-4a98-b642-59eb5e9e83b8"
i don't know what happens when #uuid is handed an actual uuid
Which is the same as this, naturally, which never returns either
#uuid #uuid "8ce4afa3-1599-4a98-b642-59eb5e9e83b8"
Nothing happens. Nothing as in things stop happening. 😃
lol
well, thanks to this discussion, i have more grammar hacking to do...and i thought i was close to finishing 🙂
Same here. To both sentiments.
However, I have my crude, line based, scanner, and you have that sophisticated super tool. 😃
actually, i'm not terribly fond of tree-sitter to be honest for certain tasks -- your scanner can be more flexible.
My scanner can be almost anything I want. Which is nice. However, the line based part of it... I wish I had lots of time so I could get rid of that!
it's pretty amazing that it can do so much even being line based -- i was surprised to be sure
have you considered looking at what edamame or tools.reader does?
I have looked in many different places. Often very shallow, though.
The scanner isn't of my creation, btw. I have modified it a lot so have some right to call it partly mine, but it was Matt Seddon creating it. I think it was the 100th time he created one, so he really knows these things at a level that boggles my mind. He is creating a new one as we speak, for a new LISP that he is creating at the same time. Haha, I guess he just likes to have work.
I am nudging things so that he hopefully will create a non-line-based thing this time. And DFA. Then I plan to bring that into Calva, replacing my current thing.
it's good if he enjoys what he's doing -- getting this stuff working can be fun, but it can be frustrating when there are no official specs
i've got some notes from digging through clojure / clojure-dev google groups archives and jira issues to try to track down what is supported by the reader