tree-sitter

tree-sitter fun
2020-03-09T02:37:35.060900Z

tree-sitter gotcha: although it looks like the rules: { ... } part of the grammar looks like a js object, order matters for the very first key-value pair. getting this wrong can (will?) cause chaos.

2020-03-09T11:25:46.070200Z

re: ignore markers and tag markers -- conceptually, i've been thinking of them as follows (hopefully correctly, but who knows?). for tags:

#my-outer-tag #my-inner-tag 'a
#my-outer-tag is "waiting" for (the function associated with) #my-inner-tag to have done something with 'a. alternatively:
#my-outer-tag (#my-inner-tag ('a))

2020-03-09T11:29:23.071300Z

for discards, what i've currently got in the tree-sitter grammar (adapted from Tavistock's) is:

discard_form: $ =>
      seq($._discard,
          optional($.discard_form),
          $._non_discard),

2020-03-09T11:30:47.071900Z

here $._discard is basically just #_

2020-03-09T11:32:29.072800Z

so for:

#_ #_ 1 2
the inner #_ causes the 1 to be ignored, and the outer #_ doesn't see the #_ 1 and ends up ignoring the 2.

pez 2020-03-09T11:33:29.073200Z

With the edit it makes sense. 😃

2020-03-09T11:33:38.073400Z

he he -- sorry for the confusion 🙂

pez 2020-03-09T11:37:11.074Z

So what happens with something like:

#my-outer-tag #_ #my-inner-tag 'a 'b

2020-03-09T11:42:48.075500Z

my guess: #my-inner-tag 'a is "swallowed" by #_ so #my-outer-tag's associated reader function(?) is passed 'b for processing

2020-03-09T11:45:27.075900Z

here is an example:

user=> #uuid #_ #inst "1985-04-12" "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"
#uuid "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"

pez 2020-03-09T11:47:29.076700Z

Cool. That's what I guessed.

2020-03-09T11:47:37.077100Z

sadly, the current grammar i have doesn't appear to handle it though

pez 2020-03-09T11:47:40.077200Z

Don't try this:

#uuid #_ #uuid "84098d2d-2ed5-4de5-bd4a-c4b967f9e468" #uuid "8ce4afa3-1599-4a98-b642-59eb5e9e83b8"

2020-03-09T11:49:07.078300Z

i don't know what happens when #uuid is handed an actual uuid

pez 2020-03-09T11:49:09.078400Z

Which is the same as this, naturally, which never returns either

#uuid #uuid "8ce4afa3-1599-4a98-b642-59eb5e9e83b8"

pez 2020-03-09T11:49:38.078900Z

Nothing happens. Nothing as in things stop happening. 😃

2020-03-09T11:49:46.079100Z

lol

2020-03-09T11:50:08.079500Z

well, thanks to this discussion, i have more grammar hacking to do...and i thought i was close to finishing 🙂

pez 2020-03-09T11:50:48.079800Z

Same here. To both sentiments.

1👍
pez 2020-03-09T11:55:37.081Z

However, I have my crude, line based, scanner, and you have that sophisticated super tool. 😃

2020-03-09T12:02:21.081700Z

actually, i'm not terribly fond of tree-sitter to be honest for certain tasks -- your scanner can be more flexible.

pez 2020-03-09T12:03:45.082700Z

My scanner can be almost anything I want. Which is nice. However, the line based part of it... I wish I had lots of time so I could get rid of that!

2020-03-09T12:04:49.083400Z

it's pretty amazing that it can do so much even being line based -- i was surprised to be sure

2020-03-09T12:05:10.083800Z

have you considered looking at what edamame or tools.reader does?

pez 2020-03-09T12:06:23.084400Z

I have looked in many different places. Often very shallow, though.

pez 2020-03-09T12:08:40.086900Z

The scanner isn't of my creation, btw. I have modified it a lot so have some right to call it partly mine, but it was Matt Seddon creating it. I think it was the 100th time he created one, so he really knows these things at a level that boggles my mind. He is creating a new one as we speak, for a new LISP that he is creating at the same time. Haha, I guess he just likes to have work.

pez 2020-03-09T12:10:38.088400Z

I am nudging things so that he hopefully will create a non-line-based thing this time. And DFA. Then I plan to bring that into Calva, replacing my current thing.

2020-03-09T12:14:31.089100Z

it's good if he enjoys what he's doing -- getting this stuff working can be fun, but it can be frustrating when there are no official specs

2020-03-09T12:15:07.089800Z

i've got some notes from digging through clojure / clojure-dev google groups archives and jira issues to try to track down what is supported by the reader

2💤