Clojurians Log v2

I have never used neither parinfer or paredit, so there may be surprises. I will by all means try to avoid direct access to :before and :after. The slider is a structure with some basic functions. Many tasks can be performed by navigating the slider, setting marks and jumping between marks. See for example "mark-paren-start" in slider.clj or "highlight-sexp-at-point". My first attempt would be something similar. At least with paredit you can focus on a single function at the time. All functions would take a slider, on which I would do internal editing operations to produce a new slider with the expected change applied. I might also reuse the strategy from the syntax highlight functions which combines slider operations with a statemachine approach that keeps track of what is at cursor position: Am I in a string, a comment, a function, etc. See for exampel clojuremdhl.clj. It is 25 lines of code to get acceptable syntax highlight for clojure and markdown in one go.

Maybe the function "read-string" could be useful applied to sexp-at-point.

I was completely unaware you had a state machine tracking "am I in a string?" "am I in a comment?" "am I in a function?" that completely resolves my objection of "you have to parse all of :before to konw the current state of the cursor"

It is not like a global state machine that always knows whats is at the cursor. It is just created during syntax highlight. But i guess syntax highlight and structure-aware editing do have some challenges in common, and probably solutions as well. I am not afraid of performance issues. Analyzing the context by navigating the slider is very fast.

I admit I'm doing pre-mature optimization and thinking about big-Oh running time, when, in practice, most files are: < 1000 lines of code , with a navg of maybe 40 chars / line, so :before / :after are a most 40,000

though actually, I think this limits you to linera / O(N Log N) time algorithms, at quadratic time algorithms, you're arledy at: 40,000^2 = 1,600,000,000 ops = 1.6 Giga Opx and if you want 20ms refresh, 20ms * 1 Ghz = 0.02 Giga Opx

correct me if my math is off -- if we do any quadratic time operation in :before / :after, we are going to experience notable loag (on the order of seconds) for a file with 1000 lines and ~ 40 chars / line

I actually do not analyze the whole file for syntax highlight. I only look a the visual part and make a qualified guess of the "current state", based on the context. In some edge cases my guess will be wrong, like sometimes a string is not highlighted, when the screen splits in the middle of a string. Considered the performance gain of this strategy and the non criticalness of syntax highligthing I can live with that. I also consider when performance is important and when not. Moving the cursor and inserting chars should be fast. Delays probably starts to be noticeable around 20ms. Commands like "barf" are not fired like 10 times a second. I bet user can live with a delay of 200ms for that kind of command.

if by "barf" you mean

( | ... a) -> ( | ... ) a

I'm now really curious what structures vim uses internally, I can't recall a single time when vim felt laggy

quoting: https://ecc-comp.blogspot.com/2015/05/a-brief-glance-at-how-5-text-editors.html

Vi represents its buffer as an array of lines. It simply adds or deletes lines where it has to, which is very similar to how GNU Moe does it, as we discussed earlier.

Vim seems to not show syntax highlight when navigating directly to the end. I guess it will only render from the top, at the same pace as the user navigates down. So it only has to render one screen at the time and not the whole file. Btw Vim only navigates up and down a whole line at the time, so it "cheats" when there are multiple wrapped lines 🙂 Which makes it more difficult to navigate long lines. When I do syntax highlight in Liquid I only traverse the visual part of the slider once and when some key characters are incountered I might have to look a fixed amount ahead. Since moving the cursor is O(1), syntax highlight is close to O(n) where n is the number of characters on the screen. Since I cannot image syntax highlight without visiting every visible char at least once, I think only small improvements can be done to that part.

I'm really glad we've having this discussion. I hope it's not an utter waste of your time (and delaying liquid work.)

I'm 99% convinced there are fundamental limitations to all these 'text' based approach (slider or array of lines or slider of lines), and I need to look into "structure editor" of editing ast directly

No, that is great. I like having discussions that make me think about, what I am doing. It sounds like a fun experiment to edit the structure directly. I actually think it should be doable. Maybe be creating a structure-buffer instead of regular buffer. Then just store the result of "read-string" as a tree insted of the text. Make the key bindings do actions on the tree instead of the text. But use a rendering function that converts to text for display and saving. Using this approach, the only thing lost seems to be the flexibility to make random text, but if the tree-to-text function is good, that should not be needed. I actually have a little urge to try it out. I just need some more time to play 🙂 I think a prof of concept is not that hard to do, but making it nice, might take a bit longer.