Clojurians Log v2

latest grammar modifications include adding fields to make picking things out of syntax nodes easier. metadata and discard forms can now be passed over by looking for fields that are named 'value'. this gives the ability to work with metadata and discard forms, but at the same time not have to have such complicated logic to sift through those things when what one is interested in is the non-metadata, non-discard stuff -- which seems like what one would want typically. use of fields comes at a cost though. parsing clojure.core is now in the 30 ms range when using the rust bindings and in the mid 70 ms range for wasm. i modified the code to locate def-related forms to work directly with the tree produced by the latest grammar (so there is no longer any secondary tree produced). now instead of 270 ms for clojure.core, it typically completes in the 120 - 140 ms range. these numbers are for the wasm bindings.

i'm attempting to use the grammar to implement a non-caching jump-to-definition implementation -- through this work i'm hoping to identify things that might help shape the grammar in various ways. so far it seems to be working (e.g. addition of fields and specifically which things to use fields for and where to position them).

tentatively, i'm aiming to have something that can work in under 1 second. the general idea is something like: 1) receive a symbol with location information from a file of clojure code 2) parse the file and locate definitions within it (+ interpret ns-related info within the file) 3) determine if the file contains the definition for the target symbol, if yes, jump to one of the locations determined in step 2 4) if the file does not contain the definition, determine which namespace it was defined in via the info in step 2 5) using a previously prepared table of ns to filepaths, determine a path to a file containing the namespace that should contain the definition 6) read in the file, parse it and locate the definitions within it 7) assuming the target definition is there, jump to it this won't work all of the time, but my guess is it will work often enough to be useful steps 2 and 6 involve parsing clojure and locating definitions -- so the measurements above are relevant for this. i'm not sure how long interpreting ns-related info will take yet -- i might try to write something manually or convert the parsed tree into clojure data structures to apply something from tools.namespace or adapt something from clj-kondo.

the table mentioned in step 5 somehow needs to be constructed -- i know clj-kondo's analysis mode can provide this type of info and measurements on clojure.core suggest it can take more than half a second to just generate the analysis. however, that only needs to be done once though the analysis info needs to be massaged.

a given code base can run using different classpaths though, so strictly speaking the user of this should be somehow involved in selecting exactly what classpath to use. the classpath is necessary to determine namespace definitions.