clj-kondo

https://github.com/clj-kondo/clj-kondo
snoe 2021-01-23T19:56:50.036900Z

@borkdude can we chat about https://github.com/clj-kondo/clj-kondo/issues/1129 ? I think my view comes down to (s/def ::a) and (def a) are very similar, and in my projects, I would treat both a and ::a as definitions even though clojure or spec, under the hood, is just storing some state for the var and the keyword. In lsp. I appreciate that kondo analysis differentiates between var-usages and var-definitions, and, I think, there's an argument to be made for keywords behaving similarly.

snoe 2021-01-25T05:22:08.048800Z

I think the reasoning for keeping it all in a single entry is because the interpretation of a keyword as a def is up to userspace.

snoe 2021-01-25T05:27:37.049Z

> And the editor needs to support listing multiple choices when you do `go to definition`. I can do that in Anakondo easily, not sure if LSP supports this? The protocol seems to allow for multiple definition results it will depend on the clients, as always, to support it.

2021-01-25T22:11:20.049300Z

> I think the reasoning for keeping it all in a single entry is because the interpretation of a keyword as a def is up to userspace. Ok, but I mean we're talking in either case, userspace needs to configure clj-kondo for it no? But, ya ok, I see the point, like it might look weird for the map to be empty most of the time, unless userspace configured some keyword defs

2021-01-25T22:12:24.049500Z

I guess from a consumer perspective, it seems like you'd have a different way to parse symbols from keywords for "go to def" and "find references". Which is why I was wondering like if that made sense.

borkdude 2021-01-25T22:13:10.049700Z

@didibus the idea is that a keyword in and of itself doesn't mean anything. e.g.:

(defn foo [x])
(foo :bar)

;; vs

(defn reg-something [k])
(reg-something :bar)
are analogous, whereas
(def x 1)
has definition meaning regardless of what semantics users give to their functions

borkdude 2021-01-25T22:14:41.050Z

This is something that isn't fully clear to me yet, that's why I propose keeping this analysis output private until it's been used by LSP for a while

borkdude 2021-01-25T22:15:14.050200Z

we could make them separate, but having a list of all occurring keywords seems useful to have anyway for renaming purposes

2021-01-25T22:19:34.050400Z

I feel like either or, we're still adding the concept of keywords can be considered as having a definition no? Even though by default they don't in Clojure, but then per-library or user usage they might. I feel for me its more what's easiest to parse in the analysis structure for the common use. I guess its a small detail, either or you can parse out the information. But say someone runs: "go to definition" on a keyword, now you'd filter through the keyword usage for that keyword for the one with "def = true".

2021-01-25T22:20:28.050600Z

Where as if it was a separate map, you'd look it up in keyword-usage map

2021-01-25T22:21:29.050800Z

Actually, sorry, maybe that's more a general critique. I remember now. When I implemented anakondo, I think I wished that things were pivoted, instead of vector of maps, it be maps of vectors of maps

2021-01-25T22:22:06.051Z

Where the map is keyed on the name

2021-01-25T22:22:47.051200Z

I found needed to actually parse everything back into maps indexed on names for the different constructs.

2021-01-25T22:23:38.051400Z

Anyways, might have missed the boat on that now.

2021-01-25T22:25:11.051600Z

I'll be happy either way, if :def is on the keyword map or in a separate one.

borkdude 2021-01-25T22:25:25.051800Z

> I feel for me its more what's easiest to parse in the analysis structure for the common use. That's a good point

borkdude 2021-01-25T22:27:15.052Z

@didibus the reason you get everything as a list is that it's not always clear what the index should be. either clj-kondo does this for you (and has to guess what is the right format), or you do it yourself. if clj-kondo would do this for you and you wanted a different index, then it would be hairy to go from one index to another. the work has to happen somewhere and it's most flexible if that somewhere is the end user

borkdude 2021-01-25T22:27:40.052200Z

people might also want to index on filename, for example, or namespace and then name, etc

2021-01-25T22:28:21.052400Z

Ya, that's true. Actually I can't remember fully, but I probably did end up indexing in a slightly more custom to my needs way.

2021-01-25T22:28:59.052600Z

I think had it been in Clojure, I wouldn't have feel the pain 😛, but transforming datastructures in Emacs Lisp is not as simple haha

borkdude 2021-01-25T22:29:17.052800Z

write a graalvm binary ;)

borkdude 2021-01-25T22:30:00.053Z

I think now you could do this part using babashka

borkdude 2021-01-25T22:30:07.053200Z

using clj-kondo as a pod

2021-01-25T22:30:40.053400Z

Ya, I've been considering it. The thing is, with any separate process option, now I need to figure out a protocol to communicate between the editor and that, and the editor needs to manage a long running process, maybe multiple if you have multiple project, etc. And I think there's a lot of brittleness there, and also I'm just re-inventing LSP at this point.

borkdude 2021-01-25T22:30:57.053600Z

we could even let clj-kondo take some processing function that is executed using sci, since it already has sci

2021-01-25T22:31:35.053800Z

Hum... like pass it a post-processor on the analysis map, that could be nice.

borkdude 2021-01-25T22:31:46.054Z

yeah

2021-01-25T22:32:16.054200Z

So you could use Clojure to re-structure it however is best for your use case, and then your editor gets a JSON in that new structure.

borkdude 2021-01-25T22:32:34.054400Z

right

borkdude 2021-01-25T22:33:03.054600Z

I think the function could be applied to the :findings as well, just one function that gets the end result before it's written to EDN or JSON

2021-01-25T22:33:50.054800Z

Ok, well, it would be cool. I don't know if I'd say its at the top of the Wishlist though 😛 Even though I'm here complaining about how painful it is to do in Emacs Lisp, that's pain I chose for myself haha

borkdude 2021-01-25T22:34:48.055Z

I think we have to rewrite emacs to clojure one day ;P

1
2021-01-25T22:35:36.055300Z

The only thing I can give props to Emacs Lisp, its use of memory is quite impressive, like its very memory efficient

snoe 2021-01-26T01:11:46.057700Z

in clojure-lsp, we probably have 4 or 5 things to index on, so I just flatten everything into a singular list and search. if perf becomes. a problem we can reify the indexes we need.

2021-01-26T02:40:12.057900Z

Even for auto-complete?

borkdude 2021-01-23T19:58:07.037400Z

This is very library specific and not a core clojure construct

borkdude 2021-01-23T19:58:25.037600Z

So if you would do something like this, you would have to include all kinds of library specific code

borkdude 2021-01-23T20:00:38.037800Z

e.g. def is a special form in clojure, but any macro can make up any kind of def thing using strings, symbols, keywords, etc. This is user-land behavior

borkdude 2021-01-23T20:01:49.038Z

What I think would be generally possible is a list of all keywords, with filename, location, source form (what we call :str so far) and maybe, for some libs we could add a key that this is the defining location or something

borkdude 2021-01-23T20:01:53.038200Z

But this will be very custom

borkdude 2021-01-23T20:02:13.038400Z

for clojure.spec it would be a no-brainer

borkdude 2021-01-23T20:02:29.038600Z

but what other libs are using this construct? we can't possibly support all of clojars?

snoe 2021-01-23T20:04:40.038800Z

yeah, I agree, it's user-space. But clojure.spec is pretty close to core. And the concept of using a keyword to mean something is generally applicable. One of the things that I ask myself about these analysis prs, is how would one implement linting from this feature?

borkdude 2021-01-23T20:05:28.039Z

When you look at the namespace graph, one could maybe derive the first mention of the keyword

borkdude 2021-01-23T20:05:57.039200Z

but since keywords (e.g. defmethods) are usually used to decouple things, this isn't reliable

snoe 2021-01-23T20:06:03.039400Z

writing a sci macro for other libraries that maybe does`(s/def ~name)` would be enough to support re-frame/reg stuff

borkdude 2021-01-23T20:06:48.039600Z

wouldn't it be sufficient to get all mentions of a keyword (regardless of alias)?

snoe 2021-01-23T20:08:39.039800Z

for lsp it would be sufficient for rename, but there'a goto def/find references (that generally doesn't include the def) that the distinction would be useful for. for linting, I imagine the distinction would allow you to say a spec is unused.

borkdude 2021-01-23T20:11:32.040Z

If I would have this use case of unused-ness of a spec, I think I would personally just go through the list of all occurrences of this keyword. Same for re-frame events. Keywords that only occur once are suspicious (for specs, but in general, these are usually typos).

borkdude 2021-01-23T20:13:56.040200Z

(ns foo)
(s/def ::bar int?)
:foo/bar
I think it's a bit weird to say ":foo/bar" is defined somewhere. I need to think about this a bit more.

snoe 2021-01-23T20:14:49.040400Z

yeah, I agree it's definitely a different way to think about it. When I added that to the lsp parser though, it made a big difference in usability.

borkdude 2021-01-23T20:15:09.040600Z

was this a feature that was supported in LSP before?

snoe 2021-01-23T20:16:42.040800Z

yup. my macro-defs were able to treat a call to a symbol like {clojure.spec.alpha/def [:declaration :element]} and I would tag the first arg (the keyword) as a :declaration . Kondo obviously has a different model.

borkdude 2021-01-23T20:20:15.041100Z

So it would be useful to have this in hooks then I guess.

(reg-keyword-def ....)
or something

borkdude 2021-01-23T20:20:42.041300Z

and have this built-in for spec

borkdude 2021-01-23T20:20:54.041500Z

brb

snoe 2021-01-23T20:20:57.041700Z

in the only one usage lint model, I think you'd have troubles if someone simply forgets to s/def so if the user just (->> x (s/validate ::a) (s/conform ::a)) it's moving to a very heuristic approach.

borkdude 2021-01-23T20:22:03.041900Z

so maybe a list of all keywords and some have :def true ?

snoe 2021-01-23T20:22:06.042100Z

I haven't looked at hooks much, but yeah, that might do it. It could add an extra flag to :keywords or we could do a similar split of usages/definitions that vars use

snoe 2021-01-23T20:22:14.042300Z

aye, that works for me

borkdude 2021-01-23T20:22:31.042500Z

re-frame calls this :reg but the intent might be clear

borkdude 2021-01-23T20:22:45.042700Z

alright

snoe 2021-01-23T20:23:00.042900Z

spec might call it a registry as well.

borkdude 2021-01-23T20:23:11.043100Z

that's true

borkdude 2021-01-23T20:23:25.043300Z

ok, we can think more about the name, but I think we have an idea what should be done now

snoe 2021-01-23T20:23:43.043500Z

yup, thanks a bunch!

borkdude 2021-01-23T20:23:49.043700Z

I'll ask some other people too what they think of this

borkdude 2021-01-23T20:26:15.044200Z

What if both re-frame and spec use the same keyword to register something?

borkdude 2021-01-23T20:26:23.044400Z

Then you would have two :def true

snoe 2021-01-23T20:27:19.044600Z

could be reg-by set, in the code I've run into it hasn't been a concern but it's a good question.

borkdude 2021-01-23T20:28:16.044800Z

:def clojure.spec.alpha

borkdude 2021-01-23T20:28:22.045Z

:def re-frame.core

borkdude 2021-01-23T20:28:23.045200Z

?

borkdude 2021-01-23T20:28:36.045400Z

:registered-by ...

snoe 2021-01-23T20:28:45.045600Z

could be, not sure how hard that would be

borkdude 2021-01-23T20:29:12.045800Z

Maybe :registered-by clojure.spec.alpha works. :registered-by clojure.core/add-watch

borkdude 2021-01-23T20:29:42.046Z

well in hook code this is easy, since you know what macro you are dealing with

snoe 2021-01-23T20:33:29.046200Z

ok I can start moving the pr towards this.

borkdude 2021-01-23T20:51:08.046400Z

yeah, we can do :def <truthy> where truthy is either a boolean or the macro / fn that registered it

borkdude 2021-01-23T21:24:30.046600Z

@snoe Since this analysis isn't fully formed in our heads yet, I think the best way forward is to add it as discussed and also write some docs, but comment out the docs (using HTML comments) so it's only private to both clj-kondo and clojure-lsp for now, so we have freedom to make changes until we are satisfied.

👍 1