Clojurians Log v2

@sdmoralesma What do you mean from now on? After discovering that clojure.set includes some relational algebra, or has something happened recently?

Help a tool smith, please. What is a good workflow for navigating namespaces that a tool like Calva could support? Calva encourages working from “inside” the source code files themselves, but many people are still using the repl window as well. Observing how people try to navigate namespaces most of them dive straight down the “How things can go wrong” path mentioned here: https://clojure.org/guides/repl/navigating_namespaces#_how_things_can_go_wrong Experimenting a bit with it I find that I can probably add a command to Calva to load the current namespace even though it has been created (by evaluating the code in the file). That would “encourage” this “wrong” path though, I think… Maybe that doesn’t matter as long as it gets the work done?

I think it's quite common to have the IDE able to run a specific "reload" command (like tools.namespace's refresh, a given app's custom reset, etc) under a keyboard shortcut So, as a IDE user I just hit the keyboard shortcut whenever my code is ready. That doesn't preclude using the repl, or even evaluating a form from the file (that's a favorite of mine; as it's a hybrid approach) When I visit a new file that wasn't required before, my emacs does just (in-ns 'that-ns) but does not attempt to automagically load its code That's my responsibility, be it via the keyboard-bound refresh, or various repl/eval interactions

That would seem a sensible approach to me as it's minimalistic/transparent and doesn't impose any specific workflow

Cursive can "load file in REPL", whatever that means. Interestingly, it seems to also load all the dependencies:

;; hgs/xxx.clj
(ns <http://hgs.xxx|hgs.xxx>)

(def a 1)

;; hgs/yyy.clj
(ns hgs.yyy
  (:require [<http://hgs.xxx|hgs.xxx>]))

(println "Hello" <http://hgs.xxx/a|hgs.xxx/a>)

;; REPL
(ns <http://hgs.xxx|hgs.xxx>)
=&gt; nil
(require '<http://hgs.xxx|hgs.xxx>)
=&gt; nil
a 
Syntax error compiling at (/tmp/form-init16785379888913250927.clj:1:350).
Unable to resolve symbol: a in this context
;; I executed the "Load file in REPL" command on hgs/yyy.clj here
Loading src/hgs/xxx.clj... done
Loading src/hgs/yyy.clj... 
Hello 1
Loaded

Thanks. Sounds like I can go ahead with my idea for having a command that loads the current namespace then, from both your answers. In my spike it looks quite exactly like that Cursive workflow.

Like so.

At line 27 there I issued the command. If that is not clear from the screenshot.

Will it also load all the dependencies in the same way?

It evaluates all the code in the file, so its requires will get loaded. If that is what you mean.

Not really - in my example above, the dependency is also evaluated, not just loaded. If I understand it correctly, otherwise evaluating yyy.clj would not create <http://hgs.xxx/a|hgs.xxx/a> because the namespace <http://hgs.xxx|hgs.xxx> already exists.

what happens if both namespaces define a? That specific feature seems more delicate

I think that will work as in your example. Let me try…

@vemv It would be ns1/a and ns2/a - no issue here. The REPL has only one namespace as currently active.

With file contents similar to your example @p-himik

Note that the load file in repl command already exists in Calva. So loading the pez.yyy namespace can be done from the file. What’s new in my spike here is that I can do it from the repl prompt. Not sure if that is what the Cursive example showed.

It's not the same. Note that in my example I execute what would be (ns <http://pez.xxx|pez.xxx>) in your case, not (ns pez.yyy).

Hmmm, that actually exposes an issue in current Calva. If I do (ns '<http://pez.xxx|pez.xxx>) before loading pez.yyy I run into the problem I posted in the thread start. I guess that is quite uncommon in practice, but anyway.

A probably more common workflow that would suffer from the same issue: 1. Evaluate xxx.clj in REPL 2. Make some changes in xxx.clj 3. Start using those changes in yyy.clj (so yyy depends on xxx) 4. Load yyy.clj in REPL 5. Receive an error

Also, I have no idea if Calva handles it but in Cursive I regularly suffer from the situation described here: https://nelsonmorris.net/2015/05/18/reloaded-protocol-and-no-implementation-of-method.html Is it possible for Calva to automatically reload relevant namespaces to avoid that error?

I know that it's not really about the initial topic but it seems quite relevant given that it's also about loading files and their dependencies.

I filed an issue on Calva now, please ensure that I have understood the problem correctly and let me know if I haven’t: https://github.com/BetterThanTomorrow/calva/issues/907

LGTM, thanks!

What happens in step 3 in the workflow you describe there?

By "start using" I meant just writing new code.

So that should work in Calva w/o error, but now I am a bit more on the defence here so must try before confirming. 😃

Please do tell how it turns out. :)

You were right with your step 5 unfortunately . 😃

At least the reason is known, so this confirmation just increases the priority, I guess.

But in this case I can reload <http://pez.xxx|pez.xxx> and things start to work. I think I do this more or less on auto-pilot.

Right.

But it would also work in the original workflow outlined in that issue #907.

I can probably get out of the mess in the reported issue by reloading <http://pez.xxx|pez.xxx>… yeah, you beat me to it.

So, fixing #907 should fix this too, I think. And fixing #907 would mean to help the user with loading the files in order. I wonder if the current command Calva has for refreshing namespaces actually does that… must check.

Nope. Hmmm…

Now to your other question there. I’m pretty sure, given #907, that this will be the same problem in Calva. Will try it now to see for myself.

So, Calva doesn’t offer any help with this currently. How would you like it to be supported?

No idea as of now - I started evaluating Calva just a couple of days ago (IntelliJ IDEA pisses me off more and more with each release). I guess it should be as non-intrusive as possible. I.e. in the article that I linked, z.core should be reloaded automatically as well. But the user has to be notified about it in an obvious way, ideally with a reason describing why z.core was reloaded at all.

I am reluctant to make Calva load things automatically. It would certainly “help” with many common support questions we get, but I really think the user must be in charge of what is being loaded, and when.

Without doing it automatically, how would you fix 907?

But a command for reloading all files that depend on a given namespace.

With such a command, would it be possible for a user to make it automatic via configuration?

I hear you about 907, but I see it as that the command is named Load Current File and Dependencies it sort of promises something it doesn’t keep.

So, a setting where Calva always loads dependent namespaces on Save, you mean?

I don’t think I can do it on mere re-evaluation…

> reloading all files that depend on a given namespace It wouldn't be necessary. I think only the files that use the protocols and other similar entities would need to be reloaded. > it sort of promises something it doesn’t keep Not sure I follow. If you do load everything, what is the promise that's not kept? > So, a setting where Calva always loads dependent namespaces on Save, you mean? It may be a setting. Ideally, it would be something that's scriptable. Sorry, I have no idea yet whether it's at all doable but AFAIK you can write CLJS code to extend Calva and/or VSCode (a huge advantage compared to IDEA+Cursive).

About 907. Given that the dependency <http://pez.xxx|pez.xxx> is not loaded by the command, (if the namespace has been created), I see that as a bug.

If we can target the reload to protocols, and similar, then I might be persuaded to make it an automatic thing, but I haven’t thought it through yet.

My question regarding Save was that that would be enough automatic. The way my workflow is leaves me quite often with a lot of unsaved files, because I mostly just evaluate my programs to existence. 😃

There is currently a setting where Calva evaluates the file on Save. So if we introduce a command that dependent files be re-evaluated, then the auto-setting for that would be in the same spirit.

Ah, I think I misunderstood one thing - I wouldn't want for anything to happen automatically on Save. I also prefer to issue the command to reload stuff manually. By "automating" I meant automatically reloading every file that needs to be reloaded to keep the app working when a user manually reloads one file.

Ah, then we're on the same page. Once a command behaves in a predictable manner I'm all for automatic. 😎

What Cursive does is, if you reload file X, and it depends on file Y, and you changed file Y but did not reload file Y, when you reload file X, Cursive will also reload file Y

Which avoids a lot of the "what can go wrong"

Also, I think it's great that you can run the command from the repl as well, so it's more like reload namespace

Cider also has something similar: cider-ns-refresh will reload all modified files

It's not only about the modified files. It's also about namespaces that have been created but were not populated properly. Although as pez said, this use case is probably quite rare.

What do you mean by not populated properly?

That you forgot to load the rest of the namespace code? Or that loading threw an exception?

I meant executing (ns xxx) in a REPL, like described in examples above.

Oh, so you switch the reply namespace manually by executing (ns xxx) and then try to call a function in that namespace, but it doesn't exist because you actually never loaded the corresponding file?

No, it's not about one namespace. Have you checked the examples above?

Ok, I was looking at the prior example, I see now.

So you mean, it be better to have it load all dependencies, not just modified ones?

I think so. Although I'm not sure how that would work in a workflow with global state atoms or something like that.

State is always the downsides to these "auto-sync" like features.

Right. So should be fine as long as users know what they're doing and the consequences of their actions.

Ya, but changing namespace in the repl is pretty rare no?

Also, I do think Cursive has both Load and Load All no?, And the latter would reload all

Above, we have agreed on this, yes. My "although" above is about the whole "reload files that were not directly reloaded by the user" thing. Regardless of the reasons for their reloading. I don't see "Load All" in the list of REPL actions in Cursive.

I was wondering, tools.namespace also unloads, but what if it didn't? Wouldn't that solve most of the caveats it has, at the expense of you could miss the fact one function calls something that no longer exists.

@didibus you'd need some kind of manual override if you changed a defmulti though, for example

True, I could get in the habit of having a (def my-multi nil) above them

But I'm also wondering here, maybe tools.namespace could be smarter and unmap defmultis but not the rest?

I also wouldn't be too bothered if when I changed a defmulti I still needed to manually deal with it

Are there other cases like this? Apart from defmulti?

Hum, I might have been mistaken. Haven't used Cursive in a while

So in Cider, you have: refresh and refresh all, which run the tools.namespace refresh and refresh-all. Both of those should work in the example given. You also have reload and reload-all. Those are based on require :reload and require :reload-all respectively. I think both of those would also work.

And finally you have Load buffer, which is just like sending all the code in the buffer to be evaled. That one would not work, since it won't force reload a lib that is already loaded.

Is it possible to force Clojure to resolve import statements at compile time? Currently import statements produce bytecode that looks like

class_ = ((Namespace)RT.CURRENT_NS.deref()).importClass(RT.classForNameNonLoading((String)"a.java.class"));

You could write

(def a-fn (memoize (fn [x]
                    (println x))))

Whoops sorry, I misspoke. My problem is memoizing no responding to recursive calls - only the outer call. Any idea?

Sorry, I figured it out…thx