clojure

New to Clojure? Try the #beginners channel. Official docs: https://clojure.org/ Searchable message archives: https://clojurians-log.clojureverse.org/
2020-12-24T06:19:08.419800Z

I'm trying to do a workflow-reloaded-esque thing with a remote nrepl server in a kubernetes pod/skaffold to sync source changes into the container. I expose a static nrepl port and port-forward the kubernetes service and connect from my laptop. I'm using clojure.tools.namespace.repl/refresh, and I'm noticing that sometimes it works, and sometimes it doesn't. Locally it works perfectly. Usually it reloads nothing:

user=> (clojure.tools.namespace.repl/refresh)
:reloading ()
:ok
The holy grail is to have a repl running in a kube pod exposed with a TLS cert and all the niceties. There are probably a lot of variables with my setup, but I'd appreciate any advice on troubleshooting this.

vemv 2020-12-24T18:45:06.443600Z

not a direct answer to your question, but IIRC a successful setup I once used consisted in having the JVM run locally, and everything else in one's k8s. That way code reloading, IDE integration, etc all need zero changes With some port forwarding in place, your local JVM connects to a dockerized db, redis, etc

vemv 2020-12-24T18:46:47.443800Z

(incidentally my current setup is also like that, with the difference that db/etc run in docker-compose instead of k8s)

vemv 2020-12-26T12:18:21.482300Z

Nice find! Yeah imo refresh-dirs should be mandatory to set, it's not rare for people to encounter issues b/c of its default

2020-12-24T06:22:58.421400Z

The doc string for refresh says it looks at the last modified times of files to determine what to reload

2020-12-24T15:01:52.423900Z

the files in the container appear to have reasonable timestamps. The host is set to UTC and all the files are UTC, and when I connect to the repl in the pod and run (new java.util.Date) I get the current time in UTC, so I don't think there is any weirdness going on with the timestamps

2020-12-24T16:52:41.426800Z

But it doesn't matter if the file has a timestamp and it is correct, what matters is the relationship between timestamps

2020-12-24T16:53:31.428300Z

The changed files need to have a last modified after the internal timestamp that refresh keeps of the last time it was run

2020-12-24T16:58:29.430400Z

Ah, you say you are using nrepl to load code into the pod? There is your issue right there

2020-12-24T16:59:56.432700Z

While your nrepl client knows how to push code locally to a remote nrepl server, there is no way for refresh, code running on the remote server jvm to get the local files

2020-12-24T17:00:53.433800Z

This all works fine locally for you because your nrepl client and server are looking at the same filesystem

2020-12-24T17:48:46.438400Z

I’m using a third party tool called skaffold that watches the local files and mounts them into the pod, so it is able to pick up file changes, ie I can make a change locally, exec into the pod, and see the changed file and updated timestamp. So I think the nrepl server can see the updated code. It works “sometimes” but I haven’t been able to figure out those conditions. It’s one of those weird “it was working yesterday but not today” things

2020-12-24T17:57:45.443300Z

Ok, so check the timestamps on the synced files, are they after whenever you past refreshed? Are they finished synching before you try and refresh? Is the last modified time on the remote end the time the files were synced, or is it last modified attribute copied from the local files, if the latter you may be racing (running refresh before the sync happens, but after the modifications)

2020-12-24T20:14:48.446700Z

I find myself using the x/reductions transducer (https://github.com/cgrand/xforms/blob/62375212a8604daad631c9024e9dbe1db4ec276b/src/net/cgrand/xforms.cljc#L491) from the xforms library together with (drop 1) a lot to discard the init value and only use the subsequent reductions from the input, such as:

(into []
  (comp
    (partition-all 3)
    (x/reductions conj [])
    (drop 1))
  (range 10))
Just want to verify, this should work as expected with only the initial value being dropped no mater which transducers are used before or after the x/reductions/`drop` pair (as long as those are consecutive) right?

2020-12-24T20:59:19.446900Z

The timestamps are after the last refreshed, and are finished syncing before I run refresh . Skaffold uses fsnotify and ships the files over as they change, so the timestamp is the same as when they were changed locally.