Clojurians Log v2

@jeroenvandijk there are some errors where boot internally (like catch java errors) that clojure 1.10 now returns as nil

Thank you. One of my tests was failing due that nil value. I just changed the code to make it pass, but I couldn't find where it was coming from. I guess it is during the eval-ing of the repl. Although that would not explain why it was failing during my (custom) boot test task. Ah well

I'm having a problem with the delay in the update of boot fileset again. An external file watcher process (hawk) is too fast compared to boot's fileset updates. Is there a way to use boot's watch task to run an arbitrary function?

you could turn your function into a task and include it in the pipeline after the watch task

i.e. boot watch do-my-thing

do you know how I could do it from the repl? I'm a bit struggling with the function composition

something like this (watch (my-other-task)) ?

(boot (watch) (my-other-task))

each task is a function that returns a function

so that they can compose together in a pipeline

Thanks!

Is there a way to cancel the watch process?

Maybe not very clean but this seems to work:

(def x (future (boot (watch :verbose true) (fn [& args] (println "fooo" args)))))
(future-cancel x)

i've seen it done that way

This is pretty cool actually 🙂

I also see why my previous (Thread/sleep 25) didn't work. For this project a fileset update takes around 90ms

[Ignorant question] Is that a hard problem to solve?

this reminds me of a post @alandipert wrote on the adzerk blog a couple years ago: https://adzerk.com/blog/faster-clojure-development/

I think it's actually more than 90ms as that's what the watcher process reports, but then there is also the picking up of the change it self (through polling?)

ah cool i'll have a look. Thanks @dave

Compared to my alternative implementation, independent of boot, with hawk (https://github.com/wkf/hawk) This is a significant slow down

hmmm ok i think i can see why it is slow

This is being done every 10ms https://github.com/boot-clj/boot/blob/master/boot/pod/src/boot/file.clj#L244

Here is the file watching deamon https://github.com/boot-clj/boot/blob/master/boot/core/src/boot/core.clj#L742-L751

I'm guessing that could be made a lot more efficient with a different approach like hawk

@flyboarder Do you have opinions on the above? I could try to make this a bit faster

hmmm maybe i don't understand how it works. I'll look into it a bit more before i say stupid things

This is how I understand it: There is a native (fast process) implemented in boot.watcher. However due to the LinkedBlockingQueue there is a default polling latency of 10ms (https://github.com/boot-clj/boot/blob/master/boot/core/src/boot/core.clj#L749). This latency is for any project. On top of that and this hurts bigger projects is a function that runs a fingerprinting process of the whole tree (watcher, boot/core.clj#L751). This function goes over all files on every change https://github.com/boot-clj/boot/blob/7709941bcef20de55b940ec18885677df33b7557/boot/pod/src/boot/file.clj#L247-L248 This is unnecessary as the native process implemented in boot.watcher would be able to tell exactly what file has changed. I'm sure i'm missing something either in the problem space or in my above analysis

I guess there is some background story behind this comment https://github.com/boot-clj/boot/blob/master/boot/worker/src/boot/watcher.clj#L87-L88

@jeroenvandijk this is an area I want to improve for sure! There is work towards a cleaned up version of the fileset (fileset branch on github) which I feel will help simplify the watch task

But I agree, my large projects are now taking a minute to build even tho the compilation step is less than a second

I also have to wait a while before changes are picked up and a recompile is triggered

So I think our current approach is flawed

Actually looking at the watcher, it should probably be rewritten anyway, it falls under the remove :boot/from code goal for v3