Clojurians Log v2

@jeroenvandijk how stable has that been?

Is there a boot noop to put into a pipeline when conditionally building it? Like (comp (when (= :env :dev) (draft))?

@jayzawrotny ok so the boot task is like a double wrapped function

when you create a task you are returning a function that returns a fileset

so to conditionally wrap that you would use a cond->

I do it often, one sec I have demos

Ah right, the middleware so would need to be like (comp (if (= :env :prod) (draft) (noop)) where noop is like (fn [next-handler] (fn [fs] (next-handler fs))

yeah I find the threading macro easy tho

(cond-> (identity)
  (= :env :dev) (comp (dev)))

that way I can build up a pipeline of conditional tasks

I could also see a really easy reduce-task that takes a coll of tasks and reduces them to a pipeline

(comp
  (perun/global-metadata)
  (perun/markdown :md-exts {:all true})
  (perun/draft)
  (perun/print-meta)
  (perun/slug)
  (perun/ttr)
  (perun/word-count)
  (perun/build-date)
  (perun/gravatar :source-key :author-email :target-key :author-gravatar)
  (perun/render :renderer '<http://eccentric-j.site.post/render|eccentric-j.site.post/render>)
  (perun/collection :renderer 'eccentric-j.site.index/render :page "index.html")
  (perun/tags :renderer 'eccentric-j.site.tags/render)
  (perun/paginate :renderer 'eccentric-j.site.paginate/render)
  (perun/static :renderer 'eccentric-j.site.about/render :page "about.html")
  (perun/sitemap)
  (perun/rss :description "Eccentric J's Blog")
  (perun/atom-feed)
  (sass)
  (cljs :optimizations (if (= build-env :production) :advanced :none))
  (clean :exclude #{#".git"})
  (target :no-clean true)
  (notify)))

I think the correct answer to your question tho is just return an identity fn as the task, so it returns the previous tasks fileset

Otherwise I’m going to have (cond->) all over the place.

Thanks, though you got me thinking I need to rethink my approach

ok I see what you are doing, take a look at what I have done here: http://github.com/degree9/meta

https://github.com/degree9/meta/blob/master/src/meta/boot.clj#L42-L54

those tasks use the thread approach, it seems you probably want to breakup the pipeline into smaller sub-tasks

Ok, I was just starting to think that 🙂

[meta] seems really nice! I may have just found a stack for clojure desktop app dev!

yeah we are using it everyday

(deftask build
  "Build the blog source and output to target/public"
  [e build-env BUILD-ENV kw    "Environment keyword like :dev or :production"]
  (let [prod? (= env :prod)]
    (->> [(perun/global-metadata)
          (perun/markdown :md-exts {:all true})
          (when-not prod? (perun/draft))
          (perun/print-meta)
          (perun/slug)
          (perun/ttr)
          (perun/word-count)
          (perun/build-date)
          (perun/gravatar :source-key :author-email :target-key :author-gravatar)
          (perun/render :renderer '<http://eccentric-j.site.post/render|eccentric-j.site.post/render>)
          (perun/collection :renderer 'eccentric-j.site.index/render :page "index.html")
          (perun/tags :renderer 'eccentric-j.site.tags/render)
          (perun/paginate :renderer 'eccentric-j.site.paginate/render)
          (perun/static :renderer 'eccentric-j.site.about/render :page "about.html")
          (perun/sitemap)
          (perun/rss :description "Eccentric J's Blog")
          (perun/atom-feed)
          (sass)
          (cljs :optimizations (if prod? :advanced :none))
          (clean :exclude #{#".git"})
          (target :no-clean true)
          (notify)]
         (keep identity)
         (apply comp))))

Going this route, I think this should keep things pretty clean. I didn’t see a natural split in the steps because they all work towards building the blog. Without any of these steps I’m not sure the result would be meaningful.

If it builds ™️

Is there a way to get watch to respond to changes in a target/public directory instead of the resources dir?

@jayzawrotny no, by design

the target dir is not immutable, it's the final IO at the end of the pipeline

Sounds like I’m trying to use it incorrectly then.

one of the reasons why tasks like watch can be made simple and reliable is that they only manipulate immutable filesets

so you can "roll back" to a previous state easily

The intention was to serve the files compiled to target/public as a website and when those files change run through boot-livereload. Sounds like I need to put those in the build pipeline.

right

you can use :asset-dirs or :resource-dirs

Actually, let me back up. I’m learning boot while developing a static site with http://perun.io. I had a watch task that would build the markdown files, gen the html files, compile the scss and the cljs. But I’m curious if I could set it up to be a bit smarter. Currently if I change a markdown file both the scss and cljs get recompiled.

are you sure they're really getting recompiled?

so there are two usual approaches to build tooling

Well the tasks are firing and output is being logged

you have an approach like Make

where it computes dependencies and triggers tasks that need to run

boot doesn't do it that way

boot always runs the pipeline every time

but each task can hold a reference to the previous immutable fileset

and it can efficiently diff the files

so each task is responsible for figuring out for itself whether it needs to do any work or not

so while those tasks will run, they probably are not actually doing any work

they diff the new and old filesets and see that no scss files changed, and just pass the cached results from the previous run onward

I see, unfortunately they do slow down the recompile a bit. It’s not the end of the world though.

perhaps there is a reason why they need to recompile something?

or perhaps there is a bug in the task that can be found and fixed

I don’t see anything that strikes me as diffing https://github.com/Deraen/boot-sass/blob/master/src/deraen/boot_sass.clj

No wait, I see it

You’re right

line 41

no line numbers there but see the fileset-diff part?

Yeah I see it now

yep exactly

Very cool!

the reasoning there was that only the task itself really knows what it depends on

it can get pretty complex

the build tool can't really know

Good point. Also, I think I see the issue. It’s setup to diff the entire fileset which is why it’s always recompiling for me. Does this mean I could create another task to compose a sift with just .scss files with boot-sass to make it more efficient?

do you have a large number of scss files?

i know sometimes those things can have a ridiculous number of files

Nope, so far just one but if I was using this build system for a bigger site it’s possible.

it's slow with only a small number of files?

the diffing is usually quite fast

since it's all immutable it can do various tricks

I don’t think it’s the diffing that’s slow, I’m just saying if I change a markdown file: the fileset this task receives will be different than the previous fileset it diffs against and will recompile

it will only recompile if a scss file changes though

at least that's how it looks from the code

the (when (seq sources) ... part

Right, and sources is the result of diffing filtered by the sass extensions

sources is the .scss files that changed

right

seems like it should skip recompiling if only a markdown file is changed, no?

Agreed, boot-sass is definitely designed to work that way.

I’ll try again tonight by removing previous steps from the build pipeline until it behaves as expected.

you can try boot -vv ... too

that will log a bunch of debug messages that might show something

Thanks

👍