babashka

https://github.com/babashka/babashka. Also see #sci, #nbb and #babashka-circleci-builds .
borkdude 2021-02-06T10:42:59.343400Z

user=> (map str (fs/which "java" {:all true}))
("/Users/borkdude/.jenv/versions/11.0/bin/java" "/Users/borkdude/.jenv/shims/java" "/usr/bin/java" "/Users/borkdude/.jenv/shims/java")
😃

borkdude 2021-02-06T10:44:05.343900Z

Oh, this is also funny:

user=> (map str (fs/real-path "."))
("Users" "borkdude" "Dropbox" "dev" "clojure" "babashka" "fs")
Path implements iterable so it works out of the box with map, etc.

👍 1
borkdude 2021-02-06T13:14:07.344600Z

Alrighty, babashka.fs alpha 4: https://babashka.org/fs/babashka.fs.html Please gloss over the docs and try it out from clojars if you have some time. I think this is the first release that will be in babashka soon. Not planning any changes unless feedback.

borkdude 2021-02-07T09:12:15.369400Z

I think it might be better to remove the follow-links behavior in delete-tree as this seems uncommon. Not even rm -rf supports it.

borkdude 2021-02-07T09:14:14.369900Z

Or I don't know. I'll ask a second opinion on this. There might be use cases, but it seems dangerous

lread 2021-02-07T13:03:03.375500Z

fwiw, I have a bb script that does recursive delete, and I have it check for symlinks first and fail if it finds any. This might have been before I realized a delete only deletes the link... just the same I was obviously feeling very cautious about the danger accidentally over-deleting something.

borkdude 2021-02-07T13:06:17.375700Z

right. so same as rm -rf

borkdude 2021-02-07T13:06:25.375900Z

not including the option anymore

lread 2021-02-07T13:12:57.376400Z

cool, good thinking

lread 2021-02-06T16:14:36.345100Z

Very cool, I will certainly make use of this!

lread 2021-02-06T16:20:23.345300Z

Minor: the docstring for directory-stream states that it “Returns a stream of all files in dir” - maybe make it clearer whether or not it includes files in subdirs too?

borkdude 2021-02-06T16:24:58.345500Z

Yeah, I might just make this function private since I don't think it will be useful to anyone really. It's more like an impl detail

borkdude 2021-02-06T16:25:39.345700Z

I think most people want list-dir

borkdude 2021-02-06T16:25:58.345900Z

It's not recursive. For recursive you want glob

borkdude 2021-02-06T16:26:05.346100Z

I'll add that to the docstring

borkdude 2021-02-06T16:26:41.346300Z

"Returns all paths in dir as vector. For descending into subdirectories use glob."

lread 2021-02-06T16:27:58.346500Z

If find docstring for file-name a bit confusing “Returns farthest component from the root as string, if any.” The name of the fn tells me the intent but the docstring kind of makes me scratch my head. The javadoc for .getName might be closer “Returns the name of the file or directory denoted by this abstract pathname.” Oh, does that mean file-name might not be a great name for the fn?

lread 2021-02-06T16:29:20.346700Z

glob looks handy!

borkdude 2021-02-06T16:31:41.346900Z

Do you have a different suggestion for the name?

borkdude 2021-02-06T16:31:44.347100Z

or docstring

borkdude 2021-02-06T16:31:52.347300Z

it's basically foo/bar/baz -> baz

lread 2021-02-06T16:32:43.347500Z

Maybe just name? Because it can be a directory name too.

lread 2021-02-06T16:33:19.347700Z

And that matches Java naming so would be perhaps familiar.

borkdude 2021-02-06T16:33:23.347900Z

ok. I was hoping to avoid clashes with clojure fn names, but yeah, name might be the most clear

lread 2021-02-06T16:33:31.348100Z

Oh… right.

borkdude 2021-02-06T16:34:07.348300Z

note that clj-commons/fs has "base-name" which I also found confusing

lread 2021-02-06T16:34:41.348500Z

Yeah. To avoid a clash maybe name-from? Or is that too contrived?

borkdude 2021-02-06T16:34:55.348700Z

I'll just make it name.

lread 2021-02-06T16:35:38.348900Z

For docstring, maybe more like Java: “Returns the name of the file or directory for x”

borkdude 2021-02-06T16:36:32.349100Z

Oh btw, no, I named it after this one: https://docs.oracle.com/javase/7/docs/api/java/nio/file/Path.html#getFileName() Note that this API is mostly based on Path, not http://java.io.File

borkdude 2021-02-06T16:36:46.349500Z

so actually this name is correct

borkdude 2021-02-06T16:37:11.349700Z

> Returns the name of the file or directory denoted by this path as a Path object. The file name is the farthest element from the root in the directory hierarchy.

lread 2021-02-06T16:37:11.349900Z

Ah… I see where your docstring inspiration came from!

borkdude 2021-02-06T16:37:42.350100Z

I'll just add an example

lread 2021-02-06T16:38:00.350300Z

Yeah, so a directory has a “file name”, I guess.

lread 2021-02-06T16:38:46.350500Z

Still this “The file name is the farthest element from the root in the directory hierarchy.” is hard to parse, at least for my little brain.

borkdude 2021-02-06T16:38:55.350700Z

> "Returns the name of the file or directory. E.g. (file-name \"foo/bar/baz\") returns \"baz\". "

lread 2021-02-06T16:39:17.350900Z

Mucho better!

borkdude 2021-02-06T16:39:33.351100Z

altered.

lread 2021-02-06T16:43:29.351300Z

which will be handy in scripts! Should docstring explain what {:keys [:all]} does?

borkdude 2021-02-06T16:43:55.351500Z

Probably yes

lread 2021-02-06T16:44:29.351700Z

If you were hosting docs on cljdoc I would suggest wikilinks when referring to other fns in docstrings.

borkdude 2021-02-06T16:45:25.351900Z

I'm a bit tired of including cljdoc into the equation because of broken analysis and not being able to update for a given version. I haven't looked at it for a while.

lread 2021-02-06T16:47:20.352100Z

After rewrite-clj v1 is out, I’m probably going to head back to cljdoc to help out with issues.

borkdude 2021-02-06T16:47:48.352300Z

Updated to: "Locates a program in (exec-paths) similar to the which Unix command. The :all option will return a vec of results instead of a single result."

borkdude 2021-02-06T16:48:17.352500Z

Should it be 'similarly' ?

lread 2021-02-06T16:49:02.352700Z

Maybe, but either reads fine to me.

borkdude 2021-02-06T16:51:33.352900Z

codox updated

lread 2021-02-06T16:53:21.353100Z

I like how you handle delete for files. A clearer design, in my opinion than <http://clojure.java.io|clojure.java.io> (delete-file f &amp; [silently]).

borkdude 2021-02-06T16:54:50.353400Z

the impl is basically dictated by java.nio

borkdude 2021-02-06T16:56:08.353600Z

Alternative implementation of which using already existing ingredients of fs:

(str (first (filter fs/executable? (fs/list-dirs (filter fs/exists? (fs/exec-paths)) "java"))))

"/Users/borkdude/.jenv/versions/11.0/bin/java"

borkdude 2021-02-06T16:56:56.354Z

"java" can be a glob like "*.{.clj, cljs}" too

lread 2021-02-06T16:58:54.354200Z

I like that delete-tree has an option to not follow symlinks. Nice for safer deletes. Should following links be off by default and with an option to turn it :follow-links on? Or are you just matching a Java API?

borkdude 2021-02-06T16:59:05.354400Z

Just matching the API

lread 2021-02-06T16:59:10.354600Z

ok

borkdude 2021-02-06T16:59:23.354800Z

I tried to make up as little as possible myself

lread 2021-02-06T16:59:54.355100Z

I think you put it together very nicely, though!

borkdude 2021-02-06T17:00:20.355400Z

thanks

borkdude 2021-02-06T17:00:32.355600Z

and thanks for the review too!

lread 2021-02-06T17:01:35.355800Z

my pleasure!

borkdude 2021-02-06T17:02:24.356Z

The hardest function by far was glob which is not a standard function in the Java API

lread 2021-02-06T17:02:34.356200Z

Docstring for delete might need an update?

borkdude 2021-02-06T17:02:47.356400Z

So I first made a function walk-file-tree which basically mirrors Files/walkFileTree and built it on top of that

borkdude 2021-02-06T17:03:15.356600Z

why update, what's wrong with it?

lread 2021-02-06T17:03:49.356800Z

(defn delete
  "Deletes f via Path#delete. Returns nil if the delete was succesful,
  throws otherwise."
  [dir]
  (Files/delete (as-path dir)))

lread 2021-02-06T17:04:09.357Z

Doesn’t delete via Path#delete?

lread 2021-02-06T17:04:26.357200Z

Oh and small typo in succesful

lread 2021-02-06T17:05:20.357400Z

Same idea for delete-if-exists docstring.

borkdude 2021-02-06T17:06:01.357600Z

Both updated now, thanks

lread 2021-02-06T17:08:29.357800Z

If I use :nofolow-links on delete-tree and there is a symlink found will it just skip the symlink or throw?

borkdude 2021-02-06T17:09:16.358Z

It will just delete the symlink itself, I think, but that would be good to check ...

lread 2021-02-06T17:09:56.358200Z

Oh right… that makes sense.

borkdude 2021-02-06T17:10:42.358400Z

I think I must make a unit test for this...

lread 2021-02-06T17:10:45.358600Z

Also, do you think a delete-tree-if-exists would be useful? To match delete-if-exists?

lread 2021-02-06T17:11:15.358800Z

Also no mention of what delete-tree returns in docstring.

borkdude 2021-02-06T17:11:47.359Z

It always returns nil right now

lread 2021-02-06T17:12:03.359200Z

And will throw if it can’t delete something, I assume.

borkdude 2021-02-06T17:12:18.359400Z

The root is deleted using fs/delete

borkdude 2021-02-06T17:12:24.359600Z

and this will throw indeed

lread 2021-02-06T17:12:36.359800Z

Makes sense

lread 2021-02-06T17:14:11.360Z

I’m probably getting too nit picky but (exec-paths _)`` shows in API docs, which is minor but a bit weird.

borkdude 2021-02-06T17:18:19.360200Z

hmm I think we can make delete-tree take follow-links instead of no-follow-links

borkdude 2021-02-06T17:18:28.360400Z

since there is no Files thing for this

lread 2021-02-06T17:19:07.360600Z

if it matches existing APIs enough, it would be a safer default choice.

borkdude 2021-02-06T17:19:51.360800Z

I think we can base it on walk-file-tree

borkdude 2021-02-06T17:19:55.361Z

instead of what we do now

borkdude 2021-02-06T17:19:58.361200Z

and that has follow-links

lread 2021-02-06T17:21:34.361500Z

Docstring, I think is incorrect here:

(defn file-time-&gt;millis
  "Converts a java.nio.file.attribute.FileTime to a java.time.Instant."
  [^FileTime ft]
  (.toMillis ft))

lread 2021-02-06T17:22:06.361700Z

Probably belongs on?

(defn file-time-&gt;instant [^FileTime ft]
  (.toInstant ft))

borkdude 2021-02-06T17:25:44.361900Z

Please continue. I'll first fix the delete-tree and then go through the others

lread 2021-02-06T17:25:45.362100Z

That’s all I got for now! Looking awesome!

borkdude 2021-02-06T17:25:55.362300Z

afk for dinner now, back in a bit

lread 2021-02-06T17:26:04.362500Z

lunchtime for me!

Max 2021-02-06T18:13:40.363500Z

Is babashka.fs usable in normal java clojure? It seems like it’d be useful in the general case

borkdude 2021-02-06T18:15:45.363800Z

@max.r.rothman yes

lread 2021-02-06T19:05:12.364200Z

Oh… one other small thought… should side-effecting fns have a !?

borkdude 2021-02-06T19:38:43.364400Z

I think in the context of immutable data structures that might make sense, but in the context of files, the function names are pretty explanatory about their side effects? E.g. the functions in http://clojure.java.io don't have !s either

👍 1
borkdude 2021-02-06T19:42:52.364600Z

Strangely enough I can't make walk-file-tree follow links...

borkdude 2021-02-06T19:43:04.364800Z

o wait, it's just my test

borkdude 2021-02-06T19:45:26.365Z

so the file gets visited, but delete needs to both delete symlink and the target dir.... hmm

borkdude 2021-02-06T19:57:27.365200Z

@lee I hope this makes sense:

(testing "symlinks"
    (let [tmp-dir1 (temp-dir)
          tmp-dir2 (temp-dir)
          tmp-file (fs/path tmp-dir2 "foo")
          _ (fs/create-file tmp-file)
          link-path (fs/path tmp-dir1 "link-to-tmp-dir2")
          link (fs/create-sym-link link-path tmp-dir2)]
      (is (fs/exists? tmp-file))
      (fs/delete-tree tmp-dir1 {:follow-links true})
      (is (not (fs/exists? link)))
      (is (not (fs/exists? tmp-file)))
      (is (not (fs/exists? tmp-dir2)))
      (is (not (fs/exists? tmp-dir1))))
    (let [tmp-dir1 (temp-dir)
          tmp-dir2 (temp-dir)
          tmp-file (fs/path tmp-dir2 "foo")
          _ (fs/create-file tmp-file)
          link-path (fs/path tmp-dir1 "link-to-tmp-dir2")
          link (fs/create-sym-link link-path tmp-dir2)]
      (is (fs/exists? tmp-file))
      (fs/delete-tree tmp-dir1 {:follow-links false})
      (is (not (fs/exists? link)))
      (is (fs/exists? tmp-file))
      (is (fs/exists? tmp-dir2))
      (is (not (fs/exists? tmp-dir1)))))

borkdude 2021-02-06T19:57:53.365400Z

The first case tests that the link + the target gets deleted

👍 1
borkdude 2021-02-06T19:58:02.365600Z

The second case tests that only the links get deleted

👍 1
borkdude 2021-02-06T19:58:53.365900Z

We could do this similarly for delete: if you provide :follow-links there as well, both the link and the target gets deleted....

👍 1
borkdude 2021-02-06T19:59:05.366100Z

but one other possibility could be that only the target gets deleted... :/

👎 1
borkdude 2021-02-06T20:00:34.366300Z

but if you delete the target, then what is a symlink without a target. that seems not right ;)

👍 1
borkdude 2021-02-06T20:00:52.366500Z

so yeah, let's include that option there too

👍 1
lread 2021-02-06T20:03:07.366700Z

yeah, agreed, deleting target but not link is wrong

borkdude 2021-02-06T20:06:56.367600Z

I wonder why Files doesn't provide this behavior of follow-link for delete. We might just support it in delete-tree. If people want to delete both, they can use that, while delete follows the normal java behavior

borkdude 2021-02-06T20:09:34.367800Z

ah, in delete I think you will have a conflicting behavior when you say: delete this link, but also follow the link. If the link then is a dir, you are not delete-tree-ing, so this can't remove the entire dir

borkdude 2021-02-06T20:10:12.368100Z

so what's now on master probably makes most sense. also updated codox with your other suggestions

lread 2021-02-06T20:10:45.368300Z

Good observation, yeah. So delete will delete a symlink but never follow it, right?

borkdude 2021-02-06T20:11:05.368500Z

yes

lread 2021-02-06T20:11:12.368700Z

Got it

borkdude 2021-02-06T20:11:16.368900Z

I'll also doc that