Clojurians Log v2

Is there any way with deps to manage source artifacts for non-Clojure dependencies?

Or rather, how could I specify that I would like them to be downloaded, and how would they be distinguishable from binary jars in the output?

This seems to be the answer:

{:deps {junit/junit {:mvn/version "4.13"}
        junit/junit$sources {:mvn/version "4.13"}}}

They do end up on the classpath, which is unfortunate. I guess they could go in an alias.

When using :local/root, the doc says: > If the jar has been packaged with a pom.xml file, the pom will be read and used to find transitive deps Is there a way to prevent this from happening? I would just like it to use the jar I give it and not look for transitive deps.

Looking at the code, it seems like not. My use case here is trying to use deps to manage the dependencies for Cursive. I need to depend on the IntelliJ framework, so I have a script which downloads and unpacks that, and I’m trying to create a deps.edn which refers to the jars within it. However a bunch of those are third party jars which include poms.

you could use :local/root to a directory which has a deps.edn that includes the jar on the :paths

Oh, I can just put jar paths directly in :paths? That’s even better.

kind of a hack, but it works

I guess that makes total sense, but never occurred to me.

:paths are just things that get added to the classpath

on the source stuff, tools.deps is about making classpaths so that's what you're going to get

Right. It’s actually much better for this use case since I don’t need to create a bunch of bogus dep symbols.

Ok. Using $sources works fine, so I’ll support that in Cursive, document how to do it and suggest putting them in an alias.

@alexmiller I'm confused about using -M with a project rather than a specific alias. In this example I use the hello-world example from https://clojure.org/guides/deps_and_cli#_writing_a_program A project deps.edn file was created containing the dependency for clojure.java-time and the source code from that page copied into src/hello.clj clojure -m hello runs the project and returns the time from running the -main function. However this gives a warning:

WARNING: When invoking clojure.main, use -M

:aliases
 {:run-project {:main-opts ["-m" "hello"]}} 

:aliases
 {:run-project {:exec-fn hello]}}

Thinking about this further, it does feel that the -X flag would be the most important flag in the long term, especially with :exec-arg allowing default arguments that can be over-written on the command line. -M seems to be more of a (very useful) convenience function for the specific case of running -main-opts configurations. This does remove the need for command line args and bash quoting, so that is valuable. -M is also useful for those aliases that dont work with -X , eg rebel-readline I could only get working with -M . As more maintainers adopt the new version of Clojure CLI, there is scope to configure all projects and tools to work with -X and eventually make -M redundant. Using the -M flag seems to help a little when making the change from Leiningen to CLI tools, as setting the main namespace is a familiar concept. I am sure I am over-simplifying this and probably missing out on a lot of understanding as of yet.

Would -M not continue to be useful for utilities that you expect to be invokable from the shell? How would you allow passing glob’ed files for example?

Hi, I have a couple questions about deps API. There is there a complement function to slurp-deps that also merges the 'user wide' deps.edn and the 'system wide' one ? Also the docs says that resolve-deps and make-classpath are supported. Then the question is: is make-classpath-map to be considered private api ? (and so subject to change?)

Yes, it seems the -M option is valuable when the :main-opts data is more intricate. Interesting to see what can be done with :exec-args verses :main-opts . I do like :exec-args as a means of using edn over strings. I dont believe I've have any use cases for using glob'ed files, so cant comment on that without specific examples. I assume something like *.jpg would just be a command line argument after the -X:alias-name and passed to the function.

iuuc, -X expects arguments to be key-path edn-value pairs, so clojure -X:my-alias :x "1 2 3" ends up passing {:x 1} to the exec-fn.

In general, if you want something that can be be composed with output from tools that don’t produce edn, I think -M is the easier (only?) option.

Ah, so I assume clojure -X:alias :x '["1 2 3"]' is required for :exec-fn , which fits in the concept stucture over strings. Although more specific options seem preferable. clojure -X:alias :x 1 :y 2 z: 3 . This is less ambiguous and ties into the general approach I use for Clojure arguments.

Still lots to learn, so interesting to hear these thoughts. Thanks.

I’m still trying to grasp the implication of the changes too.

You can use find-edn-maps to read and merge-edns function to merge edn maps once they’ve been read

make-classpath-map is public but some details of the make-classpath details are going to change, probably this week

(Regarding path ordering)

The whole make-classpath area is in a bit of transition

@jr0cket I think mostly what you’ve gleaned is correct. -M is for pass through to clojure.main. (And originally that’s the only thing clj did.) Calling with no exec opt is for repl. Right now that’s done with clojure.main but you should consider that an implementation detail. -X is for the new function execution style invocation.

We expect most tools are better off with -X, but totally possible to support both of course

For your own app, not sure yet how this will be taken up. Using -X means you are at least somewhat bound to using clj as your launcher (as the exec stub code is not available anywhere else (yet)). Rich seems to think that’s fine, I’m a little less sure about it. :)

My biggest issue with the -X arg passing is that passing a string path as the v requires wrapping in multiple quotes which kind of ruins - expansion and other glob and auto completion stuff. I’d say that’s not totally finalized

Ok Alex, thanks a lot! I have been foolishly experimenting with the idea of making https://github.com/JeremS/mbt based on tools deps and I ran into those questions today. If I may I have one more concerning tools.gitlibs. Right now it uses jgit v4. Do you plan to use v5?

Yeah, I’ve spent some time on it but not enough to get it over the line. It also plays into the jsch ssh stuff and look at their newer impl with Apache Mina.

Just FYI, I have built a tools.build tool that will hopefully be making its way toward release soon

Oh man... my rationale was that tools.deps begs a tools.build... A proper tools.build is very good! The jgit question was because the clj-git project uses the v5 making it, I believe, incompatible with tools deps (breakage between jgit 4 and 5) Thank you again for the info.

Thanks for confirming Alex. I am far more confident in my understanding of the changes. Apart from updating practicalli/clojure-deps-edn with either -M or -X variants instead of -A, there seems no impact on my development projects. My notes from a project developer are shared on this gist: https://gist.github.com/jr0cket/ae7dd745eb45870109ace59fe835ce80

Updating the Practicalli books with the new approach is pretty easy and can run with the 'new' and 'classic' options for a while. I've planned screencasts of using Clojure CLI tools for Clojurists Together over the next 2 months. I would rather not re-do them as high quality screencasts are time intensive. I will need to decide when to do these videos, so any thoughts on a release date for the new features would help my own planning (I appreciate any comments on dates are no guarantee). If a release is not planned until December, I'll opt for some lower quality screencasts and recreate them running up to a release.

I’m hoping to get to a new stable release in the next week or two

The guide hasn’t been updated for the new stuff yet - I’ll do that with the stable release of it

Great. That time frame wont impact me at all. Thanks If you want help with the docs, let me know (although you are the expert here)

Maybe aliases should also become fully qualified?

Then we can have a global (worldwide) alias registry =)

I wasn't trolling (at least, not consciously)

I didn't mean it that way 🙂

e.g. want to use seancorfield's aliases -A:org.seancorfield/rebl

I guess the problem with that is that aliases can change over time and effectively become a dep. Importing aliases from libs might be an idea, but maybe a bit too much.

I didn't see this go by before but order matters here - the exec opt should be last (same thing you mention later with -X) because in both cases you are passing arbitrary args to another program

I understand why that's particularly confusing in this example

fwiw, I don't see this as an issue - the documented behavior is what it does and what I expect it to do in the future

aliases can be fully qualified now

people typically don't because more typing, but that's fully supported

one thing we've done a bit of thinking about is - how could I specify a tool to use and have it automatically pulled from github without needing to declare an alias at all

yes, that was a bit was I was thinking with this

bit like lein new does maybe

Aliases can be fully qualified now, but will it ever be deprecated that they aren't - as with deps

Need some aliases procurers

probably not going to fully tackle that for a bit, but I think it would be pretty useful. you need some kind of naming convention that takes you to a gh repo, that can be a deps.edn with deps etc

I think just github support would take you pretty far, but yes all that needs to be thought about

there is also some open space I've been working in around discovery with -X

this can probably be solved in a tool which you can put in your aliases

like could -X have support for listing the functions available to call

I wasn't trolling! geez, what is it with people today ;)

how can you support automatic documentation of a -X compatible function

maybe metadata?

^:x-callable

all tbd, but with combination of mapping from alias to tool repo, and support for :ns-default (to find the default tool namespace), and ability to discover the -X functions, and help for each option....

that's an ecosystem

yes, the mapping from alias to tool repo was what I was getting at earlier

and then if only had a build system that could make use of those functions...

build system?

:troll:

:)

Metadata wouldn't work by itself, you'd have to require every namespace.

anyhow, I'm on "vacation", so back to vacating...

enjoy!

Yes. Begone!

Maybe aliases aren't the right fit for name -> github repo and some other concept/option should be invented for that.

clojure -Spath -A:test:runner -- works on stable; works on prerelease but produces a warning (use -M) clojure -Spath -M:test:runner -- works on prerelease (so switching -A to -M per the warning behaves "as expected") clojure -A:test:runner -Spath -- works on stable; works on prerelease but produces a warning (use -M) clojure -M:test:runner -Spath -- does not work on prerelease (so switching -A to -M per the warning "stops working") It's probably not going to be obvious to a user why prerelease is more sensitive to order than stable.

If -Spath implied -P, I think that would provide fewer surprises.

@jeremys If it helps you, take a look at https://github.com/seancorfield/clj-new/blob/develop/src/clj_new/helpers.clj#L60-L71 -- This uses the latest tools.deps.alpha (0.9.782) but still works with both CLI stable and CLI prerelease.

Re: find/read/merge

Yeah

I took a note of that

Sort of related to this line of thinking: clj-new supports templates on GitHub via naming conventions, so maybe have a look at the readme for the usage and see if it is close to what you are thinking @borkdude?

@seancorfield You mean:

clj -A:new -M:new <https://github.com/somename/someapp@c>...

or do you have some short form where you only mention the org + repo name, with automatically resolving the newest sha?

Currently, you need the full URL but that's really only to detect what type of template you're using. If you know it's GH, you could take just the org/repo-name and build the rest. And my dot-clojure file has an example of using tools.gitlib to figure out the HEAD of master SHA.

@borkdude See https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L174-L177

Thank you @seancorfield the last 2 lines are almost exactly what I need!