Clojurians Log v2

Just double checking - there is no way to get the deps of a subproject specified via :local/root including deps from an alias, correct? i.e. I cannot specify an alias together with :local/root in some way?

Nope.

Dependencies are opaque in that respect.

Although... I guess you could read the basis (in the prerelease CLI) and figure out the actual :local/root path and see if it has deps.edn and then use t.d.a to read that and then inspect its aliases... 👀

But a :local/root could point to a JAR, or a directory with pom.xml. Or project.clj 🙂

Right.

This is just specifying deps in a deps.edn file, so I can’t do anything funky or programmatic.

What is the rationale for deprecating unqualified dependency symbols in :deps? (couldn’t see one in the release notes)

@hugod It's mentioned in this post https://insideclojure.org/2020/07/28/clj-exec/ near the end.

(IOW it was deprecated in an earlier prerelease)

Thanks - I missed that release 🙂

I spent the first week of August, scurrying around, fixing all our unqualified lib names at work, and all my open source projects! 🙂

Is there any way to substitute values into a deps file that I’m not aware of, e.g. from env vars or properties? In my case I would like to be able to substitute a global version in several deps files. I’m aware of default-deps, but I’d need to be able to substitute it in a :local/root path.

Nope. No substitutions.

(I'm just a bundle of "nope" today, aren't I?)

Haha

No worries, that was what I expected, just makes my life a little more tedious 🙂

Fwiw, having played with the pre-release, I still think the following CLI switches would be simpler, as they separate the selection of aliases from what to execute, and would simplify the doc. A given set of aliases can be used for main, exec or repl, so it makes it easier to compose a command line from other tools if the passing of options is distinct from the selection of what to invoke.

clj     [clj-opt*] [-A:aliases] [init-opt*]
clojure [clj-opt*] [-A:aliases] -x [a/fn] [kpath v]*
clojure [clj-opt*] [-A:aliases] [init-opt*] [main-opt] [arg*]
clojure [clj-opt*] -P [other exec opts]

clojure [clj-opt*] [-A:aliases] -M [init-opt*] [main-opt] [arg*]

Sorry, not doing that

That’s clear 🙂

Also, I think there's more usage of -R out there than you suspect @hugod

I’m sure I don’t have a good perspective on that, but don’t think it really changes my view that just using -A would be conceptually simpler.

Given the cli changes seem fixed, I’ll just go and update my nascent build tool invoker project to cope with multiple versions of the cli.

I still think that a) given the expansion of -M to include more than just :main-opts b) the overlap between -A and -M (both run :main-opts now) c) the current uses of -R (doesn't run :main-opts but does "most" of what -A does -- based on resolve args being the most common; it would make more sense for -R to be kept and expanded as the "REPL" usage (and quietly undocumenting -A). So we'd have -R (REPL), -M (main), -X (exec) and they all three respect all arg types in deps.edn and there would be no overlap on :main-opts or :exec-fn.

I do find it unfortunate that the deps.edn isn’t authoritative. By this I mean that whatever you specify in deps.edn you still have to know whether to invoke it with -M or -X.

See https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L15-L18 -- you can support both 🙂

That works with -A today and either -M or -X tomorrow!

I don’t think that is quite what I meant. As a deps.edn writer I would like to provide an abstract, say test, alias for doing something, and have the user be able to invoke that independent of how I chose to implement that alias. Writing it both ways seems like duplication.

well the arg structures are different, not sure how that could work

Hi, I have been working on a https://github.com/JeremS/mbt based on tools.deps for some time. It has a mix of functionality inspired from several other projects and it's at a stage where I use it to version, build and deploy my projects. Maybe someone other than me might find it useful! Cheers

@alexmiller It looks like clojure -X:deps mvn-pom now adds the following dependencies to all pom.xml files:

<dependency>
      <groupId>org.clojure</groupId>
      <artifactId>tools.deps.alpha</artifactId>
      <version>0.9.782</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-nop</artifactId>
      <version>1.7.25</version>
    </dependency>

That's going to cause a lot of projects to accidentally end up depending on tools.deps.alpha (and the slf4j-nop dependency is likely to be unwelcome in many cases I suspect).

These deps are in a freshly-generated pom.xml file -- and they are also added to any existing pom.xml, potentially overwriting the dependency for any projects that use a different version of t.d.a

ah, interesting

thx, will fix

Exactly, as a tool writer you need to specifically support -M style invocation and/or -X style invocation as two separate functions in your code: -main taking & args which are strings and some-exec-fn taking a single hash map as its argument.

How will you fix it? For example, my clj-new project does depend on t.d.a (and has a dependency on slf4j-nop because of that) -- but I know that's an outlier -- so you'd need to also not remove any existing dependency on those?

I'll fix it by calculating the basis, not using the injected basis

basically what it did before

Won't that break anyone currently using clojure -A:some:aliases -Spom ?

it doesn't have to - the injected basis includes those argmaps

the calculated basis can use the project deps.edn + the injected basis argmaps

another option would be to take those additional aliases as args to the -X pom function

that would be an all new thing though

very similar thing exists in the datomic ion-dev tool which I was just helping with a bit :)

I am planning to reinstate the -Spom as well so looking at that right now too

I had some thoughts on the -Stree thing you ran into though - I think there are several better ways to detect changing deps tree rather than scraping the -Stree stdout. not sure what your requirements on that stuff are but might be useful to talk about

from a clj pov, we're writing edn files (.libs in particular) to the cache dir (can see location with -Sdescribe), so you could be slurping and diffing Clojure maps rather than comparing strings

and from a tools.deps pov, you could probably make something that did this via the api instead, but not sure if that's worth the effort

I really consider -Stree / -X:deps tree to be human consumer tools and ones likely to hopefully improve over time, changing format. I don't consider the current particular output to be something a script should rely on.

Re: basis -- so -A and -X are distinguishable via the basis? Or am I misunderstanding?

If it's just going to be identical to -X:deps mvn-pom I wouldn't have thought it was worth the effort?

oh, I see what you're asking. well let me think about it more

True, but a) how can you easily get the correct .libs file from a given invocation without scraping the output and b) that gives no sense of depth of transitivity, which is specifically what I rely on in -X:deps tree

you were the one here arguing it was breaking people last week... :)

"Because reasons"... we go one level down in the full tree (so we consider top-level deps and the first level of transitive deps, but we don't care about lower levels).

We've found that we can safely ignore the 2nd-order transitive deps and if we have a problematic 1st-order transitive dep, we just promote it to the top-level (and then its transitive deps are promoted to 1st-order transitive deps and we start to track them separately).

well the transitivity stuff is all broken in the -Stree output

but maybe you're just not encountering the cases where it's bad (mostly with exclusions)

Hahaha... yeah, but you didn't seem very sympathetic to that argument, and I'm not sure having duplicate ways to do the exact same thing is better 🙂

We don't use exclusions, for the most part.

This is our only exclusion:

com.walmartlabs/lacinia {:mvn/version "0.35.0"
                           :exclusions [clojure-future-spec/clojure-future-spec]}

the main thing is, the code never actually builds the real expansion tree as a tree. -Stree is kind of faking from the libs map

We decided that managing conflicts via :exclusions was a bit of a nightmare so we control conflicts by promoting transitive deps to the top level.

Right, but I'd have to write that "faking" code myself, whereas -X:deps tree already does a "good enough" job.

well, it exists as a function you could call (the one -Stree calls) :)

but that stuff is due for some significant changes soon as I fix up some of this stuff

Besides, I've had this shell script running with -Stree for ages and don't really want to rewrite it as a Clojure program right now. Maybe when I'm bored and want to go shave some more yaks...

ok, just letting you know that I'm not considering stdout changes from -Stree (or its replacement) to be breaking :)

but I am planning to add some more api fns to get tree-like deps data