Clojurians Log v2

Hi, I’m trying to build an uberjar for a clojure 1.8 project (yes, I know…) However the POM file that was automatically-created has a dependency of clojure 1.10.3 (which is the version of the clojure CLI I’m using)… Is this a known issue or am I doing something wrong?

clojure -X:uberjar :pom-file '"./my-app/pom.xml"' :sync-pom true :group-id xxx :artifact-id my-app :version '"1.0.2"' :aliases '[:defaults :my-app]' :repro false :jar my-app.jar :main-class my-app.main :aot true

<?xml version="1.0" encoding="UTF-8"?>
&lt;project xmlns="<http://maven.apache.org/POM/4.0.0>" xmlns:xsi="<http://www.w3.org/2001/XMLSchema-instance>" xsi:schemaLocation="<http://maven.apache.org/POM/4.0.0> <http://maven.apache.org/xsd/maven-4.0.0.xsd>"&gt;
  &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
  &lt;packaging&gt;jar&lt;/packaging&gt;
  &lt;groupId&gt;xxx&lt;/groupId&gt;
  &lt;artifactId&gt;my-app&lt;/artifactId&gt;
  &lt;version&gt;1.0.2&lt;/version&gt;
  &lt;name&gt;my-app&lt;/name&gt;
  &lt;dependencies&gt;
    &lt;dependency&gt;
      &lt;groupId&gt;org.clojure&lt;/groupId&gt;
      &lt;artifactId&gt;clojure&lt;/artifactId&gt;
      &lt;version&gt;1.10.3&lt;/version&gt;
    &lt;/dependency&gt;
  &lt;/dependencies&gt;
  &lt;build&gt;
    &lt;sourceDirectory&gt;src&lt;/sourceDirectory&gt;
  &lt;/build&gt;
  &lt;repositories&gt;
    &lt;repository&gt;
      &lt;id&gt;clojars&lt;/id&gt;
      &lt;url&gt;<https://repo.clojars.org/>&lt;/url&gt;
    &lt;/repository&gt;
    &lt;repository&gt;
      &lt;id&gt;my-private-maven&lt;/id&gt;
      &lt;url&gt;<s3://my-private-maven.com>&lt;/url&gt;
    &lt;/repository&gt;
  &lt;/repositories&gt;
&lt;/project&gt;

Note the uberjar itself contains clojure 1.8 so it only looks like a “cosmetic” issue.

Is there a way to get depstar to not ignore .cljs files? I'd like to package this jar once and have it available both for JVM and JS Clojure variants.

Oh, it seems like it's picking it up now

I must've just not retried after I moved the cljs directory to the main classpath

It seems pom.xml is identical if I just run

clojure -Spom

clojure -Spom -A:...

For clojure -Spom, you would need to include your aliases with -A before -Spom. I thought depstar used the same basis for calling t.d.a’s sync-pom as for building the JAR… can you create an issue on GH for depstar for me to take a look at @golan1w?

I am trying to get the exclude parameter working correctly for my depstar target. As an example, I have the directories backend, scripts, and user at the root of my project. They are all being packaged into the uberjar and I don’t want that. I’ve tried the following to exclude them:

:exclude       ["scripts/*" "backend/*" "user/*"]

Those aren’t valid regexes — they’re shell glob patterns.

"scripts/.*" would be a regex.

Well, I guess they are valid regexes but they’re not what you intended: "scripts/*" matches scripts followed by zero or more / characters.

So scripts, scripts/, scripts//, scripts/// etc. @markbastian

Happy to make the documentation clearer on that, if you have a suggestion?

Thanks! Yeah, I read it as regexes but was thrown off by the shell glob patterns as to if those worked or not. I’m still used to the shell patterns. As far as docs go, I think an example with directory exclusion could be good.

Is there a way to specify includes vs. excludes or set a target directory that will be uberjarred? I read all the options and didn’t see that. Having experience with maven uberjar or shade I was hoping to see a target or output directory that is the final thing that is jarred up.

I’m not quite sure what you’re asking @markbastian You can specify a full path for the :jar option which is where the JAR goes, e.g., :jar '"/path/to/the.jar"'

Not sure what you mean by “target directory”?

I mean setting the explicit target for what gets included in the jar. I was hoping to be able to only have what is in paths/extra-paths but I think that is just what is used for aot. What actually gets included in the jar appears to be everything that is not explicitly excluded besides the default excludes. It would be nice to set default includes or only use the specified paths.

AOT is transitive.

Are you building a JAR or an uberjar?

Building an uberjar, but does it make sense that every directory in the project is included in the uberjar, not just source directories? Just want to make sure I’m not doing something wrong.

I’m moving from lein (`lein uberjar`) to dpes & depstar, and compared the resulting jars and they are identical as far as I can see. I see in the uberjar only the folders that were in the :path, in my case src and resources.

@seancorfield in the README there is:

Run depstar with :sync-pom true :version '"x.y.z"' to update the <version> and SCM <tag> in the pom.xml file.

clojure -X:depstar :pom-file '"./app/pom.xml"' :sync-pom true :version '"1.1.1"'
No function found on command line or in :exec-fn

I’m wondering if there’s something wrong with my path configuration. I really do just want what’s in my :path entries from my project in the uberjar along with transitive dependencies.

you can add clojure -Spath to analyze it

Good call, gonna try that.

To be clear, is the “normal” behavior of the uberjar target to include only the contents of the :path entries or is it expected to include all contents of the project except for those specifically excluded?

Your :depstar alias does not have :exec-fn.

I ran into the same issue. You have to also create a jar when you want to update the pom

;; build an uberjar (application) with AOT compilation by default:
  :uberjar {:replace-deps {com.github.seancorfield/depstar {:mvn/version "2.0.211"}}
            :exec-fn hf.depstar/uberjar
            :exec-args {:aot true}}
  ;; build a jar (library):
  :jar {:replace-deps {com.github.seancorfield/depstar {:mvn/version "2.0.211"}}
        :exec-fn hf.depstar/jar
        :exec-args {}}
  ;; generic depstar alias, use with jar or uberjar function name:
  :depstar {:replace-deps {com.github.seancorfield/depstar {:mvn/version "2.0.211"}}
            :ns-default hf.depstar
            :exec-args {}}

That’s not a depstar issue — that’s a CLI usage issue.

An uberjar will include everything that is needed to run as a standalone application.

Is there a way to update the pom without creating a jar? I've been using:

clojure -X:depstar jar :jar reveal-exception.jar :sync-pom true :version '"1.0.2"'

So, yes, it’s “normal” that an uberjar includes “everything”

No, depstar only updates the pom.xml if you are building a JAR or uberjar.

ok that’s what I thought, I was just wondering because of this sentence in the README.

I also got stuck on the exact same step with the same error. Would you accept a pull request with an example usage?

“every directory in the project is included in the uberjar, not just source directories” — whatever is on your :paths and :extra-paths per the aliases you provide, plus all dependencies from :deps and :extra-deps (again, per the aliases you provide) @markbastian

I’m not understanding what you think needs to be changed?

It says “generic depstar alias, use with jar or uberjar function name”

And then it gives examples for each:

clojure -X:uberjar :jar MyProject.jar
# or:
clojure -X:depstar uberjar :jar MyProject.jar

clojure -X:jar :jar MyLib.jar
# or:
clojure -X:depstar jar :jar MyLib.jar

and then later on:

clojure -X:uberjar :jar MyProject.jar :aliases '[:webassets]'
# or:
clojure -X:depstar uberjar :jar MyProject.jar :aliases '[:webassets]'

clojure -X:uberjar :classpath "$(clojure -Spath -A:webassets)" :jar MyProject.jar
# or:
clojure -X:depstar uberjar :classpath "$(clojure -Spath -A:webassets)" :jar MyProject.jar

When I first was trying the clojure cli in place of lein, I was trying to follow the "My Deployment Process" section and got stuck on: > Run depstar with :sync-pom true :version '"x.y.z"' to update the <version> and SCM <tag> in the pom.xml file. I know that's a simple thing if you're familiar with the clojure cli, but I didn't know how to turn that sentence into a command at the time

In the pom.xml section it has this example:

clojure -X:uberjar :sync-pom true \
        :group-id myname :artifact-id myproject \
        :version '"1.2.3"' :jar MyProject.jar

My suggestion is to add an example to that step in the "My Deployment Process section": Run depstar with :sync-pom true :version '"x.y.z"' to update the <version> and SCM <tag> in the pom.xml file.

clojure -X:depstar jar :jar MyProject.jar :sync-pom true :version '"1.2.3"'

Oh, so just that one sentence in My Deployment Process is what is confusing you?

I know and have been using it. Because of this sentence, I thought there’s a way to set the version in the pom without building the JAR.

OK, I’ll edit that.

thanks!

Updated.

Hmm, i just added a non-pathed folder with some garbage data in a fresh deps project and uberjarred it and the folder didn’t look like it was included. I think my project that is including all the extra directories (which has a pretty hairy deps.edn file) has some configuration that is user error on my part. Gonna try simplifying it to a reduced case that works.

Ok, I have a reduced case that I can present. Using this deps.edn file I can successfully create a jar of about 5MB with the uberjar alias. This is a working deps.edn file for my case. It produces an uberjar with only src + deps with aot compilation.

{:paths     ["src/clj" "src/cljc"]

 :mvn/repos {"central"    {:url "<https://repo1.maven.org/maven2/>"}
             "clojars"    {:url "<https://clojars.org/repo>"}
             "confluent"  {:url "<http://packages.confluent.io/maven/>"}
             "<http://jitpack.io|jitpack.io>" {:url "<https://jitpack.io>"}}

 :deps      {com.amazonaws/aws-lambda-java-core {:mvn/version "1.2.1"}
             org.clojure/data.json              {:mvn/version "2.0.2"}
             hashp/hashp                        {:mvn/version "0.2.0"}}

 :aliases   {:uberjar {:extra-deps {com.github.seancorfield/depstar {:mvn/version "2.0.211"}}
                       :exec-fn    hf.depstar/uberjar
                       :exec-args  {:jar        "small.jar"
                                    :main-class my.lambda.foo
                                    :aot        true}}}}

;ls -al shows small.jar to be ~5.26MB

:clj-lambda {:extra-deps    {com.github.seancorfield/depstar {:mvn/version "2.0.193"}}
             :exec-fn       hf.depstar/uberjar
             :replace-paths ["src/clj" "src/cljc"]
             :replace-deps  {hashp/hashp                        {:mvn/version "0.2.0"}
                             com.amazonaws/aws-lambda-java-core {:mvn/version "1.2.1"}
                             org.clojure/data.json              {:mvn/version "2.0.2"}}
             :exec-args     {:jar        "big.jar"
                             :main-class my.lambda.foo
                             :aot        true}}

:lambda-backend
            {:extra-paths  []
             :replace-deps {hashp/hashp                        {:mvn/version "0.2.0"}
                            com.amazonaws/aws-lambda-java-core {:mvn/version "1.2.1"}
                            org.clojure/data.json              {:mvn/version "2.0.2"}}}

:clj-lambda {:extra-deps    {com.github.seancorfield/depstar {:mvn/version "2.0.193"}}
             :exec-fn       hf.depstar/uberjar
             :replace-paths ["src/clj" "src/cljc"]
             :exec-args     {:jar        "big.jar"
                             :main-class my.lambda.foo
                             :aliases    [:lambda-backend]
                             :aot        true}}

;ls -al shows big.jar to be ~150MB

Your suspicion is correct.

:replace-deps (and :replace-paths) are used to set up an environment for a tool to run (like depstar) that is not “polluted” by your project’s environment. It’s why the depstar README says to use :replace-deps to specify depstar itself.

When you run the tool, it will compute the project basis — reading the root deps.edn file, the user deps.edn file (only if you tell depstar :repro false), and the project deps.edn file. If you need depstar to use aliases for computing what should go into your JAR file, you pass :aliases [:your :alias :list] to depstar and it will use those aliases when computing the project basis.

You pretty much never want :replace-* for anything except a tool that you are running.

Think of it like this: * Running the tool requires a classpath that has the tool itself and very little else * At runtime, the tool itself will compute a different classpath that is used for whatever the tool is trying to do with your project (run tests, build a JAR, analyze dependencies, etc)

That’s why depstar says to use

;; build an uberjar (application) with AOT compilation by default:
  :uberjar {:replace-deps {com.github.seancorfield/depstar {:mvn/version "2.0.211"}}
            :exec-fn hf.depstar/uberjar
            :exec-args {:aot true}}

And then it talks about how depstar builds the classpath (for the JAR building) and says:

If you need to adjust the computed classpath, based on aliases, you can supply a vector of aliases to the :aliases exec argument of depstar. This classpath is used for both the AOT compilation process and for the JAR building.

For example, you can add web assets into an uberjar by including an alias in your project deps.edn:

{:paths ["src"]
 :aliases {:webassets {:extra-paths ["public-html"]}}}
Then invoke depstar with the chosen aliases:

clojure -X:uberjar :jar MyProject.jar :aliases '[:webassets]'
# or:
clojure -X:depstar uberjar :jar MyProject.jar :aliases '[:webassets]'

(and this is how quite a few deps.edn-based tools work: they need a minimal classpath to run the tool itself and then they compute a more comprehensive classpath for the task they perform)

Ok, that makes a lot of sense. The alias entries are for the tool and the and the execution arguments are for the project/target that the tool is processing.

And with that knowledge I can now do this and it works:

:lambda-backend
            {:replace-paths ["src/clj" "src/cljc"]
             :replace-deps  {hashp/hashp                        {:mvn/version "0.2.0"}
                             com.amazonaws/aws-lambda-java-core {:mvn/version "1.2.1"}
                             org.clojure/data.json              {:mvn/version "2.0.2"}}}

:clj-lambda {:extra-deps {com.github.seancorfield/depstar {:mvn/version "2.0.193"}}
             :exec-fn    hf.depstar/uberjar
             :exec-args  {:jar        "big.jar" ;Ends up only being about 5MB.
                          :main-class my.lambda.foo
                          :aliases    [:lambda-backend]
                          :aot        true}}

You should still use :replace-deps for depstar itself.

But, yes, if you want an uberjar that contains “just” the `:lambda-backend and whatever ends up transitively compiled into that, you are doing it correctly.

Hi there, I am trying to build an uberjar but I am running into the following:

$ clojure -X:uberjar :jar "~/tmp/target/setup.jar"
Unreadable arg: "~/tmp/target/setup.jar"

You are not quoting the EDN string correctly. Also, you cannot use the ~ expansion in a string.

clojure -X:uberjar :jar '"/tmp/target/setup.jar"'

From the README:

clojure -X:uberjar :jar '"/tmp/MyTempProject.jar"'

uhm..subtle, trying

That’s just how -X works — nothing specific to depstar

indeed, thanks that worked

See https://clojure.org/reference/deps_and_cli#quoting