tools-deps

Discuss tools.deps.alpha, tools.build, and the clj/clojure command-line scripts! See also #depstar #clj-new
seancorfield 2021-06-26T06:23:56.371800Z

How stable is this part of the tools.deps API:

user=&gt; (require '[clojure.tools.deps.alpha.util.dir :refer [with-dir]] '[hf.depstar :refer [uberjar]] '[<http://clojure.java.io|clojure.java.io> :as io])
nil
user=&gt; (with-dir (io/file (str (System/getProperty "user.dir") "/projects/api"))
         (uberjar {:jar "../../../build/uberjars/api-2.0.0.jar" :main-class 'api.main :aot true}))
[main] INFO hf.depstar.uberjar - Compiling api.main ...
[main] INFO hf.depstar.uberjar - Building uber jar: ../../../build/uberjars/api-2.0.0.jar
nil
user=&gt; 
We're going to start replacing our build shell script stuff with a Clojure build.clj script or something similar and the shell script cd's into subproject folders to run commands and I figured out the above is equivalent... but it feels a bit like I'm relying on some internals there? I'm hoping with tools.build that there's some standard way to specify the "user directory" for running different tasks so a build script can navigate around a repository and run tasks in different places?

dominicm 2021-06-26T08:02:32.372100Z

I assume the aim is to avoid multiple jvms, so a faster build overall?

seancorfield 2021-06-26T08:03:19.372300Z

Yes, partly. And also just to get rid of bash scripts where possible šŸ™‚

seancorfield 2021-06-26T08:03:46.372500Z

Given that tools.build is going to be taking us in this sort of direction, it seems reasonable to start planning.

seancorfield 2021-06-26T08:04:58.372700Z

I just posted in #polylith a function I created that builds an uberjar for every projects entry in a Polylith workspace, which is where we're headed with this.

seancorfield 2021-06-26T08:05:26.372900Z

(but it's applicable anywhere that you have multiple subprojects that need a task applying to them in sequence)

seancorfield 2021-06-26T08:07:23.373100Z

My next task is to figure out whether classpath-isolated subproject test runners are achievable šŸ™‚

seancorfield 2021-06-26T08:08:28.373300Z

(given that Polylith already has a version of this in its command-line tool, the answer is obviously "yes" but I just need to figure out how much work that is...)

dominicm 2021-06-26T08:29:29.373500Z

Only polylith projects have that issue of needing classpath isolation because of how it swaps out namespaces right? Otherwise you could just load them all into a single JVM?

dominicm 2021-06-26T08:43:30.373700Z

(If clojure.test could be parameterized, you wouldn't even need to do that, you could just use a protocol!)

dominicm 2021-06-26T08:48:36.373900Z

@seancorfield for cp isolation, you might be interested in https://github.com/SevereOverfl0w/clj-embed which was forked to fix a few bugs I found in upstream. It's behind the API times, but it's probably faster than shelling out.

seancorfield 2021-06-26T16:54:30.374200Z

Thanks @dominicm and, no, re: Polylith swapping namespaces: this is more about wanting potential classpath isolation between subproject test runs. We have one particular subproject that requires a specific, old version of Jackson (before they changed the null merging behavior in 2.9.0) but the other subprojects can run with "any" version of Jackson. I haven't decided yet how I want to approach this part. Currently, we run a separate clojure process for each subproject when running tests but that's (obviously) slower than just loading everything and running all the tests in a single JVM...

seancorfield 2021-06-26T16:56:37.374400Z

Looks like clj-embed does "more" in terms of isolation than I'd really need but I suspect it would get me closer. Thanks.

athomasoriginal 2021-06-26T18:13:09.379700Z

Hi all. Iā€™m just getting around to tackling the https://clojure.org/releases/tools#v1.10.2.790 https://clojure.org/releases/tools#v1.10.2.790 warning now for my monorepo. I have read through Seanā€™s posts and conversations in this channel about monorepo setup with this new constraint. I have outlined my https://gist.github.com/athomasoriginal/47913c6b946fc7416c3d839be101e600 and wanted to see if this aligns to the ā€œrecommended approachā€? > I use ā€œrecommendā€ lightly. My goal is to be sure the approach is not going against the grain too much or setting myself up for future pain. Thanks!!

seancorfield 2021-06-26T18:15:40.379900Z

Sounds about right but I'll elaborate a bit on a couple of our monodeps setup...

alexmiller 2021-06-26T18:17:09.380100Z

I assume you mean :extra-paths in there, not :external-paths

šŸ‘ 1
seancorfield 2021-06-26T18:17:18.380300Z

Our "workspace-level" deps.edn has an :everything alias which has :extra-deps listing each subproject as a :local/root dep and :extra-paths listing each subproject's test folder -- we use this alias for starting a global REPL to work with the entire codebase. I think that's an important affordance.

seancorfield 2021-06-26T18:18:03.380500Z

That top-level deps.edn file is where we declare any/all tooling -- but we don't have cljs subprojects so I don't know how that would change the equation (if at all).

alexmiller 2021-06-26T18:22:47.380700Z

why are app and app-server separate? app is what you build and app-server is what you deploy?

alexmiller 2021-06-26T18:25:03.380900Z

one thing coming is the ability for projects to publish that they must be "prepared" before use and for consumers of that project to ask to do that preparation

alexmiller 2021-06-26T18:26:01.381100Z

I'm trying to understand your bigger use case here and whether that could be used for advantage (having app-server consume app as a local dep, which has a prep step)

athomasoriginal 2021-06-26T18:31:34.381400Z

> I assume you mean :extra-paths in there, not :external-paths Yes. My bad šŸ˜ž

athomasoriginal 2021-06-26T18:38:42.381600Z

> why are app and app-server separate? ā€¢ app is the ClojureScript application (client) ā€¢ app-server is the web-server serving static assets and a REST API My preference is separation because iā€™ve found it makes it easier to understand the backend/front end divide and helps avoid coupling tooling. Before I go further though, does that clarify? Iā€™m not sure I understand your comment: ā€œapp is what you build and app-server is what you deployā€ :thinking_face:

seancorfield 2021-06-26T18:40:19.381900Z

I suspect app-client and app-server might have been less confusing names šŸ™‚

athomasoriginal 2021-06-26T18:40:49.382200Z

I think you are correct šŸ˜‰

alexmiller 2021-06-26T18:41:14.382400Z

ah ok, that makes sense

šŸ™ 1
athomasoriginal 2021-06-26T18:42:06.382700Z

I think I understand your build v. deploy question now though šŸ™‚

athomasoriginal 2021-06-26T18:44:02.382900Z

> Iā€™m trying to understand your bigger use case here and whether that could be used for advantage Does the use case make more sense now? It might be outside the scope of the feature youā€™re considering though?

alexmiller 2021-06-26T18:51:20.383200Z

not sure, might still be a good match?

alexmiller 2021-06-26T18:53:30.383400Z

app would have some build process and publish in its deps.edn how to run it. use :path including the output of that process. app-server would depend on app

athomasoriginal 2021-06-26T18:53:59.383600Z

Yeah, thatā€™s it.

alexmiller 2021-06-26T18:54:00.383800Z

there may be some tooling that will need to catch up to all that though

athomasoriginal 2021-06-26T19:11:02.384Z

> That top-levelĀ `deps.edn`Ā file is where we declare any/all tooling @seancorfield I am re-reading your post again, but in my scenario my driver for relying on paths external to the app-server project was only so I could have access to app-client s resources and target dirs because I feel itā€™s awkward to have those specified in the app-server project while your developing the appā€™s front end. So I believe a difference is that I donā€™t actually need to work with clj code, I just did it to serve static assets in development.