Clojurians Log v2

(OK, running 1.10.1.596 on all our servers at work now 🙂 )

how would one represent this in deps.edn?

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.oracle.database.jdbc</groupId>
      <artifactId>ojdbc-bom</artifactId>
      <version>19.7.0.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>com.oracle.database.jdbc</groupId>
    <artifactId>ojdbc10</artifactId>
   </dependency>
</dependencies>

if possible, of course.

for the context: https://www.oracle.com/database/technologies/maven-central-guide.html#DIY

@abdusalam Pretty sure you'll need to figure out what that Bill Of Materials expands to and add each of those dependencies directly to deps.edn

thanks, Sean. does it mean there's no direct, built-in support in tools.deps for this kind of use case?

Not as far as I'm aware.

no, deps does not support the BOM / import scope stuff

it seems radical, but deps just expects you to list the specific dependencies you're using :)

i see. oracle's jdbc driver (com.oracle.database.jdbc/ojdbc10) used to list other "extra goodies" as dependencies in its pom.xml. i was looking for an elegant way to exclude the extra stuff in order to avoid doing the "exclusion dance" but just found out that it's dropped all of the dependencies since version 19.7.0.0. all we need to do now is to add com.oracle.database.jdbc/ojdbc10 {:mvn/version "19.7.0.0"} to get just the driver itself. 🙂

all good now

yeah, that's all most people need. the whole bom thing seems like something an enterprise java developer thought up while they were high

So I really appreciate the expanded deps guide and reference, but there are two things from the previous iteration of the docs that I found useful that appear to have disappeared now: One was the old deps data-flow/arg/alias mapping diagram: https://github.com/clojure/clojure-site/blob/master/assets/images/content/guides/deps/deps.png which I appreciate might now either need to be two diagrams, or reworked into a more complex one. Whilst it’s not something I used often, I thought it was effective in the guide at showing what aliases influenced each phase of execution etc. And the other perhaps more importantly was the “Aliases” section in the old docs here: https://github.com/clojure/clojure-site/blob/285fda270445a70af9b1ec7bdb069309d2d224d7/content/reference/deps_and_cli.adoc#aliases which clearly described how keys in a deps file were used by the command line alias flags. Are there any plans to bring updated versions of these sections back?

Second question — how stable are the new features in tools.deps? In particular the -X feature and the args map data? Are they likely to experience breaking changes between now and the next stable release? I get the feeling not. Sometimes it’s not clear to me what is blessed as stable and isn’t… i.e. the installer, the command line args, or the tools.deps api. (Though I guess it’s all technically alpha because it’s t.d.a not t.d)

@alexmiller how come you are dumping all the new features into :aliases? kinda confusing to give aliases multiple meanings IMHO. maybe I'm just not seeing the bigger picture though.

like why is the exec stuff in :aliases and not a new top-level :exec key or so?

as far as I have understood it: :aliases was always intended for naming a chunk of EDN data that could be re-used elsewhere, but it's only now that we can see that being used in more places

I wonder if you can combine -A and -X to include extra deps in an -X invocation

you can, absolutely

that's kind of the idea

should be stable unless we discover some reason to change. not expecting to discover anything but that's why we're letting people try it. I'm expecting the gap to stable to be like days, not a long time.

:thinking_face: so you can change the function that executed with the default data (by changing the classpath via :extra-deps); but you can’t change the data for a function alias

I'm not sure that's "changing the function" as much as supplying the function

and you can override the data on the command line

maybe I'm not getting what you're describing

is that intended usage? It feels a bit weird to have the same fully qualified function name, but with two different definitions (in two different :paths for example). Would it not be more natural for -X :args to compose?

I don't understand why you have that

can you more fully describe what you're talking about, because I don't get it

@rickmoynihan you mean something like that?

{:aliases {:deploy-to {:fn my.ns/deploy}}
           :prod {:args {:url "<http://prod.com|prod.com>"}}
           :qa {:args {:url "<http://qa.com|qa.com>"}}}

Yeah sorry… I think I misinterpreted the motivation behind what @borkdude was saying as a way of effectively doing what @dominicm was asking earlier about composing :args data. But I misread :extra-deps as being :deps. Either way it seems :deps would kind of allow the kind of thing @dominicm was asking for; but in a nasty way… i.e. you could swap the function rather than the data. e.g. - clj -A:myfn -X:args - clj -A:myfn2 -X:args Where myfn and myfn2 aliases defined the same ns but with different function definitions.

Not saying this is a good idea btw!!!

one helper here is that :args can also refer to an alias instead of a map

oh right… that is interesting!

seems more useful than the :paths one

well the :paths one is foreshadowing :)

I did forget to mention that in my blog writeup, but I think it's in the docs

yes

or maybe it isn't!

I don’t recall seeing it

yeah, me neither

it was in some version of the docs but looks like it got lost in the N reworks I did

😆 easily done

anyhow, that's a thing

wow — I was silently disappointed that something like this wasn’t there

but it’s great that it is

is there an example of how it works?

where else can I put aliases?!?!?!

data

it's just edn

oh sorry, misread your question

if you read https://insideclojure.org/2020/07/28/clj-exec/ check out the basis injection stuff there

I'm unclear if you have a question, and if so, what it is...

Not a question, I already asked mine!

Just clarifying that I don't think this would allow for multiple functions like on after :)

{:aliases
 {:f1 {:fn my/fn1
       :args :data}
  :f2 {:fn my/fn2
       :args :data}
  :data {:something :here}}}

no

Now we have runtime basis, does that mean you can go back and interpret the exec aliases however you like?

you can write a program that uses alias data

which is broader than what you said

yeah, that's not a thing

I would probably do this:

{:aliases
 {:deploy-prod
  {:fn my.ns/deploy
   :args {:url "<http://prod.com|prod.com>"}}
  :deploy-qa
  {:fn my.ns/deploy
   :args {:url "<http://qa.com|qa.com>"}}}

but there may be other variants depending on shared config, command-line overrides etc

also my.ns/deploy could interpret data in :args as something to look up in :aliases via the basis

you don't get that for free, but it's like 2 lines of code

these 2 lines https://insideclojure.org/2020/07/28/clj-exec/#runtime-basis

Yeah, I get what you mean. Basis FTW!

I was surprised that 1.10.1.590 seems to be the latest stable version on brew already, given the flurry of bug fix releases yesterday (1.10.1.596 and 1.10.1.600).

The basis stuff looks very interesting, but you lose access to it if your build your artifacts as uberjars and run them via java -jar (although I guess you could still do -Dclojure.basis='{...}' 👀 )

It almost makes me want to go back to running code in production via clojure instead 🙂

I asked in a thread, but I'll ask in the main channel for everyone to see: given that clojure.basis is a property added by the clojure command, what is the thinking behind leveraging this in production code if you normally build uberjars and run those with java -jar? All of that lovely combined deps.edn structure has gone by that point...

It almost feels like it's worth switching to using clojure to run uberjars via an alias in the deps.edn so that all that stuff remains available in a "production" style context...

my guess is that an uberjar tool could convey the basis into the jar

the more I think about uberjars, the more disadvantages I see vs. sync'ing deps

The basis differs depending on the aliases used to run clojure tho', right?

(because the paths and lib map etc can be different from invocation to invocation -- although the set of aliases etc remains the same)

I'm shooting from the hip

i'd mostly want the alias data

for stuff like config

I'd add it to depstar but I think at this point I want to wait and see what tools.build ends up being 🙂

I might experiment with using clojure to run uberjars tho' later today puts on mad scientist hat...

wdym?

using clojure instead of uberjars?

Having an alias that treats the uberjar as a :local/root dependency and runs the expected -main function inside it, so you get easy access to the t.d.a. / Clojure CLI infrastructure

This:

:aliases
 {:run       {:deps {worldsingles/worldsingles {:local/root "../../build/uberjars/worldsingles-1.0.0.jar"}}
              :fn worldsingles.publisher/-main}}
  ...

So you get the "benefit" of an AOT'd uberjar with the benefits of the Clojure CLI. And the program can read the basis 🙂

(of course, now you need the Clojure CLI installed on a target server, not just the JVM, and you need a deps.edn file in the folder where you start the app)

And internet access

And credentials for private repos

@vlaaad Not if the entire :deps is just the uberjar, right?

There are no other deps to fetch at that point.

It shouldn’t be?

Ah, sorry, misread the situation

I was thinking about using clj without uberjar

https://github.com/clojure/homebrew-tools/blob/master/Formula/clojure.rb has 561

Right, which we used to do on production (and, on one server, for "reasons" we still do that).

So I’m curious why you said that

I just did brew upgrade clojure on WSL1 on my Windows laptop last night and it updated me to 590. Hence my surprise.

Note: brew on Linux.

That’s homebrew core, which I no longer update

clojure/tools/clojure is official

I know. Hence my surprise 🙂

Well I released 590 a week ago, just took me a while to doc and announce

But I guess somebody updated it

Not that I really care, just wanted to report that if you call a private function with -X, it also works:

borkdude@MBP2019 /tmp $ cat src/foo.clj
(ns foo)

(defn- foo [_]
  (prn :foo))

borkdude@MBP2019 /tmp $ cat deps.edn
{:aliases {:foo {:fn foo/foo}}}
borkdude@MBP2019 /tmp $ clojure -X:foo
:foo

ANybody else having issues with the recent tools-deps docker?

Specifically

Caused by: Syntax error compiling . at (aleph/http/core.clj:251:3).
Caused by: java.lang.IllegalArgumentException: No matching method setTransferEncodingChunked found taking 2 args for class io.netty.handler.codec.http.HttpUtil

(I'm making assumptions here because nothing else has changed)

that's not intentional but interesting :)

it does make sense given the impl

@p14n change in JDK version perhaps? Last night I was helping some beginners debugging a weird reflection-based problem that changed between JDK 11 and JDK 14 (it was broken on 8 and 11 but worked on 14).

Because requiring-resolve can get at private Vars and there's no access check on that dynamic call path, right?

That's what I thought. I switched to openjdk-8-tools-deps (tools-deps now 11) but no dice yet

right, just like calling through a var

Fixed by using clojure:openjdk-8-tools-deps-1.10.1.502 🤷

A change in the default Clojure version perhaps, with stricter syntax checking?

i don't use any of the clojure:* images

they are not made by clojure team, and are not necessary IMHO

How is the groupId derived in clojure -Spom? mine keeps getting reset to the artifactId when I run it.

I use an adoptopenjdk image and then add the clojure installer

That's a bug. That should not happen.

(and it's a new bug ^ @alexmiller)

ah, interesting. this is with 1.10.1.590.

I don't think anything has changed there

(! 1133)-> head pom.xml 
<?xml version="1.0" encoding="UTF-8"?>
&lt;project xmlns="<http://maven.apache.org/POM/4.0.0>" xmlns:xsi="<http://www.w3.org/2001/XMLSchema-instance>" xsi:schemaLocation="<http://maven.apache.org/POM/4.0.0> <http://maven.apache.org/xsd/maven-4.0.0.xsd>"&gt;
  &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
  &lt;groupId&gt;seancorfield&lt;/groupId&gt;
  &lt;artifactId&gt;next.jdbc&lt;/artifactId&gt;
  &lt;version&gt;1.1.569&lt;/version&gt;
  &lt;name&gt;next.jdbc&lt;/name&gt;
  &lt;description&gt;The next generation of clojure.java.jdbc: a new low-level Clojure wrapper for JDBC-based access to databases.&lt;/description&gt;
  &lt;url&gt;<https://github.com/seancorfield/next-jdbc>&lt;/url&gt;
  &lt;licenses&gt;

Wed Jul 29 12:17:19
(sean)-(jobs:0)-(/Developer/workspace/next.jdbc)
(! 1134)-&gt; clojure -Spom

Wed Jul 29 12:17:30
(sean)-(jobs:0)-(/Developer/workspace/next.jdbc)
(! 1135)-&gt; git diff
diff --git a/pom.xml b/pom.xml
index 3dce875..5632254 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1,7 +1,7 @@
 &lt;?xml version="1.0" encoding="UTF-8"?&gt;
 &lt;project xmlns="<http://maven.apache.org/POM/4.0.0>" xmlns:xsi="<http://www.w3.org/2001/XMLSchema-instance>" xsi:schemaLocation="<http://maven.apache.org/POM/4.0.0> <http://maven.apache.org/xsd/maven-4.0.0.xsd>"&gt;
   &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
-  &lt;groupId&gt;seancorfield&lt;/groupId&gt;
+  &lt;groupId&gt;next.jdbc&lt;/groupId&gt;
   &lt;artifactId&gt;next.jdbc&lt;/artifactId&gt;
   &lt;version&gt;1.1.569&lt;/version&gt;
   &lt;name&gt;next.jdbc&lt;/name&gt;

well, nothing intentional :)

The only change it makes is to overwrite the groupId

Do you host your own?

can you file a jira, I'm juggling many conversations right now

Will do

@seancorfield The expected behavior was just that it should leave that value alone when the pom.xml already exists?

Hopefully that captures it: https://clojure.atlassian.net/browse/TDEPS-157

The function that turns (merge-cli-ops {:default :opts} "[:extra :opts]" "true") => {:extra {:opts true} :default :opts} is public?

@cap10morgan Yup, it should only touch the dependencies in an existing pom.xml file.

sorry, I don't understand the question

well, it's broader than that - deps, repos, directories

there has been some work done internally here to make this more configurable and that will eventually make its way into user configurability

or how it relates to tools.deps

I think that is this one https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/clj_exec.clj#L59

I liked the "DSL" to "assoc-in" values.

Oh, yes that’s it. Treat anything about this code as implementation detail though

It may migrate into Clojure itself