Clojurians Log v2

How to pass vector as one of args for -X ? Example:

(ns xml)

(defn merge-files
  "Merges given XML files. Options:
   :files Vector of filenames, required
   :output Filename for merge output, optional"
  [{:keys [files output]}]
  ...)

clj -X xml.merge-files :files '["file1.xml" "file2.xml"]' :output "file3.xml"

why do you say it is invalid?

you also likely need to single quote the last arg

clj -X xml.merge-files :files '["file1.xml" "file2.xml"]' :output '"file3.xml"'

alternately, you can now pass a trailing map in the latest version

clj -X xml.merge-files '{:files ["file1.xml" "file2.xml"] :output "file3.xml"}'

thanks, works now, kind of Actually previously I tried to pass the vector '[file1.xml file2.xml]' I'm having now another issue: `Unqualified function can't be resolved: xml.merge-files' The full call is like that

clojure -Sdeps '{:paths ["scripts"]}' -X xml.merge-files :files '["~/.m2/settings.xml" "/tmp/settings.xml"]' :output '"te.xml"'

scripts is just a directory holding xml.clj file xml.clj file defined internally (ns xml) and (defn merge-files) Is there anything else I need to do?

Alright, I should call clj -X xml/merge-files instead of clj -X xml.merge-files

yes, sorry!

no no, my fault, thanks for help 🙂

i've got a strange bug that i just cannot figure out. We build our release with a clojure cli based program. It has local-roots for a project that builds the actual uberjar, and it includes the paths :paths ["src" "resources"]. And for whatever reason, this resources path is not showing up on the classpath for my coworker when running this release script, but it is for everyone else

I got him to run clojure -e "(run! println (clojure.string/split (System/getProperty \"java.class.path\") #\":\"))" and saved his output and my output and the only difference (after adjusting paths from his workspace to mine) is

diff ~/projects/work/diffs/my.classpath ~/projects/work/diffs/his.classpath
3d2
< /Users/coworker/workspace/metabase/bin/build-mb/resources

and we are both on Clojure CLI version 1.10.3.855

did you -Sforce on both?

does the directory exist on both?

no. and i wish i would have told him to try that. are the cache's transitively used like that?

yeah. dir exists, and the file it should find is there. it just seems to be a stale classpath. and i believe the resources addition was recent, so if there's a stale classpath around that would make sense. didn't think that it might use a cached one of a local root though.

changes in local dep deps.edn are not automatically picked up - you have to -Sforce to pick up those changes

at both locations?

or just the dep or just the top level?

top level is where it matters

since it's that classpath you care about here

awesome. thanks. think he's in an interview but we'll try that. thank you very much as always

in the leaf local dep, any changes in deps.edn should make that project stale and recompute there in that project

but things that depend on it will not automatically detect that change

oh i see. so the root's cache includes the old classpath and it doesn't know to invalidate

yep

got a good model of it now. thanks

for the wrapper to know to check that file, it would have to understand the deps model, which would mean running Clojure code, which would negate the benefit of the cache :)

i guess the cache doesn't include a hash of the deps.edn files. i've never looked inside of cpcache really.

I do think it could remember this info in the cache though and use that to determine the set of files to check for staleness

there is a ticket for this, just has never been something I've gotten around to

i'll go look and vote for your next pass through. thanks

i think this is it but it's pretty verbose: <https://ask.clojure.org/index.php/7860/classpath-caching-does-invalidate-local-manifests-changed?show=7860#q7860>

yeah, that's it

the hash includes knowledge about the current project's deps.edn files, but not ones in the cp

we have a new build script at work, and with it I (and no one else, including the CI server) sometimes get weird errors when running the tests about the clojure.core.cache.wrapped namespace being missing. I changed the test runner to pop open a repl when it hit that error instead of exiting, and poked around and found that clojure.core.cache.wrapped had been added to *loaded-libs* but the namespace didn't actually exist, when I removed it from *loaded-libs* it loaded fine. That to me sounds like a concurrent code loading issue, and some digging and monkey patching of require to dump stacktraces when called off the main thread and I have provisionally tracked it down to https://github.com/clojure/tools.deps.alpha/blob/master/src/main/java/clojure/tools/deps/alpha/util/S3TransporterFactory.java#L48 , I have seen the s3 transport do this in the middle of my test run: call back into clojure, which causes core.async and a ton of other deps to load, which races with code running the tests.

is there some way to call calc-basis synchronously?

you can pass -Sthreads 1

that's one source of multithreading in the download process, but doesn't actually get to that thing in S3TransporterFactory

I guess I could require clojure.tools.deps.alpha.util.s3-transporter in our build script, which would no-op the transport factory requiring it

I'm not sure exactly what your setup is but the S3 stuff is used to build a classpath via tdeps, so not sure why it's in process with your tests? (should be separate process)

unless you're using tdeps programmatically

our build script is

then yeah, I'd try to force the load earlier

I guess I could use the require lock in the background load

dunno that would help, the test code being loaded is using require without the lock

true

We don't need the S3 stuff -- is there some env setting that prevents t.d.a. from even trying to load it? Or is the fact that it's part of that Java class's initializer making that impossible?

it's such a pain in the ass given how it's all initialized (see https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/util/maven.clj#L166)

and there's a Named annotation on that class so ends in the META-INF in the jar etc

nothing like dependency injection frameworks to ruin everything good about dependency injection

it is intended to be tolerant of not finding the class (which is just a crutch for using it without compilation during dev)

so you could just delete the class out of the jar if you were feeling crazy :)

or use it as a git / local dep

but if you want something better, file a TDEPS ticket and I'll think about it better in a week or two

it defeated me trying to to even find how it was initialized because I was grepping for 's3' not 'S3' 🙂

if only I could just hand an instance to the thing that was using it rather than gen'ing a class and passing a class name

Yeah, when I use t.d.a from source -- via the add-libs branch -- I get the warning about S3 not being found but works 🙂 so maybe we'll just change out dependency from a maven version to a git sha and just live with the warning about S3 Transport not being loaded 🙂

@hiredman I sent you the :git/url coords, to save you looking it up.