Clojurians Log v2
tools-deps
Q: Is this a silly idea? I want a combination of git deps and “-LATEST” behviour for my CI/test builds. For prod builds I want static git shas but for automated/test builds, it would be great to automatically use the most recent git sha. Is this even possible?
thanks. I’ll look into that as well. I’ve never used them before but, if they add no complexity, I’ll give them a try
@steveb8n I'm using them in babashka, I have 4 of them now.
cool. I’ll look at your repos when I dig deeper
Not everyone likes them, I'm not entirely sure why, they work well for me.
I would certainly keep specific library dependency versions the same for any CI pipeline that had any chance of pushing things into production. Are you able to set up separate CI pipeline that uses a different alias to pull 'latest' versions of libraries? I prefer using :local/root rather than mono-repo or sub-modules as they feel simpler for the way I work. Nothing wrong with those approaches though...
local/root works, but not if the repo you're including is not part of the codebase proper, unless you're writing a script to set up the filesystem in CI the exact same way as on your machine
Is that the case when using jars built from other projects, I thought that worked. Maybe I am mis-remembering.
I mean, with local/root to some other project, you'll also have to git clone that other project in CI right
Ah, yes.
I feel the same @jr0cket in that I like them all separate. it has the added benefit of allowing me to outsource work on projects without needing to grant access to all
so for me it’s.. local deps using local/root, CI using git/sha and prod using the same git/sha
and the maintenance annoyance is updating the git shas for the CI builds anytime upstream projects change.
from this thread, I’ve realised that this is also a good thing because it means that my CI builds always match prod builds. thus it’s probably the lesser of all evils, despite the chattiness of commits
@borkdude: I’ve done the git submodule thing many times in the past. I’ve found it works quite well if you’re careful and have infrequently changing dependencies, but it can become pretty painful if you change branches a lot and have fast moving dependencies. The detached heads in the submodules can also lead to people unwittingly making many kinds of mistakes.
git pull --recurse-submodules
I almost never have such mistakes, but maybe for contributors it's different?
Temporal-basead versions are always bad IMHO
If your CI fail and you run it again, it may take a new commit and success again. So success and fail says nothing.
Pass the commit ID to your CI or fetch and use a sed / template.
Then in case of success/fail, you will know which code succeed/failed
And you will be able to run again the same CI in a older commit/retry
thx. fetch and sed is an interesting idea. I could even use bb for this script.
how would the script apply the latest sha? by writing it into the deps.edn? or is there some other trick?
fyi: the reason I want this is to avoid the endless commits I currently do for all my git deps during dev/test. but I really like using git deps in CI
Running CI and production against different versions seems like a recipe for hard-to-debug disaster...
I can understand wanting your local dev-only tooling to always pick up the RELEASE (or perhaps occasionally the LATEST) release but I really don't want any variance in versions between dev, CI, production for anything I depend on.
(so I guess my answer to your actual Q would be "yes, it's a silly idea" IMO)
that is useful perspective. I’ll apply a hammock to this a little more. thanks
I should have mentioned that these deps are all projects that I control but have split apart for re-use and/or separation of concerns. so I can be sure I know when changes are being introduced. I would never do this for an external dependency
Consider using :local/root maybe, if you have them all checked out locally?
You can always use a :dev alias to use :override-deps to replace the regular deps with your local ones.
agreed. I am using :local/root for these deps for localhost. the annoying commits are required when I push a lib which I depend on in N other projects and then have to update shas in all of them
ideally CI/test would use latest for all these deps. then when I cut a prod release, I would update the shas (once) for all the deps
but I can see that I could introduce human error later i.e. forget 1 sha update in the prod build. that would be bad
that would qualify as “silly”
this maintenance problem seems to be pushing me towards a mono-repo but I like keeping these things separate
Our 100K+ line codebase at work is in a monorepo, with 30+ subprojects, and we build about a dozen deployment artifacts from that...
it’s something I will think about. thanks
@steveb8n I’m using git submodules to prevent endless commits updating git deps. Easy to update them all to latest using one git command. Works with all build tools.
Wondering, is tools.build also going to have a test runner like https://github.com/cognitect-labs/test-runner or is this a separate concern?
Hi, I'm trying to reference the head of the manual-connector-anchors branch on dali here: https://github.com/stathissideris/dali/tree/manual-connector-anchors
and I'm struggling a bit with the deps.edn. Does anyone have a good example of how to reference a branch on github at a :git/url ?
I have it working locally in the mean time
Not building any new test runner but you can make tasks that do whatever
Git deps always use shas so you don’t need to specify the branch
Sha is sufficient
thx! I've got it working now. It was a problem with the deps.edn in the repo I was referencing.
ah