Clojurians Log v2

I made it to my first runnable JAR file! Still reading the (long) README...

I have an idea about the interface naming convention @seancorfield. In workspace.edn we currently support setting the name of the interface with the :interface-ns attribute, where “interface” is the default value. My idea is to support another option where the interface name always has the same name as the the interface, by e.g. setting interface-nsto :interface-name so if the name of the interface is database then it will expect to find a database.clj namespace at the top level (which would be the interface of that component). Then you are free to put the implementing namespaces wherever you want, e.g. at the top level and/or in sub namespaces. We could also support setting interface-ns to :component-name so that if you have two components that implements the invoice interface, e.g. invoiceand invoice-remote , their respective namespaces would be`invoice.clj` and invoice_remote.clj . I’m not sure what’s best here, and if we should support only one behaviour (then all Polylith codebases would act the same which has its benefits) or if people would prefer to choose one of the three different options (or maybe restrict it to two). We haven’t discussed this within the Polylith team yet, but I like the idea so far.

I think that's a good "alternate behavior".

I think it addresses the random/arbitrary naming concern and it makes the code easier to navigate in an editor.

I'm thinking about the :interface-name vs :component-name... I don't feel I've had enough experience with Polylith to comment, but given you can create component name:... interface:... I'm not sure how :component-name would work?

:interface-name feels like what I would want based on my (small) experience today -- and given how we already organize our monorepo.

(BTW, the biggest difference between our monorepo and Polylith is that we don't distinguish between bases, components, and projects -- those all exist as subprojects at the top-level under their unique names)

I have the same feeling, that :interface-name + “interface” would be enough.

Ok.

Yes, you get the sharing with the monorepo, but to replace things (and the encapsulation) you need the interfaces.

Yeah, that replacement isn't a pressure we've found (in a decade of work with, now, 112K lines of code).

We don't really draw a distinction between what Polylith calls bases and projects either, to be honest. Ours have always had a 1:1 relationship. So far 🙂 But it's an interesting distinction to consider.

So basically, to get Lego-like bricks, you need all of them: monorepo, project, base, component, interface. Maybe you will found pieces of code in your 40 bases that could be extracted into components and shared across these 40 projects or that you will divide these bases into components instead of only using namespaces within bases. Time will tell! 🙂

The Polylith codebase has 25 components, 3 bases, shared across five projects/artifacts + the development project:

For example the file component is responsible for accessing the file system and is used by all projects. This is quite common, that you have common functionality across projects/artefacts that can be put into a component and shared across projects.

We only have about 14 "bases" in your terms but, yes, the other 27 "components" include several that are bigger than Lego 🙂

It would be interesting to see your project, so if yo want, you could execute this and mail me: poly ws out:dot-clojure.ednbecause then I can type poly info ws-file:dot-clojure.edn and get an idea!

(or if you have pushed into a branch already)

We've also made a pragmatic choice to only build one artifact that contains several -main functions that provide a slew of cron jobs. Deploying a separate JAR for each process would add more complication than benefits. I suspect you might consider each of those jobs to be a separate project?

Not sure what you're referring to with that dot-clojure reference?

And poly wouldn't run in our work project because it's not a "poly workspace"

Ok, thought you converted the dot-clojure project into a Polylith workspace, but maybe you worked on another project when testing Polylith.

https://github.com/seancorfield/dot-clojure <-- this is what I'm referring to by dot-clojure; it's my user-level deps.edn file (and a support script dev.clj).

It's not a "project" with code.

The api artifact can answer the question what projects that have changed since the last successful build. Then it’s up to the build script to handle all projects i one build or to have separate build projects.

It's just developer conveniences for work with every project.

Okay, now I see!

It's where folks who use the CLI a lot would typically put all of their developer tooling that isn't specific to a project.

Ok, I get it.

Practicalli has one too https://github.com/practicalli/clojure-deps-edn (much better documented and intended more for beginners to learn the CLI with).

If a Polylith project is going to be entirely in-house and never publish libraries, it would seem that the :top-namespace isn’t really needed. Can it be nil so users can use whatever full namespace names they want?

I ask because I realized that at work we have older namespaces as worldsingles.* and some newer ones as ws.* — most of our newer apps are ws.<project> for the top-level entry point — but we also have some <project>.* ones. Yes, I know, potential conflicts yadda yadda yadda, but I’m just wondering about how much flexibility Polylith could allow in that area…

As a bystander in this discussion I love the way Sean is pushing polylith to be less strict without losing the important parts.

Thanks. I’m trying to find what aspects of Polylith add value over the monorepo/deps setup with already have. And I have to say that the command-line tool that track dependencies and git status and what needs testing is very interesting (we have some of that in an assortment of scripts but it’s pretty ad hoc).

I’m an old skeptic so I need to be convinced something “restrictive” really does improve things 🙂

Yeah, I worked in a monorepo and we also had lots of adhoc scripts to only test necessary code but I would like to have a standard for that. I think the problem is other people restrictions, we are usually fine with the ones we come with. 😉

We have a general shell script (called build) that does some of what poly does including “run all tests that affect this ‘project’” and “run tests for all subprojects affected by this changed subproject” — but our subprojects are pretty coarse-grained and our deployment artifacts (uberjars) and “big” — in my mind, at least: a couple of them are 70MB but the smallest are under 30MB and I’d like to tease apart our codebase to produce smaller artifacts, perhaps even if it means more artifacts. But converting it to a Polylith structure would be a massive undertaking at this point. If I can break up things into more “component-sized” pieces, that will be a win.

I actually implemented support for having arbitrary top namespace per brick (component and base) earlier, in a separate bransch. We decided to not merge that code because it made the Polylith code more complex but maybe more importantly, it complicated the use of Polylith and as you said @seancorfield it also opened up for naming clashes. But yes, it is possible.

It wouldn’t be a bad thing for us to adopt at least a standard short ns prefix at work but… this is a codebase that goes back over ten years and we were new to Clojure back then… ¯\(ツ)/¯

You can put the code for each “project/artefact” that you have today @seancorfield into a base + make sure that all of them use the same top namespace, e.g. com.company and then each base would live in e.g. com.company.mybase. You would have to refactor the namespaces for each base to conform to that pattern. Then you could create a project for each base and include only that base in each of them. That should be doable.

I was thinking more: * Anything that starts with ws leave alone * Anything else rename to add ws. prefix * Over time, rename ws.worldsingles.* to something more appropriate as I move code around 🙂

You don’t need to convert the whole codebase, you could start with one base and then migrate one “project” at a time.

But, as I’ve noted before, we have what you would call “bases”, that have namespaces like ws.billing so our repo structure tends to be like wsbilling/src/ws/billing.clj and the immediate implementation is ws.billing.stuff 🙂

We decided to also support your suggested naming convention of the interfaces, by the way!

Oh, cool! I think Polylith will be more approachable for folks if it looks a bit less like a set of “weird” naming conventions and directory structure and more like a “reasonable” convention/structure whose goal is to make tooling easier to write/use — and then it won’t distract from the architecture portion which is the Lego-like aspect of it.

I don’t really agree that the four top directories components, bases, development, and projectsare weird. It’s kind of the opposite. It really helps you understand and reason about the codebase, not the least if you are new to a codebase. I know that you like your setup that you are used to, but having a directory called componentswith let’s say 50 directories with descriptive names and one entry point (the interface) is actually super helpful. @furkan3ayraktar has worked with several Polylith projects (around five) and the benefit you get by this structure has a lot of value.

I know you don’t think they are weird, but this is part of why several folks react to Polylith as “a naming convention” or a “directory structure” 🙂

I will say that we find having the top-level directories in a monorepo being related to the business or functionality is “natural” for us (and it seems to be the more common monorepo structure).

@tengstrand Can you remind me what the thinking was behind have projects having deps.edn vs being aliases in the root deps.edn? I think you explained but can’t find it in the chat history so I don’t know where we discussed it.

There are a reason why we use interfaces, and that is the same reason interfaces are used in OO, to decouple parts of our system and to be able swap what concrete implementations to use without affecting the consumer of that interface. That is an example of why we use different concepts like interfaces. They are there to give you more value and in the end to help people to work efficiently with the code and to be able to easily arrange how to execute the code in production. As I have said before, it’s like Clojure, it’s hard to convince people that are used to work in a different way (like OO) but when you have used it for a while, you start to appreciate the fast feedback loop and the Lego-like feeling it gives you.

@tengstrand Just checking: Did you miss this Q? Can you remind me what the thinking was behind projects having deps.edn vs being aliases in the root deps.edn? I think you explained but can’t find it in the chat history so I don’t know where we discussed it.

I think it was to do with some restriction in t.d.a / deps.edn but I can’t remember the details…

At least the last time we checked, the IDE we used didn’t pick up the test directories for the bricks when we used the`:local/root`syntax, which affected the development experience negatively. For the other projects, we use our own test runner to solve that problem, which means that we can include the bricks as dependencies.

The reason we put the paths in the`:dev` alias as extra-paths in ./deps.edn is that we wanted to isolate the dependencies we use for development into one place.

That doesn’t sound like an answer to my question. I’m not sure how we’re miscommunicating here.

Okay, now I understand your question…

I’ll try from another side: in Polylith, it seems that you have to cd projects/myapp and then run clojure there to do the compile/packaging… we do it via aliases in the root deps.edn file so we don’t have to cd, we can build directly from the root folder.

Perhaps partly because we use depstar and it handles the classes folder and does AOT etc all in one place?

(it actually uses a temp directory for classes so it doesn’t clutter up your workspace)

A project can contain many bricks and also aliases. If you have many projects, we liked the idea to let each project have their own deps.edn file instead of putting everything into one huge deps.edn file. But maybe you just mean the “building” aliases?

So far, we don’t require people to install any extra tooling like depstar or similar to be able to work with the code and to build it (the poly tool only makes it more convenient, but it’s not required).

We’re still not communicating.

Maybe a video meeting some day could be an alternative?

I’ll go back search the chat history in various places and see if I can find the comments you made about projects that I’m referring to.

Ok

The way I see it, polylith projects are independent entities with public apis, so it makes sense for their dependencies to also be decoupled from the root deps.edn.

As in, let's say you're developing an rss client. You could have 4 polylith projects: rss feed api, cli, desktop, web

From the perspective of the user, would it be better to have a single deps.edn for all of them or keep them separate? Edit: by "user" I meant anyone who might want to use your code. That could include future you, or the maintainer of another repo.

Not sure if it helps at all in the discussion (feel free to ignore it if not) but that's my $0.02 :man-shrugging: