Clojurians Log v2
clojure
Is there a point cloud library or an open 3d alternative for clojure?
We strongly discourage posting the same question in multiple channels @ps
hey! I'm refactoring the state of an application, and I'd like to state an invariant that has to be checked every time some operation is done on the state. I think spec is the right tool, but if I understand correctly, instrument is only used to instrument functions, right?
I don't think it is what spec is for. Since the invariants is usually a part of your domain logic, explicitly asserting/checking it should be the way to go.
What’s a good way to downgrade the jdk installed by the homebrew clojure formula?
If I understand correctly, only a formerly-but-no-longer-supported version of the homebrew clojure formula installs a particular version of the jdk. If you use the latest officially released version of the homebrew Clojure formula, it will not install any version of the jdk for you, and you can pick any jdk you want: brew install clojure/tools/clojure as stated on https://clojure.org/guides/getting_started
So starting with brew uninstall clojure and removing the brew-installed jdk you do not want is probably a good start.
stuartsierra/component's [readme](https://github.com/stuartsierra/component) says "No function should take the entire system as an argument ... If a function depends on several components, then it should have its own component with dependencies on the things it needs."
What's the reasoning for that? That reminds me of OOP a lot, where you would make a new class just to pass in structured data to a couple of functions or something like that. Why is passing the system and destructuring is not ok, like (defn f [{:keys [db task-queue]} ...] ...), and calling (f system ...) ? He says "This is unnecessary sharing of global state.", but I don't see how that is, if each function takes only what it and the function it calls needs.
regarding routes using components, my solution has been for each component to make a wrap-foo function, and to have the routing code extract and use those, rather than passing the entirety of the component to every request handler
(the wrap-foo function is provided as a key in the component map)
@noisesmith what does wrap-foo do?
it's a middleware that offers some stateful resource to the handler - eg.
(fn wrap-connection [handler]
(fn [request]
(handler (assoc request :db-connection conn))))so the implementation detail that the connection comes from a component is invisible to the handler, which finds some resource under a key, as it would with any other middleware enabled feature
this means changing how state is managed doesn't require refactoring request-handling code
pragmatically, one of the biggest drawbacks of passing your mega state-blob everywhere is that you need to edit your entire codebase if you need to restructure that blob, this reduces the scope of that problem
which is part of what Stuart Sierra is getting at in your original link
one way to interpret "structured programming" is to see it as a series of firewalls which minimize the number of changes needed when you add features or change implementation; putting all your resources into a single hash-map that every function sees is effectively an unstructuring, spreading implementing dependencies through the cracks of your structure
@noisesmith Interesting approach. So you have explicit middleware invocations around handlers directly in your routes definitions? Doesn't that get ugly with dozens of components in play? (or do you only have a few components in your systems?)
@seancorfield this was done with an interfacing function that consumes all the wrap-foo functions and composes them in the desired order, returning a composed middleware for the handlers
so no, it's not invoked at the point of the handlers
Ah, so all handlers have access to all of the (relevant) system component parts separately but not the whole system component? Effectively "unrolling" the compound system to place just the lower-level pieces into the Ring request -- in our case that would go from having the "Application" in the request, where handlers would need to unpack it to get at :database and then :pooled-db or whatever nested resource, to the Ring request having those nested resources directly available under keys in the request?
Thus removing knowledge of how the System is structured.
precisely - nothing stops using namespaced keys / assoc-in instead of assoc for those resources
but the key is that it's not based on knowledge of the system's structure
I don't think we've ever found the need to restructure our system-level component in a way that would break functions down the call chain but I can see the benefit of "flattening" the component into the request.
it might be that our benefit was partially because we had "weird" components which exposed some implementation detail of external service usage etc., so we did have restructuring of our component map as our architecture evolved
and the bottleneck of providing things through functions in middleware helped keep things to an interface
Ah, that makes sense then. Yeah, I can definitely see the benefit in decoupling this. I'll bear it in mind for the new web app I build 🙂
Thanks Andy, that tap was key, somehow brew install clojure is hardwired to install jdk 15?
Yes. There was a change made, maybe about a year ago?, by some Homebrew maintainers to cause many projects that depend upon a JDK, to depend upon a recent version of the JDK.
Clojure devs attempted to persuade them not to do so for Clojure, but they did not agree, so a separate Homebrew project was created for Clojure that they apparently allow to have control over that.
@posobin Component library has very OOP-ish approach. Try another libraries like mount https://github.com/tolitius/mount or context https://github.com/redstarssystems/context
Well I am already using component =)
I saw @seancorfield say that mount complects things, and I am starting to agree with him after using it for a while.
Integrant is a nice one as well.
It's not so much an OOP-ism (nothing wrong with that to be honest) but it's about separation of concerns - if you have a function which is only responsible for sending out emails, why would you pass database connections, slack client and a whole config map to it?
Because it might require a db connection later on when I add logging of sent emails to the db?
That sounds like YAGNI
if you will need it later, you will add it
Believe me - I have overcorrected with making everything a component in the early days, and it definitely needs some balancing.
And yes, we had some things depending on the whole system only to realize it was a smell and things had to be taken apart - I think the biggest drawback was to understand the code after while and having the whole system passed in would lead to sub-optimal design decisions
Basically you have a global state, but you're not admitting it because it's always the first argument of all your functions 😉
But adding it is a pain, have to adjust the arity of the function in all call sites.
Do you have a special email-component-bundle in your system then?
My favourite is Juxt Clip
<https://github.com/juxt/clip>
I use that for managing all my "dependencies", such as connections to the db, redis, external APIs etc..
I have a small example of its usage here <https://git.sr.ht/~dharrigan/startrek/tree> with a tiny example project.
You don't have to adjust function arguments - usually you pass the dependency map as the first argument, and you use destructuring to pull them out. In case of the email-sender, they way we have modeled is that the email-sender itself is a component, injected into other bits which needed (and only them)
I see. One other problem I have is that there is a routing function that calls many possible functions that respond to requests, and each of those functions needs a bunch of components (some might need the db and s3, some might need access to the email and db components, etc). How do you handle that? So far the router component just depends on many other components, and passes the db s3 email etc components to the functions it calls as separate arguments. How would you structure this better?
Ah yes - that's definitely one of these cases where you sort of depend on most of the system. I don't think there's a good way around it unless your individual route handlers start being components. That's an overkill IMHO
That said, in that case - the code is always very explicit about which components it requires, so we structure it so that top level handlers responsible for each route, pick out which components to pass down (effectively a subset of dependencies). This way, for convenience you can always call a function with the whole system (e.g. in a REPL) but you can always be sure about which components it will use as these usually get pulled out via destructuring of arrgs
(damn it my macbooks keyboard is on its last legs...)
@posobin The only solution I’ve seen for that is to have the whole system available to the top-level route (usually tacked onto the request), which then destructures out the components it needs.
IME this works fine with few problems. You can still find what fn uses what components by keyword searching if you use namespaced keywords.
> Project status: Alpha.
The only thing that stops me from using clip.
@potetm @lukaszkorecki How do you pass the components to the particular route handlers? As separate arguments or make a map from the ones that are needed and pass them? Like I am passing each component as a separate argument, and some routes didn't need the db connection previously, but now I added saving some logging to the db, and now need to pass a new argument to a bunch of functions.
pass the whole system map to every route handler
is what i was suggesting
Sorry, not route handler, but the function that the route handler calls.
really just a matter of taste
I usually do separate args
Similar ☝️ the handler is a component, which passes down the dependencies as a key in the request map, and then each handler function for a particular route unpicks what they need and passes it down as a map
(logging components seems like overkill to me. I use it too prolifically.)
But if a fn takes a bunch of components, I’ll accept a map
I think Stuart’s advice is a little over done. It’s helpful for a fn signature to tell what components it needs. Namespaced keywords provide similar utility.
(arguably namespaced keywords have better utility)
Can you expand on how namespaced keywords help?
the upside of separate args for each component is you can tell by looking what the fn might or might not do
if you destructure+namespace-key, you get basically the same experience
plus: spec support and symbol-searching support
We have a rule to always pass components (even if it's just one) as a map - otherwise some functions would have 5 component args + input. A bit too much for my taste
The downside of passing a big glob of components everywhere is it’s impossible to tell who’s doing what. You can mitigate this by destructuring as high in the stack as you can bear.
I just have trouble coming up with how I would namespace components. I have the db, email, s3 components in mind.
Are you speccing your system?
In most of my code that uses Component, I put the "application" component (everything in the "system" except the web server itself) into the Ring request via middleware created at startup. So every handler has access to the "application" and then it can pass just what sub-components are needed down into the "model" functions that the handler calls.
(component/system-map :app/db (db), :app/monitoring (monitoring))
should work, no?
(I’m honestly not sure. I work in a home-rolled component system. Do not do that. Cannot recommend.)
We currently have a :database component that has several datasources nested inside it. If we were starting over, we'd probably use namespace-qualified keywords and not have them nested.
I’ve not spec’d my system. (It’s garbage, and cannot be spec’d.) But there’s nothing stopping you from saying :app/db must be a DataSource or smth along those lines.
I think worst case is you have to un-namespace those keys so that they can play w/ records (which do not support namespacing).
You can do that with component/using
Ah, the namespace is not for grouping components, but just so that you can spec them?
But you can spec without namespaces too. @seancorfield's example with several datasources makes more sense.
Spec'ing is orthogonal to this.
And you don't have to use records for Component. The protocols can be implemented via metadata on anything that can support metadata.
Ooh, didn't know that latter point.
@seancorfield You said "in my code that uses Component...", is Component your preferred library for state management, or do you like some other one more?
For components that have dependencies, you can use hash maps or records. For components without dependencies, you have much more leeway.
In next.jdbc.connection I have a function used as a component.
At work, we use Component extensively. We have a monorepo with over 100K lines of Clojure, in three dozen subprojects, that are combined into just over a dozen services.
One of our apps is structured with every Ring handler being a component with its own dependencies. The rest are less granular. We have components for database connections, caching, our HTML email templating system, ...
...our environment/configuration system, our JWT-based security component, Redis pooling, Redis pub/sub, and many more. We have 43 implementations of component/Lifecycle apparently (some in test fixtures where we use mocked components). Looks like we use the metadata protocol approach for three more component implementations. @posobin
most of the core clojure libraries have been alpha until very recently...
anyhooo, I've been using Juxt Clip in production, on systems running 24-7, processing millions of messages per day. Zero issues.
A testiment to how well it has been written and how well Clojure itself is put together 🙂
It's not the quality that concerns me, it's the probability of backwards incompatible changes.
Although, if alpha is stable enough then there might be no reason to update to a newer version that changes some API.
That is always a problem with any library one uses. A breaking change at some point. One deals with it, if it arises. However, I'm encouraged by this line The core API is unlikely to change significantly, but there may be some small changes to the system-config format
Does anyone know of a code security scanning tool for Clojure? I imagine it to be quite tricky to do without many false positives. However, it checks a box on security checklists and seems to make certain customers happy.
When running a REPL w/ tools.deps is there a way to access the names of the active aliases?
@denik No. That information is gone by the time the process starts.
The aliases are used to create the classpath that in turn is used to run the REPL.
@kenny I've never heard of a useful security scanning tool that works with Clojure code.
Running the scan is not the same as reading it, of course :).
Yeah... We're trying Fortify's static scan and I imagine it's going to be completely useless. It's unfortunate that so many people consider scans like this as a base-level security metric to work with you.
I've done a huge amount of work with static source code analyzers over the decades and while there are a few constructs which can be identified in nearly all languages as being suspect, in order to "sell" a security tool, it has to be "thorough" and err on the side of false positives, which in turn makes the tools pretty much useless really.