Clojurians Log v2

Hi, everyone! I’m moving an application over to use component. For the most part, my components are immutable and don’t change at all once the system starts. However, I do have a couple of places where this is not the case. My application makes calls to an external API which requires a new token every 60 minutes. In the current version of my application, I just always reference the token from an atom and I have a background job set up with chime that just refreshes the token every 55 minutes. When I move this to component, I plan to use an atom as property on a component, but I’m not entirely sure if that’s the best approach. Also, should I still stick with the chime loops inside of my component for keeping that token up-to-date or is there a better alternative. How would you handle this situation?

That’s probably what I would do — “encapsulate” all the updating machinery inside the component and set it up in the start function (and tear it down in the stop function if possible). I would probably have the component implement IFn for no args and return the contents of the atom rather than make user code reach into the component for a field and then deref it — that’s a pattern we’ve adopted for a lot of components that “represent” some “value”, so that client code does not need to know how to access the value within the component.

Can you explain the piece about having the component implement IFn in more detail? That sounds interesting. The main downside to passing the atom around everywhere is that any component could theoretically perform swap! or reset! on it, so I’d like to avoid that if possible while still allowing folks to effectively deref it

next.jdbc’s Component implementation for connection-pooled datasources is a function that returns the datasource https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/connection.clj#L322 but it uses metadata to implement start/`stop` so it’s a little harder to see what I’m talking about.

This is an example from work of a “configuration” component, where we implement multiple arities so that (cfg :a :b :c) can act like (get-in cfg [:configuration :a :b :c]):

(defrecord Configuration [application-name edn-path configuration]
  component/Lifecycle
  (start [this]
    (if configuration
      this
      (assoc this
             :configuration
             (load-configuration application-name edn-path))))

  (stop [this]
    (assoc this :configuration nil))

  clojure.lang.IFn
  (invoke [_] configuration)
  (invoke [_ k] (get configuration k))
  (invoke [_ k1 k2] (get-in configuration [k1 k2]))
  (invoke [_ k1 k2 k3] (get-in configuration [k1 k2 k3]))
  (invoke [_ k1 k2 k3 k4] (get-in configuration [k1 k2 k3 k4]))
  ;; shouldn't need any other arities
  (applyTo [_ ks] (get-in configuration ks))
...

Got it. That’s helpful. I think implementing IFn on a component that’s just wrapping an atom will fit my use case. I have a couple of follow-up questions: 1. Is there a reason for (assoc this :configuration nil) instead of using dissoc? 2. Is configuration just a map? In start , could you just return configuration instead of associng it onto this? I’ve only been using component for a couple of days now, so I’m not aware of all of the pros and cons of this approach, but it seems like all downstream components would have configuration injected as a map and would be able to use it, but you lose the ability for any kind of cleanup to happen (since component will be holding a reference to a map instead of an actual component). Is that accurate?

If you dissoc a declared field of a record, it becomes a plain hash map and is no longer a record.

configuration is just a hash map, but start should return a component which can be stop’d.

Component start/`stop` should be idempotent: when you start a component, you should be able to call start on it again and it should be a no-op. In addition, when you call stop on a component, what you get back should be startable — so to call start on a stopped configuration here, we’d need to keep track of application-name and edn-path so that load-configuration could be called again.

Got it. So when I’m designing components, should the fields in my defrecord declaration be all of my component’s dependencies as well as all of the fields my component initializes in start?

Our approach has always been to have three types of field, all declared: 1. configuration for the component (either passed in via map->Component or defaulted inside the start function) 2. dependencies (wired up via using externally) 3. state (computed and maintained in the start/`stop` functions)

Not every component has all three.

However, since Component added support for metadata-based protocol implementations, we are moving more to hash maps and/or functions — like next.jdbc for example — and then it’s not important to “declare” anything or worry about keeping fields around just to stay as a “special” data type.

Thanks for the clarification on the fields question. What’s the “metadata-based protocol implementation” stuff all about? I don’t recall seeing that in the component docs

For example, here’s our “host services” component that figures out the hostname and artifact version at startup:

(defn system
  "Build a 'system' component that contains the cached hostname
  and this process's full version string."
  []
  (with-meta {}
    {`component/start
     #(assoc %
             :hostname (or (:hostname %)
                           @system-starting-message
                           (lookup-hostname))
             :version  @process-version)
     `component/stop
     #(assoc % :hostname nil :version nil)}))

(if-let [address (java.net.InetAddress/getLocalHost)]
      (or (.getHostName address) "")
      ""))

“metadata-based protocol implementation” — it’s a feature added to Clojure in 1.10: https://github.com/clojure/clojure/blob/master/changes.md#22-protocol-extension-by-metadata

Component was updated to add :extend-via-metadata true to its defprotocol form — that’s all it took.

Most protocols I define these days have that flag, unless they can only be extended to things that cannot carry metadata.

Got it. That’s cool. I remember seeing that in the patch notes, but I rarely deal with defprotocol so I didn’t pay much attention to it. I see two main advantages to this approach: 1. I can call dissoc on my map and not have to worry about the whole “it’s no longer a component” thing, right? 2. I can use a “let-over-lambda” approach to wrap values in my component and hide them from being accessed by folks who can access the component. Is that accurate? Are there other advantages?

Also, do you have an example of a component defined via the “metadata-based protocol implementation” approach that has dependencies which are injected by component?

I’m assuming that component still just places those values on the map that’s passed to start and stop?

I don’t have any public examples of metadata-based components with dependencies.

One thing to remember is that Component requires associative data structures for injecting dependencies (and you have to rely on them being metadata-preserving — which assoc is), but you can implement protocols via metadata using things that are not associative, such as functions.

But, yeah, if you’re dealing with a hash map with metadata, you’re safe to dissoc any fields and you still have a “Component” in the sense that it still carries metadata that implements the protocol:

dev=> (meta (dissoc (with-meta {:a 1} {:b 2}) :a))
{:b 2}

And, yes, you can use captured bindings to encapsulate things if you want.

I talked with Stuart about extending dependencies into metadata so that Component would support dependency injection on non-associative objects but he felt that didn’t bring enough benefit to be worth the additional complexity (because associative things still need dependencies assoc'd in as well as having the dependency metadata updated).

It’s an experiment I may still move forward with myself at some point, where both lifecycle functions and dependencies are handled entirely via metadata and for associative things, dependencies would still also be injected. I believe I could make a library that was compatible with Component but I just don’t know if it’s really worth the effort 🙂

Okay, that all makes sense. And yeah, supporting DI on non-associative objects does sound tricky. I saw the example for how next.jdbc.connection/component is effectively turning a DataSource into a component with non-trivial start / stop methods. I’ll be doing something similar with core.async chans in my app. Thanks for all of your assistance on this!

Oh, nevermind on that. core async chan’s don’t appear to implement IObj, so I don’t think I can attach metadata to them I’d still have to pass them around as functions

there are protocols that define channels (ReadPort, WritePort, Channel) so you can create a defrecord that is both a channel and a component if you like