observability

o11y, monitoring, logging, tracing, alerting and higher level discussions
athomasoriginal 2020-07-08T16:27:47.088800Z

Finally had a chance to setup log4j2 based on recommendations from this channel. It works great! My question is around what needs to be excluded (if anything). For example, I followed https://gist.github.com/ataggart/ac208c289c5d01dacf3b4b341a1c37f0 + Sean’s suggestions. Now, TMK the exclusions from the gist, because i’m using deps.edn , don’t happen globally but at the individual dep level. So, how can I figure out which dep I need to add the excludes to? Do I even need to exclude anything?

seancorfield 2020-07-08T16:41:53.089300Z

@tkjone I've never needed to exclude anything.

👍 1
plexus 2020-07-08T16:46:50.090100Z

if you do want to check if something is being loaded that shouldn't then clj -Stree is your friend

athomasoriginal 2020-07-08T16:54:29.093600Z

So, just to confirm I understand, we would just run clj -Stree and if any of the deps specified in https://lambdaisland.com/blog/2020-06-12-logging-in-clojure-making-sense-of-the-mess (as an example) appear as a dep of one of my logging library, those might be good candidates to exclude? As an example, I can run clj -Stree right now and I get something like

; ...
org.apache.logging.log4j/log4j-jcl 2.13.0
  commons-logging/commons-logging 1.2
; ...
Perhaps I would exclude commons-logging/commons-logging from org.apache.logging.log4j/log4j-jcl.

seancorfield 2020-07-08T16:57:08.094800Z

If you have all the log4j2 bridges as top-level dependencies, I don't think it matters if anything else brings them in as transitive dependencies?

seancorfield 2020-07-08T16:58:02.095700Z

I don't exclude other logging implementations and I haven't seen any problems (and we depend on a ton of stuff that drags in other logging implementations, I'm sure... but let me check).

seancorfield 2020-07-08T17:00:29.096500Z

Hmm, apparently less than I thought.

org.clojure/tools.logging 1.1.0
  commons-logging/commons-logging 1.1.1
The rest is all org.apache.logging.log4j stuff (which is log4j2)

seancorfield 2020-07-08T17:01:57.096900Z

(we do not exclude commons logging from tools.logging)

seancorfield 2020-07-08T17:07:47.098100Z

@ataggart That was what @tkjone linked to about half an hour ago that started this discussion 🙂

ataggart 2020-07-08T17:08:02.098600Z

ha ok, I just have a slack alert on "logging"

seancorfield 2020-07-08T17:08:19.099200Z

I've never needed to exclude other logging dependencies -- do you know when that is needed?

seancorfield 2020-07-08T17:08:33.099700Z

(given that deps.edn doesn't have global exclusions)

ataggart 2020-07-08T17:08:40.099900Z

transitive dependencies

ataggart 2020-07-08T17:09:06.100500Z

some libraries declare dependencies on implementations

ataggart 2020-07-08T17:09:16.100900Z

unfortunately

seancorfield 2020-07-08T17:09:22.101100Z

I can see it might matter with Leiningen/Boot because of Aether.

seancorfield 2020-07-08T17:09:57.101600Z

So... maybe I'm just lucky I've never been bitten by it...?

ataggart 2020-07-08T17:10:55.103Z

yeah, I included it in the example to remove one possible source of problems

athomasoriginal 2020-07-08T17:11:23.103600Z

heh, so maybe the take away is that based on my current set of dependencies I am using + the fact that lein + tools.deps have a different process for resolving deps, I am good to avoid exclusions for now (because they are not appearing as transitive deps from what I can see).

athomasoriginal 2020-07-08T17:11:42.104200Z

However, should I add new deps, it would be good to check if any exclusions need to be made

seancorfield 2020-07-08T17:12:25.104700Z

Here's all we have at work

;; use log4j 2.x:
    org.apache.logging.log4j/log4j-api {:mvn/version "2.13.3"}
    ;; bridge into log4j:
    org.apache.logging.log4j/log4j-1.2-api {:mvn/version "2.13.3"}
    org.apache.logging.log4j/log4j-jcl {:mvn/version "2.13.3"}
    org.apache.logging.log4j/log4j-jul {:mvn/version "2.13.3"}
    org.apache.logging.log4j/log4j-slf4j-impl {:mvn/version "2.13.3"}
We don't even have log4j-core

seancorfield 2020-07-08T17:12:33.105Z

(and no exclusions)

seancorfield 2020-07-08T17:13:42.105600Z

Looks like the bridges bring in core anyway:

org.apache.logging.log4j/log4j-slf4j-impl 2.13.3
    org.apache.logging.log4j/log4j-core 2.13.3
from -Stree

ataggart 2020-07-08T17:14:00.105900Z

log4j-core is a dependency of log4j-slf4j-impl

ataggart 2020-07-08T17:14:06.106100Z

damm slow typer

ataggart 2020-07-08T17:14:30.106300Z

ok, going back to work

athomasoriginal 2020-07-08T17:15:22.106500Z

Thanks all!