immutant

http://immutant.org Note: dev discussion happens in #immutant on FreeNode IRC.
lorddoig 2017-06-07T11:33:21.230050Z

afternoon channel. I’m having a pretty awful time trying to pump immutant logs through timbre. My deps are set up like so:

[com.taoensso/timbre "4.7.4"]
    [org.slf4j/slf4j-api "1.7.21"]
    [com.fzakaria/slf4j-timbre "0.3.5"]
    [org.slf4j/log4j-over-slf4j "1.7.14"]
    [org.immutant/web "2.1.5"
     :exclusions [ch.qos.logback/logback-classic]]
With this I get logs from org.projectodd.wunderboss.web.Web coming through timbre, but logs from xnio and from when an error is thrown mid-request are sent in some default-looking format to stdout. If I add log4j-core I get the same thing but in a slightly different format. Does anyone know the incantation to make this work? Google suggests it’s possible (https://twitter.com/statonjr/status/541250845300768768)

2017-06-07T11:39:00.297563Z

@lorddoig can you gist the xnio output? It may be writing to stdout in some cases

lorddoig 2017-06-07T11:39:40.305368Z

Sure

Jun 06, 2017 6:21:51 PM org.xnio.Xnio <clinit>
INFO: XNIO version 3.3.6.Final
Jun 06, 2017 6:21:51 PM org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.3.6.Final

lorddoig 2017-06-07T11:39:51.307720Z

That’s it until requests start hitting

2017-06-07T11:45:06.373071Z

if you comment out your timbre & logging deps (and just use the default) does the output from xnio change?

lorddoig 2017-06-07T11:47:26.402161Z

I can’t really comment out timbre as the app won’t boot, but with these:

[com.taoensso/timbre "4.7.4"]
    ;[org.slf4j/slf4j-api "1.7.21"]
    ;[com.fzakaria/slf4j-timbre "0.3.5"]
    ;[org.slf4j/log4j-over-slf4j "1.7.14"]
    [org.immutant/web "2.1.5"
     :exclusions [ch.qos.logback/logback-classic]]
I get
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See <http://www.slf4j.org/codes.html#StaticLoggerBinder> for further details.
Jun 07, 2017 12:46:02 PM org.xnio.Xnio &lt;clinit&gt;
INFO: XNIO version 3.3.6.Final
Jun 07, 2017 12:46:02 PM org.xnio.nio.NioXnio &lt;clinit&gt;
INFO: XNIO NIO Implementation Version 3.3.6.Final

lorddoig 2017-06-07T11:47:49.407011Z

ie the SLF4J warnings are added

2017-06-07T11:54:15.487516Z

I'm playing with it locally to see what I can figure out

lorddoig 2017-06-07T11:54:22.489169Z

Thank you!

lorddoig 2017-06-07T12:16:19.788797Z

I’ve skimmed the source of xnio and see that it delegates to jboss for logging. Have also tried manually setting org.jboss.logging.provider to both log4j and slf4j—makes no difference.

2017-06-07T12:17:33.806336Z

I hate logging, btw :)

2017-06-07T12:18:01.813305Z

yeah, what's interesting is if you go with the default config, you don't see any messages from XNIO at startup

2017-06-07T12:18:21.818034Z

so something is shipping a config to suppress XNIO's INFO messages I think

2017-06-07T12:18:26.819144Z

but that's not the issue here

lorddoig 2017-06-07T12:18:51.824785Z

java isn’t my forte, java logging seems especially mystical

2017-06-07T12:19:11.829475Z

it's terrible

lorddoig 2017-06-07T12:19:24.832658Z

so I hear. and experience.

lorddoig 2017-06-07T12:20:36.849891Z

just noticed i’m on immutant 2.1.5, will try bumping to latest

2017-06-07T12:20:55.854264Z

I've already tried 2.1.7, same thing unfortunately

lorddoig 2017-06-07T12:21:11.857973Z

Indeed same thing

2017-06-07T12:21:29.862498Z

is the end goal here to get the log messages to a file that you set up via timbre?

lorddoig 2017-06-07T12:22:10.872410Z

It’s to get it into a timbre appender function so that I can log to file/stdout/kibana and instrument error counts with riemann

2017-06-07T12:22:19.874949Z

gotcha

2017-06-07T12:23:17.888416Z

jboss-logging is supposed to recognize that slf4j is available, and designate to that, but it looks like that's not happening

lorddoig 2017-06-07T12:23:49.895946Z

well interestingly, I do get this coming through timbre when I start the server:

INFO [org.projectodd.wunderboss.web.Web:145] - Registered web context /

2017-06-07T12:24:19.903348Z

yeah, that's from wunderboss, which doesn't use jboss-logging, and writes directly to slf4j

lorddoig 2017-06-07T12:24:25.904763Z

aha

lorddoig 2017-06-07T12:25:39.923462Z

It’s odd that when I manually set jboss to use slf4j (have checked it is set properly) it doesn’t make a difference. Perhaps it’s not detecting log4j/slf4j but, in the absence of logback, is just writing directly to stdout

lorddoig 2017-06-07T12:25:58.927883Z

But when I include log4j I get errors about multiple bindings and a warning about a missing config file

2017-06-07T12:26:13.931414Z

how are you setting jboss to use slf4j? by setting org.jboss.logging.provider?

lorddoig 2017-06-07T12:26:22.933831Z

Yeah

2017-06-07T12:26:30.935680Z

dangit, I was just about to try that

lorddoig 2017-06-07T12:27:44.953758Z

😞

2017-06-07T12:39:10.130078Z

I'm at a loss - @jcrossley3 should be in soon, he may have some more insight

lorddoig 2017-06-07T12:49:24.295834Z

Thanks for your help dude

2017-06-07T12:49:35.298656Z

I'm stepping through jboss-logging in a debugger now

2017-06-07T12:49:46.301563Z

so I guess I'm not completely lost yet :)

lorddoig 2017-06-07T12:50:08.307780Z

awesome 😀

2017-06-07T12:55:05.392920Z

weird. it's failing because something in clojure-land is throwing an AssertionError: "Assert failed: (nil? arg-array)", but with no stacktrace

lorddoig 2017-06-07T12:57:23.435055Z

hmmmm that is odd. elide asserts?

2017-06-07T12:58:08.448843Z

i have no insight. i always defer to James Perkins for jboss logging problems.

2017-06-07T12:58:35.457206Z

I suspect it's a pre-condition on a fn somewhere, can those be suppressed?

lorddoig 2017-06-07T13:01:30.514943Z

(set! *assert* false) i think

lorddoig 2017-06-07T13:06:47.617404Z

although we’re having trouble with that…

lorddoig 2017-06-07T13:07:07.623555Z

if you’re using lein you can do it via :global-vars in project.clj

2017-06-07T13:08:41.653012Z

in a meeting atm, but will look after

2017-06-07T13:40:53.321725Z

@lorddoig I'm back. It looks like this the culprit: https://github.com/fzakaria/slf4j-timbre/tree/master/src/slf4j_timbre/adapter.clj#L95

2017-06-07T13:41:22.332381Z

it doesn't expect (and doesn't allow) the .log method to be called with an arg array, and that's happening somewhere

lorddoig 2017-06-07T13:42:06.348455Z

Aha. But would that explain why the xnio logs are diverting to stdout?

2017-06-07T13:43:51.387750Z

I think so, since jboss-logging fails to init slf4j (that assert fails when jboss-logging makes a debugf call to the slf4j logger it found during init, and jboss-logging swallows the exception and assumes slf4j failed to init, and moves on to other logging methods)

2017-06-07T13:44:04.392566Z

I'm playing with it a bit more to confirm

lorddoig 2017-06-07T13:45:48.432527Z

Brilliant! I will investigate too

lorddoig 2017-06-07T13:46:16.443006Z

Hopefully we can just modify slf4j-timbre and be done. Happy days.

2017-06-07T13:49:58.528444Z

if I comment out that assert, the XNIO messages do make it to timbre

lorddoig 2017-06-07T13:50:36.543807Z

🙏 thank you so much

2017-06-07T13:51:14.559006Z

happy to help! though messages that come through that -log fn won't have the arg-array included in the message, it is just ignored

2017-06-07T13:51:33.566701Z

so this is something that should probably be fixed in slf4j-timbre for real at some point

lorddoig 2017-06-07T13:52:02.577950Z

I’ll fork for new and open an issue/PR. Unless you’re planning on doing that? you seem to know more than me

2017-06-07T13:52:12.581910Z

@tcrawley nice catch!

lorddoig 2017-06-07T13:52:31.589761Z

Indeed! It would have taken me a while to find that, if ever

lorddoig 2017-06-07T13:53:39.616170Z

The team here at http://style.com are extremely grateful. Thanks so much guys.

2017-06-07T13:55:14.653481Z

@lorddoig you're welcome to file the issue and fix it if you like, and I'm happy to assist.

2017-06-07T13:55:31.660639Z

and it's our pleasure!

lorddoig 2017-06-07T13:56:09.675916Z

I’ll give you a nod on GH, is your username the same?

2017-06-07T13:56:22.680980Z

no, I'm tobias on gh

lorddoig 2017-06-07T13:56:29.683735Z

:thumbsup:

lorddoig 2017-06-07T14:49:37.016022Z

@tcrawley I’ve just tried commenting that assert out but I’m seeing the same thing happening. What deps are you using?

lorddoig 2017-06-07T14:54:26.140619Z

Scratch that, got it working by setting org.jboss.logging.provider=slf4j

2017-06-07T14:54:50.151187Z

ah, good deal. I was just checking my deps :)

lorddoig 2017-06-07T14:55:41.173763Z

Finally. I’ve tried to get this working on and off for about 18 months now. Thanks again, just about to file an issue

2017-06-07T14:56:27.194579Z

my pleasure!

lorddoig 2017-06-07T15:01:55.345541Z

https://github.com/fzakaria/slf4j-timbre/issues/25 for reference

2017-06-07T15:03:05.376542Z

right on