Clojurians Log v2
announcements
Want to easily tap> those let bindings? Please give taplet a try. It has a macro named let> that you can use instead of let that will tap the binding box. It is WIP and I would love me some feedback. Read a bit more about it at ClojureVerse: https://clojureverse.org/t/lets-tap-with-let-a-k-a-my-first-macro-taplet/7361
I am announcing the Girouette CSS library. • Specifically designed to be as easy to customize as possible. • Purely functional, no side effect or global registry. • Written in CLJC, runs everywhere, suitable for virtually any use case or Clojure tech stack integration. • Has grammar rules which, when used, provides a full compatibility with all of Tailwind's CSS classes. • Its source code processor finds the CSS classes your program is using. • Generates and updates a CSS file while you are developing. • Does basically the same as Tailwind-JIT, but in pure Clojure. Github project: https://github.com/green-coder/girouette Video presentation: https://www.youtube.com/watch?v=Tnv6SvZM6tg Slides: https://app.pitch.com/app/public/presentation/7282e9fa-8789-43bc-8b2d-eaec38711d98 Short GIF demo: https://twitter.com/VincentCantin/status/1365980792325726212
Bravo Vincent ! Little typo, "Written in CLJC, run everywhere" instead CLJS.
Thank you 🙂 Typo fixed.
Ah, I see. That confused me. I thought there were some magic involved. 😃
A great stuff, thank you! And another typo > Girouette also make it easy to use you own grammar rules Should likely be "your"?
thank you, it's fixed
People interested to talk about the project can join the #girouette channel.
Monthly update of the practicalli/clojure-deps-edn project containing a wide range of community tools for Clojure CLI (deps.edn) projects
Monthly library version update via the alias `:project/outdated` which uses antq project. The full changes are documented in the change.log.
Highlights include:
• new :service/webserver zero-config command line HTTP server using the nasus project
• new domain names for antq, depstar and next.jdbc (reported by antq 0.12.0)
• com.github.liquidz/antq - version 0.12.0
• cider/cider-nrepl - version 0.25.9
• djblue/portal - version 0.10.0
• lambdaisland/kaocha - version 1.0.829
• org.clojure/data.json - version 1.1.0
• seancorfield/clj-new - version 1.1.264
• com.github.seancorfield/depstar - version 2.0.193
• com.github.seancorfield/next.jdb - version 1.1.646
• vlaaad/reveal - version 1.3.196
https://github.com/practicalli/clojure-deps-edn
Announcing clods - a (niche) health and care library/microservice/HL7 FHIR server for organisational reference data in the UK. Find local hospitals or GP surgeries within a range of any UK postal code, or track organisational change over time. https://github.com/wardle/clods Designed to be embedded in larger services, requiring only a local filesystem to operate, but can also run as a standalone server if needed, including a FHIR R4 option if required.
@jr0cket clj-new has also started to be published under a new domain com.github.seancorfield (as well as the old one). All of my libs will be double-published for a while, to allow folks to switch over gradually.
new minimal clojure wrapper over log4j2: https://github.com/henryw374/clojure.log4j2 My requirement was to be able to log data - meaning logging that data into a db, without going via string representation, whilst keeping all existing java logging config working. As far as I could see the only thing capable of that is log4j2 and although clojure.tools.logging can use log4j2, it stringifies messages before they get to the implementation
@henryw374: Well done for taking the time to make this library.
I feel like it does fill a hole.
As others have mentioned I dislike timbre, as I’ve been burned by it pretty much everytime I’ve seen it included in a project (maybe 2 or 3x).
We’ve standardised most of our apps on log4j2 via SLF4j and clojure.tools.logging.
I 💯 % agree with staying close to an established java logging system; and also think log4j2 is one of the best (if not the best) choice for a backend.
As far as I’m concerned the benefits of using edn over xml to configure the logger are far outweighed by the practicalities.
I’d also certainly like all logging to be data; but we also use SLF4j to capture and redirect logs from java libraries that use different logging frameworks.
Unfortunately these can’t really be retro fitted to log data, so I’m not sure it makes much sense for us to switch to this… though I guess in principle we could have a clojure data log and send everything else to other non-data logs.
Regardless, I just wanted to say, that knowing far more than I ever wanted to about the java logging eco-system, I agree with your analysis and approach here.
Thanks @rickmoynihan!
To explain more about my own plans, I'm aiming for a gradual approach. I had a logback/clojure.tools.logging setup with log events coming out as json string in stdout, e.g. {level "INFO" message "foo" anMDCfield "yay"} - and that continues to work after switching the backend to log4j2, for both my application's log statements and those from 3rd party libs. now I'm adding/changing my application logging to use the log4j2 data api, and having the existing appender keep working - only now I'm optionally adding other data fields ofc. In addition, I've now got other appenders that work directly with the data - in some cases filtering events to those that are in the new 'data' style, ie not just containing message field.
Yeah that’s exactly what I was thinking. If I was starting again fresh I’d probably do that too, or if I needed to integrate something to datafy my logs. We have some custom datadog integrations that already essentially log data to datadog, however because we want to send data we currently do it outside of the logging system. If I were to do it again now I’d probably integrate it with this and implement it as an appender as you say; however it doesn’t feel like it’s worth the churn to change right now.
:thumbsup:
Incidentally one feature that I don’t think you have are log4j2 ThreadContexts. They’re very useful. We have a macro in some of our projects that use those to for example assign every incoming http-request a unique id, and associate it with every (asynchronous) action related to that request. If you want I can open an issue and paste the code for you to mull over including.
thanks yeah that's on my list. we have a similar macro that puts stuff in MDC context. so.. yes please to issue 🙂
ah is there a good reason why you’ve chosen the Apache license over EPL?
The original code was ported from a library @malcolmsparks wrote. It was dual licensed under EPL 1 or AGPL 3. Most clojure code tends to use EPL, as that’s the license clojure uses too. Apache/EPL may be compatible, I don’t know.
If you change to EPL you’ll be able to just use it. I’ll not supply the code incase you want to do a clean room implementation (as there’s not a lot to it). Though if we ask malcolm nicely he might be willing to grant you a waiver. It’s only a 15 line macro though, so most implementations will be more or less the same. Alternatively I could describe the interface / usage. Up to you.
hmm, yeah not my area of expertise I'm afraid. looking here, https://resources.whitesourcesoftware.com/blog-whitesource/top-10-eclipse-public-license-questions-answered it looks like they are not compatible - apache is more permissive. if you can post a gist for now with the license header that'd be ok? or, I'm supposed to be meeting Malcolm later so I can ask him. he uses MIT for all new stuff, which afaik is even more permissive than apache
Yeah MIT is pretty unencumbered, so you’d be ok in that case. I suspect malcolm will be happy to grant it, and just have the code used, but it’s totally his choice.
Of course if you don’t care strongly about the license you could also just move to EPL, MIT or public domain.
but it’ll probably only take you 10/20 mins to implement the macro yourself 🙂
I find it hard to say no to at least looking at working code. :) MIT wouldn't work, as that is less compatible. I'll ask him and ping you back thanks
Yes sorry MIT definitely won’t work for you using this code.
unless malcolm were to explicitly grant permission to you under the license you choose.
https://gist.github.com/RickMoynihan/8714a86f7f1f30b22136597305e50ca4
cheers
In case you weren’t aware, you can do this kind of thing with timbre ( https://github.com/ptaoussanis/timbre ). It allows you to send the logging info to a clojure function, which you can then update to do whatever you want with the structured data. built in postgres appender (which would need modified to not force a string): https://github.com/ptaoussanis/timbre/blob/master/src/taoensso/timbre/appenders/3rd_party/postgresql.clj and here’s an example of an appender I wrote to send structured json data to insightops (formerly logentries), which is probably a decent model for how you might want to send data over to your DB https://github.com/yummly/logentries-timbre-appender/blob/master/src/yummly/logentries_timbre_appender.clj#L55
to get it to work with log4j, in case you already have java libraries using that and you need to capture those logs as well, you’d need https://github.com/fzakaria/slf4j-timbre as the log4j appender
Thanks.. if you see in the references there is a clojureverse discussion where people mention scaling issues with timbre, so that put me off
ah, interesting. We use timbre in all of our clojure services at Yummly and have never noticed any scaling issues. We’ve overloaded the underlying insightops logger before, but never had issues with timbre itself
I wonder what those issues were (don’t see a link in the clojureverse explicitly talking about it, but I might just be overlooking it)
Using antq picked up the change in depstar and next.jdbc, but hasn't reported the change for clj-new. I'll do a little experimenting as to why and update clj-new soon anyway. Thanks. Great to have this approach being adopted.
Yeah I don't know.... It may just be FUD. Also ppl talk there about dependency hell probs with it. It might be great now I don't know. Sticking with log4j2 seemed like safe bet.. big community etc
yep! definitely can’t hurt sticking closer to the raw log4j2. Just wanted to make sure you were aware of the other existing options!
Thanks @tanzoniteblack, much appreciated!
my experience with timbre + slf4j-timbre has been mixed. essentially timbre is a library that, in its own words, rejects the whole java ecosystem; that kind of makes sense but at scale, inevitably one has to deal with arbitrary Java dependencies that do assume slf4j and friends. So delegating such a crucial functionality to an unofficial 3rd-party lib is not exactly ideal. I'm happy clojure.log4j2 exists; thanks Henry! Will keep an eye on it
μ/log is another logging library that logs data rather than strings, though it does not aim to wrap or target Log4J. The design notes on its internals mention considerations (among others) for speed and having few dependencies, so you may find it interesting if these matters are of concern. https://github.com/BrunoBonacci/mulog
If you want to log data with log4j2 you, can use the JSON Layout. Right now we are using ZMQ appender with JSON layout, and then a simple subscriber thread on clojure part that sends the logs to Loki. (I know, an other option is create an appender, but this solution was very easy to setup). If you wan't code references just ping me.
Thanks @steffan for mulog info - I did see that one before but I was looking for full integration with existing log setup ideally. Also, when you have a well established java library, I feel clojure projects would do well to explain what pros/cons they have in comparison. Having read the readme and now watched the video, the appenders, context info & api seem basically the same. the trace bit of mulog looks nice, but would just be one simple macro to add? on log4j2 pluses, I have docs, stackoverflow-ability, blogs, recommendations etc etc re:performance - I can't see in mulog readme what tests have been done vs log4j2 or others
@niwinz - it seems getting data via a layout is a bit of hacky though? and writing appender seems straightforward... got one in progress for influx for example https://github.com/henryw374/influx-log4j2-appender/blob/master/src/com/widdindustries/InfluxDbProvider.clj thought I'd have to AOT it but no