Clojurians Log v2

thank you, it's fixed

@henryw374: Well done for taking the time to make this library. I feel like it does fill a hole. As others have mentioned I dislike timbre, as I’ve been burned by it pretty much everytime I’ve seen it included in a project (maybe 2 or 3x). We’ve standardised most of our apps on log4j2 via SLF4j and clojure.tools.logging. I 💯 % agree with staying close to an established java logging system; and also think log4j2 is one of the best (if not the best) choice for a backend. As far as I’m concerned the benefits of using edn over xml to configure the logger are far outweighed by the practicalities. I’d also certainly like all logging to be data; but we also use SLF4j to capture and redirect logs from java libraries that use different logging frameworks. Unfortunately these can’t really be retro fitted to log data, so I’m not sure it makes much sense for us to switch to this… though I guess in principle we could have a clojure data log and send everything else to other non-data logs. Regardless, I just wanted to say, that knowing far more than I ever wanted to about the java logging eco-system, I agree with your analysis and approach here.

Thanks @rickmoynihan! To explain more about my own plans, I'm aiming for a gradual approach. I had a logback/clojure.tools.logging setup with log events coming out as json string in stdout, e.g. {level "INFO" message "foo" anMDCfield "yay"} - and that continues to work after switching the backend to log4j2, for both my application's log statements and those from 3rd party libs. now I'm adding/changing my application logging to use the log4j2 data api, and having the existing appender keep working - only now I'm optionally adding other data fields ofc. In addition, I've now got other appenders that work directly with the data - in some cases filtering events to those that are in the new 'data' style, ie not just containing message field.

Yeah that’s exactly what I was thinking. If I was starting again fresh I’d probably do that too, or if I needed to integrate something to datafy my logs. We have some custom datadog integrations that already essentially log data to datadog, however because we want to send data we currently do it outside of the logging system. If I were to do it again now I’d probably integrate it with this and implement it as an appender as you say; however it doesn’t feel like it’s worth the churn to change right now.

:thumbsup:

Incidentally one feature that I don’t think you have are log4j2 ThreadContexts. They’re very useful. We have a macro in some of our projects that use those to for example assign every incoming http-request a unique id, and associate it with every (asynchronous) action related to that request. If you want I can open an issue and paste the code for you to mull over including.

thanks yeah that's on my list. we have a similar macro that puts stuff in MDC context. so.. yes please to issue 🙂

ah is there a good reason why you’ve chosen the Apache license over EPL?

The original code was ported from a library @malcolmsparks wrote. It was dual licensed under EPL 1 or AGPL 3. Most clojure code tends to use EPL, as that’s the license clojure uses too. Apache/EPL may be compatible, I don’t know.

Hey Alex, do you know when the official docker images will get updated by any chance?

If you change to EPL you’ll be able to just use it. I’ll not supply the code incase you want to do a clean room implementation (as there’s not a lot to it). Though if we ask malcolm nicely he might be willing to grant you a waiver. It’s only a 15 line macro though, so most implementations will be more or less the same. Alternatively I could describe the interface / usage. Up to you.

hmm, yeah not my area of expertise I'm afraid. looking here, https://resources.whitesourcesoftware.com/blog-whitesource/top-10-eclipse-public-license-questions-answered it looks like they are not compatible - apache is more permissive. if you can post a gist for now with the license header that'd be ok? or, I'm supposed to be meeting Malcolm later so I can ask him. he uses MIT for all new stuff, which afaik is even more permissive than apache

Yeah MIT is pretty unencumbered, so you’d be ok in that case. I suspect malcolm will be happy to grant it, and just have the code used, but it’s totally his choice.

Of course if you don’t care strongly about the license you could also just move to EPL, MIT or public domain.

but it’ll probably only take you 10/20 mins to implement the macro yourself 🙂

I find it hard to say no to at least looking at working code. :) MIT wouldn't work, as that is less compatible. I'll ask him and ping you back thanks

Yes sorry MIT definitely won’t work for you using this code.

unless malcolm were to explicitly grant permission to you under the license you choose.

https://gist.github.com/RickMoynihan/8714a86f7f1f30b22136597305e50ca4

cheers

https://github.com/henryw374/clojure.log4j2/issues/1

No, I don’t have any connection to those

Thanks anyway, I'll try to find out who to nag, then 🙂

Here is a simple video encoding library for Clojure: https://github.com/cnuernber/avclj I wrote this as an example of how simple interfacing with C systems can be and to show how nice using the deps.edn pathway is for a clojure library. It sits directly on top of the libavcodec series of shared libraries used by ffmpeg - it loads them dynamically and then you pass frames directly to the library. I have included a simple example of using Skija to render to a native buffer that is then passed directly into the encode-frame! method. Use this if you have a need to render videos of your crazy Clojure pixel art 🙂.

Dear friends, a new much improved version of BinF, a Clojure/script library for handling any kind of binary formats /protocols in a simple yet powerful way. I've used it with great success for things like MIDI processing and a WebAssembly decompiler. I find it really capable, flexible, and feature-rich. It provides probably more than you need. Hope it will sprout new super use cases: https://github.com/helins/binf.cljc

Nice. I have a Node-hosted sysex librarian to finish. This could well be useful.

This looks great! I also nominate your READMEs description of your support for 64-bit integers for the "README Hall of Fame":

It is not the most beautiful experience one will encounter in the course of a lifetime but it works and does the job pretty efficiently.

This is really neat! Would it be possible to make a simple video player using avclj?

Yes it would be possible. The decode pathways in my opinion are much harder than the encode pathways but all the necessary pieces are there. I could provide a function that gives you a sequence of close-able buffer objects or something that had each frame's information such as the image, the microsecond time and duration. The waters are fairly deep with FFmpeg; the encoder took a lot of head scratching before it produced videos that worked on cellphones.

After a slight delay, https://clojure.org/releases/devchangelog#v1.11.0-alpha1 is now available! • https://clojure.atlassian.net/browse/CLJ-2603 Clojure keyword argument functions now also accept a map, see https://clojure.org/news/2021/03/18/apis-serving-people-and-programs

user=> (let [[& {:keys [a b] :as opts}] [{:a 1 :b 2}]] [a b opts])
[1 2 {:a 1, :b 2}]
user=> (let [[& {:keys [a b] :as opts}] [:a 1 :b 2]] [a b opts])
[1 2 {:a 1, :b 2}]

Compared to 1.10.3:

Clojure 1.10.3
user=> (let [[& {:keys [a b] :as opts}] [{:a 1 :b 2}]] [a b opts])
Execution error (IllegalArgumentException) at user/eval140 (REPL:1).
No value supplied for key: {:a 1, :b 2}
user=> (let [[& {:keys [a b] :as opts}] [:a 1 :b 2]] [a b opts])
[1 2 {:b 2, :a 1}]

So, no, it changes destructuring “everywhere”.

@zane You’re trying to destructuring a sequence as a map — the change in 1.11 is the other way round, effectively, destructuring a map as a sequence of named arguments.

I will update us to 1.11 Alpha 1 at work and see if it breaks anything!

I should also mention that the destructuring docs have also been updated (they actually never had been updated to include keyword argument support and there were no published semantics about that anywhere). Also, the doc has been extended to cover this new 1.11 capability https://clojure.org/reference/special_forms#keyword-arguments

FFmpeg is such a useful library, it's something I've been meaning to dive into myself. I'm familiar with jna and it would probably be good for me to also learn from how you've been using jna based off your excellent work with libpython-clj. I'll check it out and see how it goes.

wow, I did not know about this trick, https://github.com/cnuernber/avclj/blob/master/cpptest/compile64.sh#L1 Very cool. I've been using python-clang to try to get struct info.

🤯 , this stuff is awesome, https://cnuernber.github.io/dtype-next/tech.v3.datatype.ffi.html. Is the ffi stuff graalvm compatible?

This seems like a clever little change though to be honest I don't like the kw-args approach, I'm always happy enough to just use an optional map parameter on my functions. A lot of confusion to save two characters. It's feels like a rare example of Clojure doing something "to be cool looking" (the kw args in the first place, not this new change).

Also, I think the example in the release notes would be slightly more useful if it showed a mix of kw args and a map.

Not at this time. I am working on it right now. It is JNA and JDK-16 compatible.

Well, either way, there's some really great c interop code here and I look forward to simplifying some of my c interop code in the future. Thanks! :thumbsup:

You are so welcome! I will post of course when I get a graal native pathway working. The target is some clojure code along with some python stuff all wrapped up into a graal native shared library that is then loaded via cython.

Eventually, I'd like to be able to spit out apps a shared library that can be loaded and ran on mobile using a prebuilt iOS/android app

Is there a performance difference between using the new & and no & at all? (i.e. a vanilla, fixed map arg)

Then I shall accept the honor!