Clojurians Log v2

Yes you can teach it not to try to coerce the value, by returning :exoscale.coax/invalid, in that case it will just leave the value passed untouched and you can let it blow up when you validate. In general it is good to add some checks on what the expected input is in the coercer fn, like cond string? etc etc... (about invalid, it's useful to differentiate know about impossible coercion internally sometimes, like with s/or. coax tries hard to never do any s/valid? call to do coercion, it does just one thing and with as little "code dependency" as possible, so we must have this possible return value on coercers otherwise I cannot differentiate between a coercer that just returns the input untouched because it's wrong or because it was acceptable in current form).

You might want to check the default coercers for examples https://github.com/exoscale/coax/blob/master/src/exoscale/coax/coercer.cljc

Also on the repl be aware of coercers caching (we cache the coercer fn returned for a spec form/key, so you pay for spec parsing only once per spec) , you ll have to turn it off not to have surprises https://github.com/exoscale/coax#caching

I should put a big bold warning about this on the readme

I have a weird error in production sometimes. It’s an uberjar by Leiningen, and it’s a mix of .clj and .cljc files. Sometimes (but only sometimes!) after a deploy, the new instance will start reporting clojure.lang.Compiler$CompilerExceptions for a few forms in .cljc files. These are files that have been built and tested locally and on CI without error, and runs fine on the other instance. In other words, a moon-phase bug. 🙂 Restarting the app server (with the same code) always solves the problem. I don’t know how to even start to form a theory of what’s going on here. Here’s a sample excerpt of a stack trace (with names pseudonymized):

...
                                                                    clojure.core/require                       core.clj: 6007 (repeats 2 times)
                                                                      clojure.core/apply                       core.clj:  667
                                                                                     ...
                                                                  clojure.core/load-libs                       core.clj: 5969
                                                                  clojure.core/load-libs                       core.clj: 5985
                                                                      clojure.core/apply                       core.clj:  667
                                                                                     ...
                                                                   clojure.core/load-lib                       core.clj: 5928
                                                                   clojure.core/load-lib                       core.clj: 5947
                                                                clojure.core/load-lib/fn                       core.clj: 5948
                                                                   clojure.core/load-one                       core.clj: 5908
                                                                                     ...
                                                                       clojure.core/load                       core.clj: 6109
                                                                       clojure.core/load                       core.clj: 6125
                                                                    clojure.core/load/fn                       core.clj: 6126
                                                                                     ...
                                                                        foo.bar/eval3949                       bar.cljc:    1
                                                        foo.bar/eval3949/loading--auto--                       bar.cljc:    1
                                                                                     ...
                                                                    clojure.core/require                       core.clj: 6007 (repeats 2 times)
                                                                      clojure.core/apply                       core.clj:  667
                                                                                     ...
                                                                  clojure.core/load-libs                       core.clj: 5969
                                                                  clojure.core/load-libs                       core.clj: 5985
                                                                      clojure.core/apply                       core.clj:  667
                    java.lang.Exception: namespace 'foo.xyzzy' not found
2020-12-15T17:38:09.900000+00:00 i-002af1a1b62baa6d3 clojure.lang.Compiler$CompilerException: Syntax error compiling at (foo/bar.cljc:1:1).
    data: {:clojure.error/phase :compile-syntax-check,
           :clojure.error/line 1,
           :clojure.error/column 1,
           :clojure.error/source
           "foo/bar.cljc"}

the first thing I'd look for is def calls with code that might throw exceptions

in the namespace that isn't found

also, be sure you don't have any code that uses a namespace without requiring it

I had no idea there were so many

Thanks, @noisesmith! Yeah, I’ve checked those things. No side-effecting defs, and all namespaces are required before use.

I think it might be a concurrency issue. I noticed it happens when two web requests are processing concurrently on the same instance, just after restart. There is a dynamic require for a view (the .cljc file) in there. I don’t know much about the inner workings of require, but might it be possible that some Vars that are being (re-)defined in one thread are “invisible” to other threads? The first CompilerException I get after a new instance launches is always with an error like this in the dynamic require call: “java.lang.RuntimeException: No such var: xyzzy/some-var”

tree src/api/logs
src/api/logs
├── CustomLayout.java
├── CustomMessage.java
└── core.clj

https://github.com/ztellman/virgil

Although it's designed to work with Leiningen and Boot, you can still use it in any Clojure project - you just have to write some code for it. In my case, most of that code was just copied from virgil itself and then changed a bit.

thank you

But it is strange there is no tool for that already. Maybe separate project is the right call then hmm.

What about all dependencies and versions then between 2 files java project and whole deps.edn project?

It will become conceptually simpler but harder to implement, support, and use. And the hot reloading capability will be gone.

on the other hand I don’t want to maintain my own personal virgil 🙂 It looks I will lose whatever path I will choose 🙂

It's not "your personal virgil" - I just copied 30 lines from it. And even those were because I'm using a newer JVM that's somehow incompatible with the way virgil does things (I had to change literally one line) and I wanted to provide an explicit set of Java files (virgil searches for Java files itself).

BTW "I don’t want to maintain my own personal virgil" is a great answer for why "there is no tool for that already" ;)

Hi, I’m trying to call Clojure from Java/Kotlin and a function requires a map. Is there a list/place where the interfaces for these objects are defined?

FWIW I just use Clojure's source code as such a place.

Do you by any chance know if you can directly use the Class clojure itself uses? or is it common to define your own implementation to the interface?

I have seen both. Depends on what you need. If you just need a Clojure map then reusing the existing classes makes sense. If you need custom behavior then a custom implementation makes sense.

You can, probably, but I think you won't need to use PersistentHashMap just use whatever Map you use in Java and Clojure will work its magic when you call the function

sounds good, thanks 🙂 !

there is another alternative that is working smoothly with clojure cli https://github.com/EwenG/badigeon

(ns ops.java
  (:refer-clojure :exclude [compile])
  (:require [badigeon.javac :as javac]
            [badigeon.classpath :as classpath]))

(defn compile [{:keys [source-path output-path aliases]
                :or {aliases []}}]
  (javac/javac source-path
               {:compile-path output-path
                :javac-options ["-cp" (classpath/make-classpath {:aliases aliases})
                                "-target" "8" "-source" "8"
                                "-Xlint:-options"]}))

:compile-java {:extra-deps {io.xapix/ops {:local/root "../ops/"}}
                          :exec-fn ops.java/compile
                          :exec-args {:source-path "src/java"
                                      :output-path "target/classes"}}

Clojure implements Java native interfaces, such as Map, Iterable and Callable. So most of the time if you want use any interfaces of the java.lang package, Clojure implements it. Clojure runtime source code it's extremely simple to read, so you just directly read the implementation to understand how it works. https://github.com/clojure/clojure/tree/master/src/jvm/clojure/lang Also, Clojure has some shortcuts for the Java Reflection API, you could just call ancertors in a class in order to see its parent classes and implemented interfaces. For instance (ancestors (class {}))gives me this:

#{clojure.lang.IPersistentCollection clojure.lang.MapEquivalence clojure.lang.IMeta clojure.lang.ILookup clojure.lang.IEditableCollection java.util.Map clojure.lang.IMapIterable java.lang.Iterable java.util.concurrent.Callable java.io.Serializable clojure.lang.IFn java.lang.Object clojure.lang.IKVReduce clojure.lang.Seqable java.lang.Runnable clojure.lang.IHashEq clojure.lang.Associative clojure.lang.Counted clojure.lang.AFn clojure.lang.APersistentMap clojure.lang.IObj clojure.lang.IPersistentMap}

ah thats a nice trick to know!

thank you, I will take a look on this

package api.logs;

import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.PluginAttribute;
import org.apache.logging.log4j.core.config.plugins.PluginFactory;
import org.apache.logging.log4j.core.layout.AbstractStringLayout;

import java.nio.charset.Charset;

@Plugin(name = "CustomLayout", category = "Core", elementType = "layout", printObject = true)
public class CustomLayout extends AbstractStringLayout {

    private static final String DEFAULT_EOL = "\r\n";

    protected CustomLayout(Charset charset) {
        super(charset);
    }

    @PluginFactory
    public static CustomLayout createLayout(@PluginAttribute(value = "charset", defaultString = "UTF-8") Charset charset) {
        return new CustomLayout(charset);
    }

    @Override
    public String toSerializable(LogEvent logEvent) {
        return logEvent.getMessage().getFormattedMessage() + DEFAULT_EOL;
    }
}

imho, doesn’t worth a try) small helper like that is better to delegate to java

If that was a simple class, I would've definitely used proxy. But since it uses Plugin, it can end up being arbitrarily complex. But I have no idea how Java annotations work under the hood - maybe it's something that you can do with proxy or something else as well.

require is known not to be thread safe. https://ask.clojure.org/index.php/9893/require-is-not-thread-safe

The discussion at that link contains some approaches to mitigate this property of require

thank you for feedback, I was looking confirmation it is bad idea to do java->clojure for this 🙂

You can invoke the Clojure.read() function to read a literal map or Clojure.var() to look up Clojure vars and invoke them

Do you know master of Java interops in Clojurians? 😉

Ah, this is exactly what I needed. Much obliged!

ech there are still issues around this, logging in structured JSON format in Clojure is a pain 🙂

has anyone tried Clojure 1.10.2-rc1? Any feedback good/bad?

All tests pass in our biggest Clojure service, no issues whatsoever

Will probably ship it to production next week

:thumbsup:

not to steal sean's thunder but we've got it in production on one service that got deployed yesterday, it should go out to the other services whenever they are next deployed

cool, thx

been using it in our test environment, nothing unusual to report.

anything usual?

yes, it works 🙂

good :)

Been using it in one of the apps I work on (not yet in prod, though) — everything’s been working as expected so far.

What hiredman said 🙂 We're on Alpha 4 for all our other production services so the differences are minimal for us.

(also as a comparative data point: we have 1.10.1.754 CLI on our production servers and 1.10.1.763 on dev and QA so that will also go to production with our next full deployment)