Clojurians Log v2

hello, newbie here 🙂 quick question about developing a restapi server with clojure. I’m currently using http-kit and compojure, and I wonder if there is any clojure-specific conventional structure for api server? Typically with Java (springboot, for example), I’m accustomed to dividing the project into controller, service, and dao. I’ve seen some github projects in clojure that did almost the same; but I wonder if there’s any better way (or clojure way) of doing so.

@tlonist.sang The only real guidance in Clojure is to try to keep side-effecting code from pure code but folks organize their web apps in all sorts of different ways, depending on which approach to design they follow.

You won't find things like DAOs really because we have no objects 🙂 but for basic CRUD most folks will just traffic in hash maps going in and out of the DB via next.jdbc (which supersedes clojure.java.jdbc).

Separating your pure business logic out from both the HTTP side of the house and the persistence side of the house is fairly common.

@seancorfield Wow, from the author himself! I’ve been using next.jdbc throughout my project this year. Thanks for the guidance. I like how there is no set-rule!

If you've read Clojure Applied, I suspect that's going to be some of the best "guidance" you'll find. Perhaps a few of the Domain-Driven Development talks around Clojure (I haven't really followed that path but some people like it a lot).

is anyone using deps.edn on nix(os) and got jogl to work? it seems i can't for the life of it figure it out and could very much use any help 🙂

Good morning Clojurians, can anyone point me to a good way to handle passwords in an edn file? I do not want to push passwords into repo, is there a common way to handle it? For example in configuration like this:

:postal/email-config {:host  "<http://smtp.gmail.com|smtp.gmail.com>"
                        :email "<mailto:email@gmail.com|email@gmail.com>"
                        :pass "password"
                        :port 587
                        :tls true}}

Aero handles this kind of thing quite well, I think: https://github.com/juxt/aero#hide-passwords-in-local-private-files If you don't (want to) use Aero, you could just add a tagged literal (https://clojure.org/reference/reader#tagged_literals) that does something similar, I guess.

For GMail specifically, you should use an application token instead of the full email account credentials.

On that particular section of Aero documentation, see some discussion here: https://github.com/juxt/aero/issues/73

FWIW, Heroku uses this approach: https://12factor.net/config Some more in-depth description of the Heroku approach specifically: https://blog.heroku.com/twelve-factor-apps

@p-himik, assuming that the machine is secure and I am the only user of the machine, being able to read the env data is not a concern?

When a machine is indeed secure, it is not a concern. When a machine is not secure, everything is a concern - you simply cannot store sensitive information there, at all.

I am looking for a way to check whether two huge maps (around 600K entries) are equal. The issue is that some of the values are of type org.joda.time.DateTime and the values are not always equal, although they refer to the same time. I would like to find an implementation of a function like clojure.data/diff that allows to customize the definition of equality. I could use clojure.data/diff but the performance is not good enough for my huge maps.

Maybe someone knows a lib out there that provides such a function. I’d prefer avoiding to write an implementation of my own, if possible

OK, this makes sense.

One possibility would be to replace all occurrences of org.joda.time.DateTime in the two maps you want to compare with a new type of value that does return true for clojure.core/= when you compare them, and then compare those maps using clojure.core/=

or clojure.data/diff

Or if there is some kind of method you can call on instances of org.joda.time.DateTime that returns a "canonical" value for each instance, such that the canonical values are clojure.core/= to each other when they are the same time, then you don't need a different type.

An example of the "wrapper type" approach would be to create a custom Clojure deftype, and define your own custom equiv method (which is what clojure.core/=calls under the JVM hood) that calls the isEquals method of joda.time instead of equals: https://www.joda.org/joda-time/apidocs/org/joda/time/base/AbstractInstant.html#isEqual-org.joda.time.ReadableInstant-

@andy.fingerhut Is equiv a method of a protocol in Clojure?

I know that in ClojureScript it is

A question about reading Clojure docstrings:

(dotimes bindings & body)

bindings => name n
 Repeatedly executes body (presumably for side-effects) with name
bound to integers from 0 through n-1.

I always have to look on http://clojuredocs.org to see an example usage for functions like this, I find the dosctring doesn’t give me enough explanation to know how to write the form.

It is not in Clojure, no. ClojureScript was implemented years after Clojure, and some have asked whether going back to reimplement Clojure using protocols, similar to how ClojureScript does, would be useful, but it would be a big churn in the code base for probably not many advantages, and would break any libraries that rely on the Java class structure in Clojure's implementation today.

don't know where it is documented but bindings are vectors yes

For dotimes, it just isn't documented

> (doc let)
-------------------------
clojure.core/let
  (let [bindings*] exprs*)
([bindings & body])
Special Form
  binding => binding-form init-expr

  Evaluates the exprs in a lexical context in which the symbols in
  the binding-forms are bound to their respective init-exprs or parts
  therein.

  Please see <http://clojure.org/special_forms#let>
Spec
  args: (cat :bindings :clojure.core.specs.alpha/bindings :body (* any?))
  ret: any?

More common stuff like let has Specs for this purpose

Note: I haven't actually tried writing the deftype that defines its own equiv myself, and could be forgetting some nuances, but I can point you at existing libraries that do this successfully, e.g. priority-map https://github.com/clojure/data.priority-map/blob/master/src/main/clojure/clojure/data/priority_map.clj#L325

is there some built-in utility to "parse" error traces of type clojure.lang.ExceptionInfo ?

ordered-map: https://github.com/clj-commons/ordered/blob/master/src/flatland/ordered/map.clj#L36

It's just a normal exception? https://docs.oracle.com/javase/8/docs/api/java/lang/Throwable.html#getStackTrace--

Hmmm, looking at the definition of clojure.lang.Utils/equiv again to remind myself, it looks like overriding equiv will work if your type implements the IPersistentCollection interface, but otherwise your custom type would need to override equals

there is also Throwable->map to convert exception to data

and clojure.main/ex-triage to analyze that exception

not sure it's necessarily just a normal exception. clj-http generates this whenever a request bounces, for example: clojure.lang.ExceptionInfo: clj-http: status 401 {RESPONSE BODY HERE}

i'd just like to parse the response body directly

perhaps it's just (.getMessage ex) then

at least that should give me status 401 or the body

Hopefully ex-data gives you the body

I thought you wanted to traverse the stack trace

clj-http seems to use Slingshot, so I don't know whether ex-data works like I would like but yeah at least ex-message/`.getMessage` gets you something

In a Clojure project, it doesn't matter if I AOT with Java 11 or Java 8, because the Clojure compiler only ever emits Java 8 compatible bytecode, is this correct? I understand it matters for Java classes but not for compilation of Clojure units, true?

the reflector might choose methods that exist in java 11 that do not exist in java 8 if you build in 11 and then run in 8 right?

True, but then you're programming against JDK 11 specific methods anyway, so AOT isn't the culprit

i thought there was some "first acceptable method" type thing to the reflector as well. Doesn't that mean it could emit a call to something that won't exist in java 8? Even though you aren't explicitly using methods from java 11?

Hmm.

(I'm way out of my depth here so perhaps I'm stretching, but i bet the nature of any issues that pop up will be quite annoying to diagnose, so perhaps stretching is called for)

At least theoretically they could add some overload Object get(Object) when previously there was only Object get(long) and now you are linked to the new one without reflection whereas if you AOT in Java 8 you get a reflective call

i suspect ghadi or alex would have some experience in this if they see this conversation

javac can mess this up too: > Note: Merely setting the target option does not guarantee that your code actually runs on a JRE with the specified version. The pitfall is unintended usage of APIs that only exist in later JREs which would make your code fail at runtime with a linkage error. To avoid this issue, you can either configure the compiler's boot classpath to match the target JRE or use the http://www.mojohaus.org/animal-sniffer/animal-sniffer-maven-plugin/ to verify your code doesn't use unintended APIs. In the same way, setting the source option does not guarantee that your code actually compiles on a JDK with the specified version. To compile your code with a specific JDK version, different than the one used to launch Maven, refer to the https://maven.apache.org/plugins/maven-compiler-plugin/examples/compile-using-different-jdk.html example. > https://maven.apache.org/plugins/maven-compiler-plugin/examples/set-compiler-source-and-target.html

@nilern in javac you can set --release

from JDK perspective, there is some danger of resolution using newer JDK methods if they are available in the JDK you are using

this happened with Clojure in fact when they added a new Collection method override in Java 10 (11? don't remember) that made existing Collection impls ambiguous

so tl;dr it's probably always better to use a jdk 8 if you target jdk 8 in your CI?

if you want it to be compatible with Java 8+ that is probably safest

you can actually tell javac to use a different version of the jdk, I guess you could run the Clojure compiler with a jvm using a bootclasspath set to an older jdk possibly

do you mean --release ?

I don't know what the modern affordance is, maybe that's it

but the Clojure compiler doesn't need / use javac right?

right

Hmm I see they added that https://www.mojohaus.org/animal-sniffer/

was just making an analogy

thanks y'all

--release is for javac?

Yes it is a javac option

I guess replaces the old source/target stuff

http://openjdk.java.net/jeps/247

somewhat related, and realise this is an open question, but what kind of JVM settings do people apply when running clojure code specifically - thinking of things like tuning GC, string compression..? are there any settings that are more relevant for clojure code compared to straight java..?

I think it may matter in rare cases like ByteBuffer.flip() method - I saw errors like this when an uberjar was compiled with Java 11 but run with Java 8: https://stackoverflow.com/questions/61267495/exception-in-thread-main-java-lang-nosuchmethoderror-java-nio-bytebuffer-flip

I guess there's no workaround for that or? (javac seems to have --release parameter for cases like this)

@dazld I think that’s going to be very dependent on the particular application you’re running.

Are you going to tune for response times, throughput, fast startup? etc.

Ah, I didn't notice the rest of the conversation in the main channel - so I suppose that this can become a problem and there's no workaround other than using an older java when building the uberjar

I suppose I had at the back of mind that clojure code might not look like typical java code, when it's been compiled, and as such the default settings might not be quite right. If that's not true, then that answers the question.

Clojure is designed so that the default settings should be sufficient

if you're running an app in production, you should probably think harder about which GC you're using and how it's tuned

but that will be application specific

gc throughput -> parallel collector gc latency -> G1 or ZGC speaking broadly

in terms of HotSpot/compiler - leave the defaults

The https://docs.oracle.com/en/java/javase/16/gctuning/introduction-garbage-collection-tuning.html#GUID-8A443184-7E07-4B71-9777-4F12947C8184 is A+

@dazld As a follow-up, specifically on Ghadi’s response, at work we’re on JDK 11 in production and using the G1 collector and that works great for interactive apps and services where latency is important.

@alexmiller @seancorfield @ghadi thanks a ton, that was all really helpful

I've only used the parallel collector for batch imports and such

Hello! Which symbols can be a word separators in Clojure(script) exept of all the types of brackets and spaces?

Editors default is ~!@#$%^&*()-=+[{]}\|;:'",.<>/?` but seems have to be excluded most of them

You can probably glean the answer from this page: https://clojure.org/reference/reader

Thanks

note that there are things that are explicitly allowed, things that are not allowed, and things left intentionally ambiguous for possible future expansion

Bummer!

Why a bummer? The same approach of defining your own deftype as a wrapper object for times, with your own custom definition of equals, should work, I think.

I'm just clarifying that the method you need to override is not equiv, but equals. clojure.core/= calls equiv on some objects, typically ones defined in Clojure itself, but falls back to Java equals for all others.

I’m watching Nada Amin’s “Programming should eat itself” Strange Loop talk and the concept of meta levels was interesting. Has anyone built something similar on top of Clojure?

you can start nested clojure.main/repl's from inside the Clojure repl - they can be set up to support exiting the nested repl, so you can get some small part of this. what she demonstrated is probably beyond that though

(talk: https://www.youtube.com/watch?v=SrKj4hYic5A for reference)

Is there a way to re-implment equals for DateTime or is it sealed?

Cool! I was just thinking how to launch nested repls:ok_hand::skin-tone-2:

Note that nREPL currently doesn’t support nested REPLs, though. Need to use socket REPL or prepl.

Right, I’ll start playing with those repls then

nesting clojure.main/repl's is also a neat way to support remote repls. a nested repl takes over the io streams until it exits, so for remote repls you do the same thing, but instead of taking over the io streams and running a repl, you take over the io streams and wire them to a repl running somewhere else

I'm kinda interested in everything related to nested REPLs. When would you do something like that instead of just starting a socket REPL in the remote process and connecting to that?

haven't watched the talk but https://github.com/gfredericks/debug-repl might be considered a nested one?

https://gist.github.com/hiredman/1789849d21be38310694dbf214d60d34 is a poc for interrupting prepl evaluations, but it shows kind of half of the remote repl thing, connecting and sending things, but not wiring up the current repls io to the remote repl

https://gist.github.com/hiredman/a2630ea6153d06840a2723d5b2c9698c let's you open a repl at a point in your program and "send it" to connect to your existing repl

i made a history saving repl recently. and i want to add support to saving history and annotating each subform with it's evaled result, similar to the CIDER debugger

made a question on http://ask.clojure.com for cool things done with subrepls: https://ask.clojure.org/index.php/10425/what-cool-things-do-you-do-with-clojure-main-repl

This is/was also one of those: https://github.com/technomancy/limit-break

that's awesome!

it even works with bb :) (that's how I discovered it, someone told me he used it with it)

This page of Java doc says that the class org.joda.time.DateTime is final, meaning no subclassing in Java: https://www.joda.org/joda-time/apidocs/org/joda/time/DateTime.html?is-external=true

And there is no way to override a method for an existing class in Java that I know of (others might). Wrapping the class in another type and delegating all methods but equals is always possible regardless of final or not.