Clojurians Log v2

What’s the recommended way to turn Clojure into EDN? I ran into some issues with pr-str. It works fine with pure data, but less so if data contains lazy seqs. I tried to make naive edn-response middleware for ring. I gave it some lazy seqs (a map containing something that had passed through for ) and this resulted in a string of log messages bundled up with the stringified data :thinking_face: Interestingly, the ring-json-middleware does not have this problem, as cheshire/generate-string does not use printing. Is there a way to produce EDN data that isn’t based on printing, or am I barking up the wrong tree?

https://clojuredocs.org/clojure.core/print-method there is a way to adjust printing for some data based on type in general there is nothing wrong with using pr-str to produce edn. cheshire is not using printing because the serialized form of data not generaly compatible with json specification

I just tried to reproduce and it works just fine for the case like that:

(def x (for [i (range 100)] i))

  (type x)

  (pr-str x)

The problem is lazy sequences that do side effects, like logging. When creating a generic tool (like an edn-middleware for Ring), you have no control over what crazy shenanigans the lazy data structures you’re passed will do. So, using pr-str to generate EDN, might include log statements printed to `*out*` .

Yes, logging is my exact problem 🙂

This reproduces it:

(pr-str
 (for [i (range 10)]
   (do
     (prn i)
     i)))

;;=> "(0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n0 1 2 3 4 5 6 7 8 9)"

Does anyone have a snippet that enables http/2 on jetty without using info.sunng/ring-jetty9-adapter

Jetty's official docs are very "xml focused", which does not make it easy

and i really don't want to stray off the "official" path to get something like this working

It's impossible to serialize a lazy-seq without realizing it, unless you don't want to serialize the data contained in it?

And so realizing it will always trigger the side effect

But to get the serialized string without the printed stuff that should be possible. Hum... Let me check

Right, you can use pr-on instead

Though that seems to be a private fn

There's an issue here: https://ask.clojure.org/index.php/2791/captures-stdout-printing-effects-lazily-evaluated-expressions

You should upvote the issue if you want it fixed, the core team does take into account votes for priorities

I'd also recommend upvoting this one: https://ask.clojure.org/index.php/1905/there-should-also-be-writing-in-clojure-edn

Hi Clojurians, I wrote a list multiplication program which multiplies two numbers represented as lists using standard multiplication algorithm taught in school. https://www.mycompiler.io/view/E5DB2XN is the first program I wrote. This program helped me pass a bunch of problems on Hackerrank, until it started giving me timeouts at https://www.hackerrank.com/contests/projecteuler/challenges/euler016/problem project euler problem. So, I tried improving the functions with https://www.mycompiler.io/view/4owO1yk shorter but slower attempt. I am not able to figure out why the latter program is slower than the first one. Can anyone help?

In the new attempt there's this code:

(reduce (partial map +) ls)

Oh, ls is a list of lists, I see.

@p-himik Yes sir, that is list of products.

It's almost night here, so I'm rather lazy to go through the code right now. But at the very least, that particular part should be written as (apply map + ls).

This way, it won't create a (dec (count ls)) of lazy collections.

@p-himik yes, I already tried that. That was even slower. (apply map + ls)

I can't offer a more constructive advice right now other than "try profiling". Then you'll know for certain.

@p-himik thank you Sir. Appreciate that. I will look into profiling.

what's the state of core.typed right now? Is anyone using it after its renaissance? is it ready?

The author is interviewed fairly recently here: https://youtu.be/8A8EKTBlFRM which I watched when it came out but unfortunately I somehow don't remember what the summary of the current state is. I know he talks about what he's been working on and I believe has been actively working on core.typed but I'm not sure if those changes are available yet

Actually the changes I'm talking about are probably just the renaissance you refer to

(unrelated) maybe a malli thing might work well. Like, a subset of what malli can specify might be translatable to typedclojure

oh wow the video here is almost along those lines, just talking about spec

@p-himik I found the cause. It was map function retuning lazy sequences which delays the execution. I used mapv at two places and it made it twice faster. Thanks for putting me on the path and letting me explore this lazy sequence angle.

Is there as strong difference between wrapped/lookup-or-miss and wrapped/through-cache? (I observed that the former uses the non-wrapped through-cache so it might be a 'superset' of the functionality?) I tried to understand the difference but sadly in the repo wrapped/through-cache has no unit tests and a worse docstring than wrapped/lookup-or-miss so I'm confused about what the wrapped/through-cache fn actually does

ah good point. i think that's the one that can prevent needless spinning under contention

i'd do some experiements. spin up 30 threads hitting your cache and count the number of times something is computed

i think there was something like that in that article

Yeah I got that impression from the article and source I expect contention to be low/irrelevant so probably I'll sleep calmer with max 10 attempts

Curious, you include clojure.core.cache.wrapped/evict under the 'footgun' category But wanting to immediately overwrite a key with a different new value seems a quite extremely common use case? (`lookup-or-miss`/`through-cache` doesn't appear to help here, not without waiting for the cache entry to naturally expire) Maybe the mistake is mine for thinking of these caches as atoms (as opposed to things that can have disparate implementations)?

i think you really shouldn't call evict yourself. the cache has strategies to call evict. but i suppose if you wanted to override you could

yes I suppose that eviction is essential to all these caches, they have to perform eviction sooner or later :) but I suspect depending on the use case, waiting an indeterminate amount of time (or having dropped writes) is not what one wants. OTOH, I suppose that if one used evict recklessly one could increase contention or get nil reads

(btw I'm just testing out my own knowledge, and not really questioning anything)