Clojurians Log v2

Moin moin

დილა მშიდობის!

morening

👋

Morning

Good morning. And ❄️

good morning!

Good Morning!

Any pointers on this Monday morning regarding extracting all values for a specific key in a very nested map/vectors of maps?

Would a nice little recursive function work here?

Yup, I think so

treewalk 🙂

postwalk and collect into a seq

(let [xs (atom 0)] ((postwalk (fn [x] (when-let [v (:some-key x)] (swap! xs v]))) my-ds) (untested)

good morning

@ordnungswidrig you could use a transient or a volatile there as well right?

Sure, but I love to optimize only until the very end

(given that your xs doesn’t leave the call stack)

@ordnungswidrig in case of k-v pairs, postwalk goes through key first and value in the next, so not quite sure how to get value using key in this case 🥺

@chokheli on the way “up” it will once receive the map as a whole 🙂

Oh 😇

Maybe tree-seq also works

tree-seq is cool, if not only because of a blog post that I only vaguely remember because of the illustrations.

And the only way I find that blog is by searching for images related to clojure and tree-seq…

https://mwfogleman.github.io/posts/20-12-2014-flatcat.html

@ordnungswidrig have you ever made a studies of typical json payloads for liberator?

Reason I’m asking is that I’m working on speeding up data.json and it would be interesting to have an idea of “real world” json payloads.

With some typical payloads from work (shallow maps around 1k), I read json faster now with data.json than with both cheshire and jsonista, but as payloads grow in size and complexity, cheshire and jsonista are faster.

I didn’t. But most projects I know register cheshire to generate actual json reponse 🙂

Yeah, we do that at work too (I think) but I want to see how close I can get with a somewhat pure Clojure json lib so that I don’t have to use Jackson.

There is a startup cost to Jackson that is not insignificant, and then there is the dependency management with all libs having their own version of Jackson which is incompatible with every other version of Jackson.

having suffered the pain of having to resolve classpath clashes with jackson versions this sounds amazing.

Anyone recall why Rich Hickey (PBUH) doesn’t like pattern matching? I have a bunch of nested ifs I want to flatten and core.match seems like the right tool for the job.

@simongray I don’t think he minds pattern matching as such, but it’s a somewhat closed system.

I recently ran into an issues where Cheshire threw an exception on JSON responses coming from the metabase API. Worked fine with data.json.

So, imagine some type A which is the union(?) type of B, C, and D, so you’d have code like:

case a
 B "it's a b"
 C "it's a c"
 D "it's a d"

I see. So if you have a neatly delimited set of possible options, you’re ok?

And, I guess, unlike protocols and multimethods, consumers of your code are not free to extend A themselves, aka the expression problem

https://en.wikipedia.org/wiki/Expression_problem

in this case, I think core.match and nested ifs would be about equally extensible 😛

Somewhat related https://insideclojure.org/2015/04/27/poly-perf/

I have precisely 2*2*2=8 cases I need to handle and these cases are already occurring inside a couple of nested ifs, so I thought it was becoming a bit unreadable.

Nested if/else sounds like cond btw

sure, I could do cond too

probably should…

just annoyed me how I would have to make a let in that case and create another indentation level

🙂

core.match kinda merges the two

There is another lib, though, don’t remember quite the name, but it was sponsored by clojurists together which was doing somthing like core.match…

Morning

@slipset Meander?

exactly

There is also https://github.com/xapix-io/matchete which is a lot simpler, works with babashka and is data/function vs macro driven

Perf is probably not as good as meander as it doesn't try to optimize as heavily

Have you created an issue at cheshire? I'm curious what the issue was

About jackson: isn't the same true when including transit? This also depends on Jackson

no, didn't have time to dig into it. Just switched to data.json which we were already using elsewhere

Could be.

Haven’t used/looked into transit.

morning

Where does transit depend on Jackson?

transit-java?

Jupp

It seems jackson has a pretty sane way of dealing with breaking functionality: https://github.com/FasterXML/jackson/wiki/Jackson-Releases#general So I wonder what the problems with it are? I've never experienced those myself, but I hear some people mention it here and there

I worry about this from the perspective of babashka, which has both transit and cheshire that depend both on jackson (choices I made early on, they seemed legit choices from a community / track record perspective).

morning

:morning:

Morning!

@jasonbell https://github.com/lilactown/autonormal

@simongray Because bootstrapping comes out of my pocket and RDS is overpriced 🙂

Almost every project where I started with persisting EDN files ended up using postgres in the end

@jasonbell it was a joke based on the github repo I referenced 😉

Yeah but I’m the CEO/CTO/CFO/COO/CMO #“C[A-Z]O”

Ending up with postgres didn't have anything to do with management

More with consistency / corruption

if there was an EQL->SQL query generator then you could have a smooth migration path @jasonbell

@mccraigmccraig https://walkable.gitlab.io/

ooo, lovely!

nice!

everybody who says that databases avoid consistency and corruption hasn't done as many migration or DR jobs as I have 😉

🙂

just storing edn files on disk doesn't always improves the situation though ;)

nothing more fun than figuring out what proportion of the records have been trashed by a bad migration and then figuruing out how to fix them

@borkdude true, depends on what you are storing in the edn files

and how many processes/threads are reading from/writing to those

Single processes for me. Fairly slow moving.

and what needs to be shown when you are reading

and then we get into jepsen territory 😄

I recently learned that a postgres connection always maps to one OS process (not thread!)

so everything you do in a postgres extension is thread-safe by nature

Pattern matching is fine when there is a pattern.

🙂

And often it’s preferrable over nested conditionals

I had a clever use of pattern matching recently. And after some real world tests and checks and adding some resilience it all fall together into clever use of (first xs) (rest xs) and some defaulting and nil punning 🙂

that is a nice design. I hope it scales well 🙂

I think it's more because of ancient design and they can't change anything about this model anymore because so many people depend on it

so with 4000 connections this means 4000 os processes?

Maybe same for Python + GIL + C extensions

yes

This is a good reason to use a connection pool

not sure about modern os (read linux) but I think the context-switch impact is relevant

I'm not sure if mysql does this differently

I’m kinda lost, code runs from repl and produces the output files, but from main clojure -M:run-m doesn’t executes that last function :man-shrugging:

P.S. @ordnungswidrig thank you, postwalk did the job 🥷

@chokheli Did you try putting lots of println everywhere? It's my equivalent of "have you tried turning it off and on again" for clojure

@chokheli It could be a laziness issue. E.g. when you evaluate a lazy seq in the REPL, the printing forces it

Not many, just a few. Regarding laziness - I’m using juxt at the end of the previous function and I think I’m “evaluating it” it, perhaps not.

(defn f [coll]
((juxt f1 f2) coll)))

it depends on what f1 and f2 do

f1 at the end runs map over some collection and f2 returns a vector

map is lazy

maybe try mapv

nope, mapv didn’t change anything 👀 still not evaluating

but it does reach function f? Have you put a println there?

Oh, it works now, mapv after juxt did the job! Thank you!

a delightful afternoon of classpath hell

.cpcache doesn't seem to help much

@otfrom what version of tools.deps art thou using?

Anyone got a Raspbery Pi 64bit?

I have one

spinning it up now

whatever comes with Clojure CLI version 1.10.2.796

@otfrom There were some problems with gitlibs in the preview releases, but I think you have a "normal" one. I often have to kill .cpcache and/or .gitlibs if something is up with a gitlib or local/root. It's a bit of a pain sometimes

Also for people interested in clojure on microprocessors, @mfikes just release esprit 1.0.0. https://github.com/mfikes/esprit

I'm interested yet have zero uses for it :thinking_face: