off-topic

https://github.com/clojurians/community-development/blob/master/Code-of-Conduct.md Clojurians Slack Community Code of Conduct. Searchable message archives are at https://clojurians-log.clojureverse.org/
solf 2021-01-25T03:02:18.104500Z

Seems like a good use-case for (the recently released) https://github.com/nextjournal/clojure-mode

emil0r 2021-01-25T08:28:30.105700Z

https://chriskiehl.com/article/thoughts-after-6-years

💯 1
respatialized 2021-01-26T21:20:43.135400Z

in the take economy, detail and correctness are liabilities - being loudly wrong drives way more engagement than being boringly correct. that's why these lists are always at the top of HN: everyone feels the need to correct them

lukas.rychtecky 2021-01-25T08:44:26.105800Z

Hah it seems that someone put things from my head on a paper! Anyway didn’t we have a call two weeks ago, did we?

simongray 2021-01-25T09:39:11.106600Z

Somewhat agree, but I have to say that these lists have a certain circle-jerky flavour to them. What did we really learn reading any of those bullet points? The statements are never qualified in any way. This exact blog post or something close to it floats to the top of Hacker News nearly every single day. It’s the kind of stuff I will read when I sit on the toilet to pass the time.

➕ 2
lassemaatta 2021-01-25T09:52:38.106800Z

"Cargo-cults are bad. Anyways, here's my strong opinions about a bunch of stuff without any explanation." 😄

😁 4
😂 3
simongray 2021-01-25T10:11:18.107100Z

Precisely.

lukas.rychtecky 2021-01-25T10:40:35.107400Z

I don’t think that these bullets try to qualify anything, they are based probably by some experience that’s it 🙂

orestis 2021-01-25T11:17:52.107700Z

IME when you’re 6 years into development you go through the phase of denouncing some “accepted wisdom” that you consumed uncritically at the beginning. It helps 🙂

jcburley 2021-01-25T14:36:26.108500Z

• “90% – maybe 93% – of project managers, could probably disappear tomorrow to either no effect or a net gain in efficiency.” Even if true, IMO it’s a challenge determining which are the 7-10% worth keeping around.

Timur Latypoff 2021-01-25T15:18:21.113200Z

Argh, just getting this off my chest: C#’s record and ImmutableDictionary are so terribly implemented, I am almost tempted to include ClojureCLR into the C# project just for collection classes with proper equality semantics and composabilty.

😅 1
orestis 2021-01-25T15:37:37.113800Z

Hah, as someone with PM duties, I think that people really underestimate the role PMs play — or haven’t been in any reasonable large or long project with no PM, and haven’t seen what a shitshow it can be 🙂

👍 1
2021-01-25T16:46:50.114400Z

Using Clojure for a while, then looking in detail at how other languages implement these things, can certainly be a way to see why Clojure's implementation of equality semantics can be measurably slower than those others, I think.

2021-01-25T16:47:46.114600Z

Clojure's equality semantics is definitely doing some extra work for you, e.g. in Java land at least, to compare different storage sizes of integers as equal in integer numeric value to each other, not equal in (type, integer numeric value) combination as Java equals() method does.

Timur Latypoff 2021-01-25T16:54:37.114800Z

@andy.fingerhut that is true. On the other hand (in my not so big experience with Clojure), it is so much more robust and predictable in the wayit works than what happens in other languages, it frees up a lot of mental resources and lines of code. Also, it seems to me, if it comes to a point where Clojure's equality semantics is a performance bottleneck, probably rethinking the whole idea of simply equating nested data structures for this specific use case would yield much better results than any possible optimizations.

2021-01-25T18:14:21.115200Z

I agree with you on the freeing up mental resources, no question. I have often looked into some of the implementation details and performance, which is the only reason I am aware that there is some performance cost for Clojure's = behavior. My current guess is that most of that performance cost is not due to the nested data structures reason (if you wanted that deep-equals behavior, you couldn't optimize it away in any way I know of, anyway), but the "making different JVM types be equal, despite their different types". Take that with a grain of salt, as I haven't done actual experiments/measurements myself to authoritatively say that X% of the cost of Clojure's = is due to reason X

👍 1
coby 2021-01-25T18:53:10.118800Z

I'd like to set up a reverse proxy (nginx or some such) to go to a different backend depending on the HTTP method - POSTs go to an instance optimized for writes, GETs to one optimized for reads. Is there a name for this specific technique?

coby 2021-01-25T18:53:24.118900Z

I've heard the term "sharding" used in the context of databases, but I think that term is even more specific and has to do with how indexes are divided up. Or maybe I'm splitting hairs...

2021-01-25T19:17:29.119100Z

the most common version of this I've found is with CDN used for get of static assets - I don't know the technical term you want, but adding CDN to your search query might help

1
2021-01-25T19:17:59.119300Z

this is more specific than just segregating GET, it also looks at paths to determine which are static

Stefan 2021-01-25T20:27:40.121300Z

Can there be such a thing as too off-topic? 😉 Our Clojure servers are communicating through Apache Pulsar. This was setup by somebody who has left the company. Is there anyone here who has a very good understanding how to setup a Kafka cluster and is willing to help us out?

Ben Sless 2021-01-26T10:28:08.128300Z

While you can't read or commit single messages in Kafka you do have transaction semantics which you can use to narrow your scope down to single messages, no?

gklijs 2021-01-26T11:05:32.128600Z

@ben.sless not really. If you know the topic-partition and the offset you can just get one message, but it’s cumbersome. Or you can use ksqlDB, or similar things that need additional middleware to make it easier.

2021-01-25T20:29:46.121400Z

sounds like a good case for hiring a contractor. kafka is a distributed immutable log, it can do some of the things that pulsar is used for, but isn't well suited for most pulsar uses IMHO having used both.

2021-01-25T20:30:07.121600Z

if what you want is persistent communication between services that spin up / change servers, pulsar is simpler than kakfa

2021-01-25T20:30:32.121800Z

if you want immutable data logs that can be used flexibly, kafka does the job (but is much more finicky to keep running)

2021-01-25T20:34:17.122Z

pulsar is about messaging (which means it has logic about messages, senders, and recipients baked in), kafka is a distributed log, using it for messaging requires a lot of hacks and conventions and leads to weird corner cases in setup

Stefan 2021-01-25T20:52:19.122200Z

Yeah hiring a contractor is definitely an option. Pulsar is serving us well for the most part, we’re not really looking to change to something else. But when there is a network outage or something in the data center, or on one of the nodes things go down due to out-of-memory, they take all of the other nodes down as well. That was not the point of having a four-node cluster I thought 🙂

2021-01-25T20:56:57.122400Z

in distsys even numbers tend to be a bad idea generally - you need majority so 4 nodes perform worse than 3 (or at least no better)

👍 2
2021-01-25T21:14:51.122700Z

more relevant to your original question: confluent offers a docker compose config to spin up kafka locally, and you can purchase a kafka cluster as a managed service from aws

gklijs 2021-01-25T21:44:03.123200Z

Confluent also has an open source operator for Kubernetes that seems to work fine. But like @noisesmith said, the typical use case for Pulsar is quite different from Kafka. The major drawback of Kafka is you can't commit a single message, and also can't easily read a single message. It's really made for streaming.

2021-01-25T21:46:40.123500Z

also IIRC pulsar has straightforward request - > response, which is something service clients tend to take for granted, making request->response on top of kafka gets super weird and messy

☝️ 1