Clojurians Log v2

Just nitpicking a bit regarding the above, channel operations in a thread are blocking the thread instead of parking. They have different names so you can’t confuse them (<!! instead of <!). Also, an async/thread isn’t exactly normal. It’s a daemon thread so you can’t rely on the JVM to wait for it at the end of the program.

Yeah true, I find it a bit cumbersome to write go block because of the macro expansion. But that’s a whole different topic

Do you have any good resources about something like “concurrency theory”. Because I hear daemon thread, JVM thread, native thread, POSIX thread. But I wouldn’t be able to explain the differences

What would the shape of that look like, i.e., to use a go block, spawn a thread and wait for the data to be emitted back into the go block for processing (if I've understood that correctly)?

(go
  ;; ... Do whatever
  (let [result (<! (thread-call query-data))]
    ;; ... Do something with `result`
    ))

That’s an example of a one-off thread. A thread might also continuously produce data into a channel, which you can read in a go-loop.

(go-loop [result (<! (thread-call query-data))]
    ;; ... Do something with `result`)

?

why would you use go blocks if you are using full threads anyway?

at that point you might as well use futures and a threadpool

I don’t know why you’d want to create a new thread instead of using cached thread pool

Maybe you have multiple go block in your architecture, but 1 of them needs to do IO. That would make sense to me at least

On the other hand maybe that’s not how you build concurrent systems in Clojure. I’ve only written them in Erlang so I’m trying to switch my mindset

it depends on your IO boundaries’ implementation

if you’re using a standard synchronous implementation then you need to have a thread pool that handles that IO

what you do elsewhere is up to you

you can stuff the result of IO threads into channels and use go blocks to process that

or you can simply pass IO thread a callback function, that it executes with the results, that function is executed on IO thread then, but if it’s short its not a problem

and generally you have a threadpool on inbound IO anyway (web server has its own threadpool which processes request handlers)

so if you’re not careful, even if you use go block, you might end up blocking on the result of that go block, which means you await on request thread for result anyway

so to have a fully aync solution you need to have async inbound IO, so a webserver that gives you a callback to call with the results or a webserver that will take a channel as response

Very useful information!

@kevin.van.rooijen "Java Concurrency in Practice" by Brian Goetz is a good on

Thanks!

Great discussion indeed, very timely in my case too. Overall, async/thread makes a lot of sense for long running processes (i.e. daemons) such as IO. For these you do not want to take a thread from a bounded pool (such as clojure.core/future which uses the agent pool). And since async/thread creates daemon threads, you need to either take from the channel it returns within a non-daemon thread, or have an entirely separate control mechanism, so that the JVM does not exit before your processing is done. Is that a fair summary?

Unless you really need long running daemon threads, e.g. to consume from kafka and put messages into a channel

For databases a typical scenario is a connection pool with its own thread pool

Yeah, that’s what I was thinking

But I’m also reading that certain websocket libraries use core async to handle connections

If that’s true, wouldn’t it make sense to query a thread pool from a go block?

I also think that @roklenarcic and I had a bit of a miscommunication, because my intention is to use a thread pool. I was just using an example

websockets.... clojure side or clojurescript side?

Both

But I could be mistaken

on cljs side there are no threads...

There is, one at least 😛

I haven’t looked into it in detail so I can’t really comment

But I’m mainly talking about the Clojure side

IO are not daemons

If you are listening to something yes, e.g. kafka consumer

I see that people are confused. Daemon threads in Java are just threads that do not prevent JVM from exiting when running.

JVM will exit when all non-daemon threads stop

Or a web server waiting for new connections and handing off requests to your code

those threads are explicitly NOT daemon

otherwise JVM would exit while waiting for requests

hence my earlier comment

Usually the main function starts the server (a threadpool of non-daemon threads) that wait for requests

and main function then exits

if threads were daemon then JVM would exit too

Ah, that’s a good distinction to know