Clojurians Log v2

(future
    (try
      (println "inside future body")
      (assert false) ;;
      (catch Exception e
        (println "error" (.getMessage e))))) ;; no error printed

  (try ;; not in a future, but just normal
    (println "inside future body")
    (assert false) ;;
    (catch Exception e
      (println "error" (.getMessage e)))) ;; error printed

  (future
    (try
      (println "inside future body")
      (throw (ex-info "throwing" {}))
      (catch Exception e
        (println "error" (.getMessage e))))) ;; error printed

the top one doesn’t go to the catch clause

but the others do

but why?

user=> (assert false)
Execution error (AssertionError) at user/eval17874 (REPL:1).
Assert failed: false
user=> (instance? Exception *e)
false
user=> (instance? Throwable *e)
true

That is, assert throws AssertionError if it fails. AssertionError is not an Exception, but it is a Throwable.

See also https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions

How do you handle interopt with Java's Functional Interfaces? Right now it's Reify for me and some nice to have functions & macros. I've looked into libraries such as https://github.com/athos/power-dot

I use reify - it's simple, transparent and flawless. The only problem IMO is discovery (what interfaces/methods should one implement). Extending https://github.com/alexander-yakushev/compliment/ might help if you use cider-nrepl

clj-refactor.el's cljr-add-stubs also helps. I discovered a minor bug in it though https://github.com/clojure-emacs/clj-refactor.el/issues/479 (discovered while doing Functional stuff)

Hi there. Is there a way to restart the agent thread pool after shutdown-agents has been called?

cool, thanks for the insight @vemv 🙂

I’m very curious what your use case is leading you to ask - the core team has been looking at this

cc @fogus

No

my experience: https://blog.valerauko.net/2020/11/23/clojure-and-java-functional-interfaces/

@rextruong technically you can set the agent executors to something custom that could support restart

https://clojuredocs.org/clojure.core/set-agent-send-executor%21 https://clojuredocs.org/clojure.core/set-agent-send-off-executor%21

Can we set the thread pool size used by future?

those methods @emccue linked can be used to customize the executors

That uses clojure.lang.Agent/soloExecutor

which is what set-agent-send-off-executor! changes

that said, there is no thread pool size for that pool - because it can do IO, you can lock up the runtime if you constrain it

Yup. That’s why I want to make the threadpool a little bit larger.

there is no thread pool size

it adds threads as needed

volatile public static ExecutorService pooledExecutor =
	Executors.newFixedThreadPool(2 + Runtime.getRuntime().availableProcessors(), 
		createThreadFactory("clojure-agent-send-pool-%d", sendThreadPoolCounter));

volatile public static ExecutorService soloExecutor = Executors.newCachedThreadPool(
	createThreadFactory("clojure-agent-send-off-pool-%d", sendOffThreadPoolCounter));

https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Executors.html#newCachedThreadPool--

Thanks. Was checking that too.

and none of this is there in the docs of future

Takes a body of expressions and yields a future object that will
invoke the body in another thread, and will cache the result and
return it on all subsequent calls to deref/@. If the computation has
not yet finished, calls to deref/@ will block, unless the variant of
deref with timeout is used. See also - realized?.

but it is alluded to in a comment in one of the community examples

;; Note: If you leave out the call to (shutdown-agents), the program
;; will on most (all?) OS/JVM combinations "hang" for 1 minute before
;; the process exits.  It is waiting for a thread created by the
;; future call to be shut down.  shutdown-agents will shut them down
;; immediately, or (System/exit <exit-status>) will exit immediately
;; without waiting for them to shut down.

if you want to file a docstring request, please log a question at https://ask.clojure.org

What is a really good fullstack demo clojure application? Looking for something I can just clone and run docker-compose up, play around with some metric tracking that interests me on the devops side of the house

Maybe juxt/edge?

https://github.com/juxt/edge?

Looks close... was hoping to find something working with a database like postgresql that I could just run docker-compose up on

This looks like a good fit: https://github.com/tbsschroeder/clojure-webshop-app

Not sure about the "really good" part - I've never seen it before. Just something I was able to find quickly.

Hey all, do you have a favorite process manager for your apps? I've got multiple Clojure/ClojureScript apps running on a single VM, and I'm trying to find a good way to wrangle them. Not only for things like automatically restarting apps on server reboot, but also for development (streaming logs for debugging, stopping the app, restarting the app, etc.) I've enjoyed using Supervisor a lot (http://supervisord.org/), but I'm finding that there are logs coming out of my apps which don't end up in either the stdout or stderr logfiles. It's got such a nice CLI, but if I'm missing errors or failed ShadowCLJS compilations, then it's hurting me more than it's helping me. Anyway, any recommendations?

systemd if you're not going to go the container route

Is this is for a dev setup then maybe something different

What I'm trying to achieve is a unified prod/dev kind of thing. I want to develop in the same environment that my production app will run in. I've got everything in a Vagrant VM, so I've got the environment part of it figured out, but what I'd like is something that will make developing on the VM as "easy" as developing directly on my local. If I was developing on my local for example, I'd probably open up a few different terminal windows and invoke clj -M:alias or whatever the startup command is for each app. My dream VM version of that is to have a little CLI that can invoke all of the startup commands, can foreground/background the apps easily, and can still give me that same direct access to streaming logs. Supervisor can almost do all that, but logging everything from one app to one streaming output doesn't seem to be possible, at least as far as I can see.

I do like systemd for automatically starting processes and stuff like that, but it would only solve part of my problem.

To be honest, I'd go with Docker and Docker compose for single-machine, multi-app deployment. Your dev environment and production will always be different

there’s also https://github.com/ajoberstar/ike.cljj/

I'm familiar with all these, what I'm looking for is: what problems are you having in your code base where you reach for these?

"I wanted to do X but I couldn't or it was cumbersome... "

I'm once again running into the wall that is my ignorance of JVM multithreaded constructs. looking for advice: I working on a bit of CLJC code that schedules a task to be run in some way that doesn't hog the current thread. It sounds like in the JVM nomenclature this could be easily handled by an ExecutorService?

Any good ideas for extracting all calls to (log/warn ...) from .cljc files? I thought about simply using clojure.edn/read and then prewalk the data but it cannot be read because of #?(..) Is there some cljc aware reader? Thanks!

Yes. Or just use (future..) if you don't need to set up a dedicated thread pool and are ok with using the default one...

ah, I guess I should have specified that multiple tasks could be queued up at once and I want to process them in some order

hence the desire for something more than (future ,,,)

this might be more than you're looking for, but https://github.com/borkdude/edamame I think fits the bill

you can configure it to process reader conditionals

clojure.core/read-string also takes :read-cond :allow, but note that it can execute code if *read-eval* is true.

I think you can also use https://github.com/jpmonettas/clindex or clj-kondo to find var usages.

Thank you!

What would be the benefits of Docker over Vagrant? I'm fairly comfortable in a *nix environment, but pretty new to developing on a VM. After a little research, Vagrant seemed like a good option for my goals. And I do think I've achieved that 1:1 parity between dev and prod to be honest (complete with SSL and everything). I do like the idea of hitting all of my "production problems" early on so I don't have to encounter and solve all of them in the 11th hour before I deploy, but it does seem to add a lot of complication too. I'd be curious to get your perspective though.

I've used Vagrant at multiple shops as means of standarizing dev environments, and it never looked like production (even before Docker), on top of that developer experience is not so great, especially when it comes to 'edit locally see changes in the VM', that part is always flaky. Of course you can work in the VM directly, and that (IMHO) works great if you accept some trade-offs (performance being the biggest one). Now for production, even if we're talking about a single machine - Docker simplifies this in many ways: you can test and build software locally with the same compose setup, but you get the additional benefit of not tying your application's runtime to the host, and forces you early on to figure things like secrets management, repeatable builds and so on. Sorry for the random thoughts, I'm just speaking from experience of trying to achieve dev-prod parity and never seeing it realized fully (or even in 50%)

you might be familiar with all these, but @jarvinenemil might not be, and he asked the original question 🙂

oh for sure, thx for dropping them

I am hijacking :)

@alexmiller I asked because the functional interface interopt becomes cubersome due to the proliferation of interfaces that one has to explicitly implement. example in the kafka streams DSL Java

streams
.filter((k, v) -> value.equals("Purple haze, all in my brain"))
.mapValues(value-> value.toUpperCase());

(-> streams
    (.filter (reify Predicate
               (test [_ k v]
                 (= "clojure-rocks!" v))))
    (.mapValues (reify ValueMapper
                  (accept [_ v] (clojure.string/upper-case v)))))

Do tasks support pausing? I guess look at the E.S.api,does it do what you want?

Are there any current libraries for SFTP? Or does it make sense to just wrap the apache commons functions for SFTP?

there's tools.reader also with .cljc support

I don't think so out of the box

I'm tempted rn to build my own task running/scheduling machine using core.async but I also really don't want to depend on core.async

@jmckitrick I've been using this one for a long time and it hasn't disappointed me https://github.com/miner/clj-ftp

I saw that, but didn't see sftp support on the home page. Guess I should dig a little deeper?

As an alternative to Docker, it might be worth looking into podman. It's almost a drop-in replacement that doesn't require root privileges.

hmm, I guess you're right. sorry

Rats. I was hoping you were right.

The repo is active though so you may have luck requesting that feature. Or maybe simply hacking that .clj file

maybe https://github.com/clj-commons/clj-ssh

but why are you using streams at all? why not just filter / map in clojure

where are you getting the streams from?

kafka I guess

do you have a link to where you're getting that example?

you're correct @alexmiller, it's an example from the official java client kafka streams lib: Simple Clojure example (not mine, but identical to the code that I gave as an example) https://github.com/perkss/clojure-kafka-examples/blob/master/kafka-streams-example/src/kafka_streams_example/core.clj Java example mapValues in the official docs: https://kafka.apache.org/20/documentation/streams/developer-guide/dsl-api.html

If you prefer a gist with 100% identical java/clojure I can fix that

nah

that's helpful, thx

@jarvinenemil you’ve probably seen this, but there is also https://github.com/fundingcircle/jackdaw

core.async already has a running/scheduling machine for tasks - pipelines. They guarantee order maintenance and predetermined parallelism. They're pretty good at their job

But not suitable for pause-resume

I have seen it, it seems nice but I prefer embracing java-interop