I'm trying to figure out if the variation I see is because of GC or thread competition. Is there a way to see what calls get "paused" for GC?
Depending upon which GC algorithm you are using, it might be no calls, e.g. some GCs run in separate threads, without pausing any of your computation threads.
makes sense. I'll read more on the subject!
what is a non-sequence called in Clojure? In traditional lisps, non-lists are called atoms. But Clojure uses the word atom for something else.
Don't think there's a general name for that. At least never heard of one.
I normally just call things as they are: list, vector, map, set, symbol, keyword, number, string, etc.
I think no one will be offended if you refer to it as a scalar
As far as I can tell, side effects are okay in reduce? Is that correct?
Sounds complex...
I guess it kind of is, it is a bit of a unique case. it's a function that traverses a ktable within an open connection, sometimes it makes sense just to do something side effecting, i.e. for each value push a new message to kafka. In a few cases where we know the table is small I want to just get all the results into a seq to do something else with. reduce
lets me either build a seq from each value, or do something side effecting and then throw the value away. I can't let things be too lazy as the connection might close before things are realised.
I have actually hidden that behind two functions, one which just gets all values, and another which only allows side effects on each row, but since the underlying stuff is some funky java interop I only wanted to write the general version once.
Yes, reduce is strict
thanks!
Take a look at run!
for example. It is implemented with reduce
scalar? that's a pretty good name actually
can someone remind me the idiom to use with try
/`catch` to catch the exception if a condition, else don't catch it, or perhaps rethrow it? I was looking for an example in the https://clojuredocs.org/clojure.core/try and https://clojuredocs.org/clojure.core/catch, but I didn't see any.
should I simply call throw on the given exception? will that destroy the stack trace?
Yes, calling throw on the given exception should do what you want. And it shouldn't destroy the stack trace. Also before throwing you have full access to the given exception meaning you can build new exception based on the stack trace from the one given
I don't need it now, but if I test the exception object and decide to rethrow it, is it still subject to the other (catch ...)
clauses in the same (try ...)
? That could be good or that could be bad depending on the situation. It would be excellent if that were specified behavior.
Am I missing a clojure.core fn to get the dispatch val of a multi-method given some input, I thought it would go hand in hand with get-method
but I can't see anything obvious. Would it be bad to use ((.dispatchFn ig/init-key) "blah" "blah")
or is that unlikely to change?
Does anyone here also program in Scala? If so, maybe you can help me answer a https://clojurians.slack.com/archives/C16LEKSLT/p1604667792002600?
how do you deal with exceptions that happen in anonymous functions that run in a future
?
I had to run the code without the future and de-anonymize the function to even get the exception to show up instead of being silently discarded, it was quite the painful debugging experience
yes, it is unclear from documentation but expressions in catch
evaluated outside of try context
should I not use futures for non-trivial work?
ah, it was slingshot's throw+
/ try+
that somehow hid all that
What I mean is, if you do care, then dont implement a fire and forget behavior. The "forget" part implies that you don't care if it succeeds or not.
So I assume you want to say log a message, output an error metric, perform a retry, etc. So for any of these, you need to explicitly handle the error
You can also check the status of the futures from the main thread without blocking, if you want to handle them here (or since you're error handling is a side effect as well, you can handle them in the future thread).
So like in your main thread, you could check if the future is realized? yet, and if it is, check if it errored and do whatever you want in that case.
But in your case, probably just have a try/catch inside the future itself.
right, I have a try/catch in the future
but I start to feel like I'm using futures incorrectly, and should be using another multithreading technique more appropriate
I can't speak to that, but it's fine to use futures to fire async tasks that only do a side effect. Just make sure you try/catch them.
(future
(try (task)
(catch Throwable t
(handle-error t))))
You can make yourself a little util for it:
(defn fire-async
[task error-handler]
(future
(try (task)
(catch Throwable t
(error-handler t)))))
With future, you can fire an infinite number of concurrent tasks. There will be one thread per concurrent task. So at 100 concurrent tasks, you will have 100 threads. As the number of concurrent tasks goes down, the number of threads will go down as well, but it will lag behind, as threads wait for 60 seconds in the hope they can be reused.
This means that in theory, you could run out of memory if you fire too many concurrent tasks. And really, it will be the total set of concurrent futures and agent send-off executing in your program.
That's why sometimes people will recommend in server code where you can't predict how much concurrency you'll end up having (nothing upstream of the future call is bounded for example), to either use your own executor service. Or to change the default executor for future to a bounded one, or to use core.async with bounds.
thank you very much
actually, with core.async, I could probably create a ~10 channels that work concurrently, with extra work waiting in the queue
and when my program exits, I could ensure they are all done with alts!
which is neat (I wondered how I could do that)
I will play around with that
When your app closes, running futures will prevent it from shutting down, so it will wait for all futures to complete as well, nothing will get interupted
Even if you call shutdown-agents
this will be the case. All running future and agent actions will need to first complete before the app terminates
you can write defmulti with a var in place of dispatch function
(defmulti foo some-dispatch-fn)
this function you can use to get dispatch value
(some-dispatch-fn "blah")
Hi all, "lein ring server" command is for starting ring server, what is the command to stop already running server?
It should behave like Java does. So if the exceptions in the catches are in an inheritance hierarchy the most specific one is the one caught. A re-throw would not be caught in the same try-catch but, obviously, any finally expression is eval'ed at the end.
Hi!
What is an easy way to save different configurations for test and production? I want to use a different database name when running tests and a different database for development/production code when not running tests. I first thought of hardcoding the database name in the dbfuncs.clj
file (using def
. But then I changed it to a function. I need to now add code that reads the database name from somewhere. Is it possible to say something like: if running tests then use dbname gamesdbfortest
and if not running tests then use dbname gamesdb?
(def hds-opts (atom nil))
(defn db-config
[dbname]
{:dbtype "postgres" :dbname dbname})
(defn set-datasrc
[dbname]
(let [^HikariDataSource hds (connection/->pool HikariDataSource (db-config dbname))]
(reset! hds-opts (j/with-options hds {:builder-fn rs/as-unqualified-lower-maps}))))
there's environ
and lein-environ
, works well for me
Not sure if itβs relevant, but in Java every non-main thread discards unhandled exceptions by default, but you can change that behaviour as described here: https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions
thanks, I'll add that, even though I mostly use futures
(Hopefully) simple question about clj-time
: Is it possible (and if so, how) to get out non-traditional date values? For example, I would like to get day-of-year
, week-of-year
and day-of-week
from a parsed UTC date.
I suppose on day-of-week
can just translate from text abbreviation to a numeric value, so nvm about that one. π
@lkottmann Thank you. Will look into environ
. I am using deps.edn
so the other may not work.
Anyone has deps.edn
examples for environ
? I see only lein
and boot
examples in the readme.
Hi guys, can anyone help me and translate the following code snippets into json structure or something I could understand. I don't have a Clojure background and I need to recreate a similar structure in a javascript. i will appreciate your help
(merge {:event/stream :users
:event/partition-key user-id
:event/type :users/created
:event/actors #{:user-id :referring-user-id}
:user-id user-id
:referring-user-id referring-user-id}
(select-keys new-user [:first-name :last-name :username :email]))
And this one:
{:event/type :users/created
:event/created-at (java-time/instant)
:user-id #uuid "d415fb11-3d42-4a5d-911e-484b1baad7af"}
Pretty sure futures are dispatched on non-main threads, so unless your code is catching exceptions and doing something with them, theyβll disappear silently (as per JVM default behaviour).
the translation is trivial, what's the specific issue you are having?
{:foo/bar baz}
in a clojure data literal becomes {"foo/bar": baz}
in json
or "foo_bar"
if you prefer - json doesn't have namespaces so there's no direct equivalent
does it mean the
{:event/type :users/created
:event/created-at (java-time/instant)
:user-id #uuid "d415fb11-3d42-4a5d-911e-484b1baad7af"}
Will be translated into something like:
{"event/type": "users/created",
"event/created": "2020-06-02T21:34:33.616Z",
"user-id": "d415fb11-3d42-4a5d-911e-484b1baad7af"
}
as a json objectThank you @noisesmith for a quick reply
(println (json/generate-string {:event/type :users/created
:event/created-at (java-time/instant)
:user-id #uuid "d415fb11-3d42-4a5d-911e-484b1baad7af"}))
{"event/type":"users/created","event/created-at":"2020-11-06T16:02:42.287334Z","user-id":"d415fb11-3d42-4a5d-911e-484b1baad7af"}
you can check what it serializes to to double check your work. or let cheshire do the work for you
the json alias here is cheshire.core
Great! As I think that's all what I need to know π Thanks guys and have a good weekend!
@mafcocinco As one of the clj-time
maintainers, I'll remind you that the library is deprecated and we recommend folks use Java Time instead, either directly via interop or through one of the available wrappers (which are listed on the clj-time
README).
cool thanks.
No, a thrown exception inside the future is returned as the value of the future
And the future :status will be :failed
which libraries do you use for data visualization? (charts / graphics)
vega-lite is the way to go. Oz is good for getting started, but it's pretty easy to create your own wrapper around vega-embed when you need more flexibility.
Can both be used from Clojure and not ClojureScript?
Yes
Hi, I'm trying to get a Clojure+ClojureScript project working with figwheel-main. I'm somewhat stymied by CORS errors -- do I need to allow calls from localhost:9500 on my server at localhost:8080 or is there any way to just serve everything from the same port?
https://github.com/juxt/aero is a very good library for managing different environments
I think it's just Ctrl-c
to kill the process.
Using wrap-reload with ring can reduce the need for restarting the server, although it's still required when adding dependencies.
Use doseq
if its only side effects
Adding Access-Control-Allow-Origin and Access-Control-Allow-Credentials to the headers works. I guess that'll be that.
The problem comes from futures that I never dereference
Are they ever cleaned up btw?
you can configure your cljs output options and a "wrap-resource" ring middleware to serve the cljs from the same port that you serve your site pages
When the reference to the future is gone, it will be garbage collected
Why would you care about a failed future you don't deref? This fire and forget behavior would imply to me that you choose to not care about what happens to the task
True, I do use scalar for non collections
I can serve the js which figwheel builds but it doesn't live reload for me (although it does fix the cors issue). I've been using ` (defn app [] (html [:head [:title "Welcome to blah"]] [:body [:div {:id "app"} [:script {:src "cljs-out/dev-main.js"}]]]))`
I think Figwheel uses websockets to make live reload work. In that case, you might have to forward websockets traffic as well as static files. I would start by looking for websockets traffic in the browser's network monitor.
forgot to press enter, sorry! Unfortunately in my case I don't own the defmethod, it's integrant/init-key, the method it uses is anonymous and delegates to a private method, .dispatchFn
worked fine for me for now, just suprised there wasn't something in core for it
Thanks, I actually was replacing a bit of code that was using a doseq
as I needed it sometime to be side effecting and othertimes not and to give back all of the values. reduce
gives me the best of both π