https://docs.oracle.com/javase/9/docs/api/java/lang/Process.html#pid--
nice
oontz oontz oontz
? ? ?
the sound of techno
In the #beginners channel? Perhaps more appropriate for #music or #off-topic ?
@mario.cordova.862 the first thing I would think about is to upgrade your JDK and garbage collector. Since you're still using concurrent mark and sweep (which has already been removed in recent JDK versions) it suggests you have something older - I suppose JDK 8?. JDK 11+ is perfectly fine to use these days and even if you cannot upgrade JDK I'd suggest you try G1 Gc (available in JDK 8 too.
Also, are all the errors you mentioned happening about the time when the app gets unresponsive?
For lightweight investigation I suggest using jcmd
(much better in recent jDK versions) or similar tools to quickly inspect JVM (e.g. class histogram). You can also configure JVM to make a heap dump on OOM via -XX:+*HeapDumpOnOutOfMemoryError*
(https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/clopts001.html) and examine it later e.g. via VisualVM (or Yourkit, Eclipse MAT, etc.)
Yes, we are using JDK 8. And the errors all occur at the same time. All API calls seem to fail. Most request handlers just barf. The CPU spikes. Locally I added the -XX:+*HeapDumpOnOutOfMemoryError*
and set the Metaspace to a low value so I can replicated the issue. Now I am using the heap analyzer tools but honestly I can barely make any sense out of it. I see references to some of our code but I can't really make sense of the big picture.
Note that even a simple clojure application demands at least 30-40 MB - depends on how much code & dependencies you load. If your app consumed gigabytes then that might be a real problem - would be good to have a continuous monitoring of various aspects of the JVM memory consumption; it's not super-specific to your problem but I always recommend reading this SO question to get a better understanding of JVM memory consumption: https://stackoverflow.com/questions/53451103/java-using-much-more-memory-than-heap-size-or-size-correctly-docker-memory-limi/53624438#53624438
And here some nitty-gritty details about Metaspace: https://stuefe.de/posts/metaspace/sizing-metaspace/
Does anyone know how local variables are handled w.r.t. the GC in the case that clojures are maintained for a long time?
What do I mean. I mean that if I have a function which has local variables whose values are HUGE data structures, and the function returns a clojure which DOES NOT reference those variables, is the GC able to deallocate them or is the entire stack-frame maintained.
e.g., in this contrived example, the function returned only references p
whose value is an integer, so m
and q
, can be deallocated.
foo
returns a function. Suppose that I hold onto that function, in an global variable.
Is it my responsibility to deallocate m
and q
or will the GC do it for me?
(defn foo [f]
(let [m (range 10000000)
q (map inc m)
p 12]
(doseq [x q]
(if (f q) (throw (ex-info ...))))
(fn [g]
(g p))))
I wonder whether excessive use of lazy sequences might be the culprit. A raw sequence only contains final values, even if very many of them. However, a lazy sequence must contain references to all values necessary to compute the rest of the sequence. It could be that in extreme cases lazy sequences occupy much more memory than the sequences themselves.
(defn foo [x]
(let [very-long-sequence (compute-very-long-sequence-of-sequences x)]
(map count very-long-sequence)))
For example, foo
returns a lazy sequence which must retain a pointer to x
and also to very-long-sequence
.... however, the final sequence is just a list of integers. Right?right
so as I understand, I can wrap all generations of such lazy sequences with doall
for force them to be non-lazy. But finding all such lazy generations may be challenging. Not really sure how to do that.
or use eager constructions like vectors, transducers, reductions
sorry, don't know how to do that. And it sounds like a difficult and invasive refactoring of my codebase, especially since it is not guaranteed to solve the problem.
then doall
sounds good
map
is documented as returning a lazy sequence. but it is not clear whether mapcat
returns a lazy sequence or not.
after a bit of testing, it is promising but not conclusive. I get far fewer out of memory errors after forcing all calls to map
and mapcat
to be non-lazy. My tests now run in about 2 minutes, whereas they previously took a half hour to run, But occasionally still give out-of-memory errors. But less often.
very interesting results.
AND the cpu meter I'm using on my Mac was showing huge stress, now is much happier and the cooling fan is much quieter.
A closure only retains values bound to its free variables. Both sequences are candidate for GC when foo
returns.
Hello there. I'm trying to deploy a simple reagent app to heroku and I'm lost.
Following this guide doesn't work: https://devcenter.heroku.com/articles/using-node-js-with-clojure-and-clojurescript-applications
I get this Error: browser bootstrap used in incorrect target
when I run heroku logs --tail
Wait, let me post more from the logs
2020-10-14T09:15:45.000000+00:00 app[api]: Build succeeded
2020-10-14T09:15:46.930655+00:00 heroku[web.1]: Starting process with command `node public/js/compiled/main.js`
2020-10-14T09:15:50.116752+00:00 app[web.1]: Setting JAVA_TOOL_OPTIONS defaults based on dyno size. Custom settings will override them.
2020-10-14T09:15:50.323232+00:00 app[web.1]: /app/public/js/compiled/main.js:1564
2020-10-14T09:15:50.323261+00:00 app[web.1]: throw new Error("browser bootstrap used in incorrect target");
2020-10-14T09:15:50.323262+00:00 app[web.1]: ^
2020-10-14T09:15:50.323263+00:00 app[web.1]:
2020-10-14T09:15:50.323263+00:00 app[web.1]: Error: browser bootstrap used in incorrect target
2020-10-14T09:15:50.323264+00:00 app[web.1]: at Object.<anonymous> (/app/public/js/compiled/main.js:1564:11)
2020-10-14T09:15:50.323264+00:00 app[web.1]: at Object.<anonymous> (/app/public/js/compiled/main.js:1695:3)
2020-10-14T09:15:50.323264+00:00 app[web.1]: at Module._compile (internal/modules/cjs/loader.js:1015:30)
2020-10-14T09:15:50.323265+00:00 app[web.1]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)
2020-10-14T09:15:50.323265+00:00 app[web.1]: at Module.load (internal/modules/cjs/loader.js:879:32)
2020-10-14T09:15:50.323266+00:00 app[web.1]: at Function.Module._load (internal/modules/cjs/loader.js:724:14)
2020-10-14T09:15:50.323266+00:00 app[web.1]: at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
2020-10-14T09:15:50.323266+00:00 app[web.1]: at internal/main/run_main_module.js:17:47
2020-10-14T09:15:50.376140+00:00 heroku[web.1]: Process exited with status 1
2020-10-14T09:15:50.412256+00:00 heroku[web.1]: State changed from starting to crashed
2020-10-14T09:15:51.886696+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=<http://evening-caverns-84275.herokuapp.com|evening-caverns-84275.herokuapp.com> request_id=b6bb93ae-1e8d-4b0f-ae8c-168effede9dc fwd="152.32.111.228" dyno= connect= service= status=503 bytes= protocol=https
2020-10-14T09:15:52.666612+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=<http://evening-caverns-84275.herokuapp.com|evening-caverns-84275.herokuapp.com> request_id=8031aa23-9a08-4d57-839f-0864b8b33081 fwd="152.32.111.228" dyno= connect= service= status=503 bytes= protocol=https
There's also an interesting version of fn*
using :once true
metadata for one-shot closures: https://stackoverflow.com/questions/23586710/what-is-fn-and-how-does-it-differ-from-fn
@leonoel, so closures contain only the values bound to free variables, not to the environment frames which contain those variables? If that's so, that's terrific.
Is this the sample app from the Heroku guide or your own reagent app?
Posting a link to the logs or adding them as a thread would be a preferred way to share, avoiding taking up lots of space in the main thread. Thanks
I essentially copied everything over
@signup867 You say a "simple Reagent" so I suppose you only target the front-end. If so you don't need NodeJS.
The error seems your trying to run the Reagent/React compiled file with node which that cannot works.
> 2020-10-14T09:15:46.930655+00:00 heroku[web.1]: Starting process with command node public/js/compiled/main.js
If you tried this command on localhost, you'll get the same error.
Node don't know about JS targeting the browser https://github.com/thheller/shadow-cljs/blob/b05b8f41afd45564ef7b9831a07201d09a5c4a07/src/main/shadow/boot/browser.js#L9
The sample reagent app deploys correctly, btw.
@signup867 The third part of the guide your mentionning talks about targeting NodeJS so you can run an CLJS app like this one https://github.com/PrestanceDesign/cljs-sample-express but not a Reagent one.
Ahh, hmm so does that mean I have to roll out my own server?
My only thought is how bootstrap is being pulled in. I use a CDN for CSS libraries. If using local libraries then check the path to them is correct when deployed on Heroku
No you have a bunch of options like Netlify, Github pages, Surge, even Netify if you want to stay with: https://create-react-app.dev/docs/deployment/
https://blog.heroku.com/deploying-react-with-zero-configuration
I think I remember having exchanged with you last month about Reagent on Netlify, right? > Ahh, hmm so does that mean I have to roll out my own server? Maybe i didn't fully understand your needs for Heroku, do you need a backend?
Not really. In fact itโd be great if I can just copy paste my public
folder and let it serve my files
The reason why I think I need a backend is because I need to store input from users. Iโll need a database for this, right? My impression is that you canโt do this with Netlify and other static hosts.
(Sorry, Iโm really new to web dev but otherwise not that new to programming in general. Hope you understand.)
I don't think this behavior is documented anywhere but that's what the compiler actually does and it's very reasonable to rely on that IMO
Hey folks! I'm trying to monkey-patch a function in one of my deps. I've tried both (intern 'dep-ns dep-fn (defn ......
as well as:
(in-ns 'dep-ns)
(defn dep-fn ....
(in-ns 'my-ns)
But I'm getting a java.lang.ClassNoFoundException: dep-ns
Any ideas? ๐ I can resolve dep-ns
functions from my-ns
, so it's not some issue where I can't require the dependency.clj
Clojure 1.10.1
user=> (require 'clojure.set)
nil
user=> (in-ns 'clojure.set)
#object[clojure.lang.Namespace 0x5a772895 "clojure.set"]
clojure.set=> (defn union [x y] (prn "i've changed this"))
#'clojure.set/union
clojure.set=> (in-ns 'user)
#object[clojure.lang.Namespace 0x1c12f3ee "user"]
user=> (clojure.set/union #{:a} #{:b})
"i've changed this"
nil
user=>
did you require 'dep-ns ?(ns my.ns
(:require [clojure.set])
(in-ns 'clojure.set)
(defn union [x y] (prn "i've changed this"))
(in-ns 'my.ns)
(clojure.set/union #{:a} #{:b})
This gives me an arity exception saying 3 args are passed to union
. Note that I'm not running at the REPLcan you show the stack trace?
Yeah hang on
But when I move it into a function (e.g. wrap the above lines inside a function) and then evaluate that function at the REPL, it works
So it would seem that union
is being called elsewhere after I redefine it, when the reader goes through?
Would that make sense?
what does the stack trace say?
it should tell you where its called with three args. and union has a much more rich signature than what we did just for example purposes
([] [s1] [s1 s2] [s1 s2 & sets])
Hi, clojure beginner here. i also use java and javascript at work.
i am on windows 7. i use lein.bat. but i read this new other tool, clj, which needs powershell. which one should i use? i am now learning the basics of clojure. i plan to create web apps and games in the future. the goal is to use clojure in the back end and clojurescript in the front end.
Thanks Sean and John.
John, i found your Youtube channel just now. I subscribed to it. Watching 4clojure guides playlist now.
Feel free to ask any questions about my content in the #practicalli channel or send me a direct message.
Sorry I have to run off but I appreciate the help and will let you know if I get to the bottom of it :)
spanish article https://medium.com/@jorgetovar621/clojure-101-619646e142a6
If you're stuck on Windows 7, you probably can't install a recent enough Powershell to be able to use the new Clojure CLI.
But ask in #clj-on-windows in case folks there know more about Windows 7 and Powershell than I do (very likely -- I'm a lot more familiar with Windows 10).
Hello everyone! Is there a more โelegantโ way to write this?
(defn remove-empty-key [m k]
(if (empty? (k m))
(dissoc m k)
m))
(-> (get-from-db {:some :person})
(remove-empty-key :kids))
I suppose the more elegant way would be not to add the key to the map in the first place if the value is empty.
It comes from DB. It returns things like:
{:name "Maria"
:kids []}
But remove-empty-key
is simple and straightforward, so Iโd say thatโs as elegant as anything else.
Hello everybody! I add one more Clojure implementation on the website http://www.todobackend.com/. This time using a more recent "trending" stack with Reitit and Next-jdbc. You can check the repo directly here: https://github.com/PrestanceDesign/todo-backend-clojure-reitit I think that can be useful. Cheers!
Thanks. Hmmm upgrade to Windows 10 with the new laptop will be next year. I guess lein.bat is sufficient for now to learn clojure basics.
A lot of the tutorials out there still use Leiningen as well as many of the books, so you should be fine with lein
for learning. Windows is a bit of a second-class citizen in the Clojure world, so even with Windows 10, it's a lot easier to do everything using WSL2 which is how I work with Clojure: WSL2 with Ubuntu, Clojure CLI, use VcXsrv (Xlaunch) on Windows to support X11 apps, then run everything on WSL2, including my editor.