clojure

New to Clojure? Try the #beginners channel. Official docs: https://clojure.org/ Searchable message archives: https://clojurians-log.clojureverse.org/
Gleb Posobin 2020-12-19T20:07:45.239600Z

What tools are there to find a memory leak?

Gleb Posobin 2020-12-20T08:20:37.247500Z

Server's memory usage grows daily, becomes 2x the initial after a couple of days.

borkdude 2020-12-20T08:29:59.247800Z

Have you called (System/gc)? Does it continue to grow?

Gleb Posobin 2020-12-20T08:38:13.248Z

I haven't, but it OOMs at some point soon afterwards, so I assumed it would have tried that path.

borkdude 2020-12-20T09:34:57.253700Z

That makes sense yes. It could me many things. I've had an issue with memoize once because memoize forever holds on to all of its arguments. This caused a memory leak in clj-kondo.

jumar 2020-12-20T10:01:34.262200Z

That memory consumption grows it’s normal but GC should be able to deal with that eventually assuming the app has enough memory for what it’s supposed to do (no need to call System/gc) If it’s growing beyond reasonable limits then you can ise jstat or jcmd for quick inspection -is it heap that’s growing or other parts of jvm memory ? If heap then it’s useful to use jcmd to take a heap dump and analyse it as the guys already suggested. clj-async-profiler also supports profiling allocations but I haven’t used that part much

👍 2
2020-12-21T21:34:03.302Z

You can also use https://www.eclipse.org/mat/ to analyse a heap dump.

2020-12-19T20:09:04.239700Z

Clojure or ClojureScript?

Gleb Posobin 2020-12-19T20:09:28.239900Z

Clojure.

2020-12-19T20:10:19.240100Z

If Clojure on the JVM, then there are free and commercial tools for analyzing all live objects in a running JVM, or in a dump file created from a running JVM, e.g. YourKit Java Profiler (which can also help with profiling run time of code), which is commercial, but free trial period, and I believe also free for use on open source project development.

2020-12-19T20:11:14.240300Z

There are several others listed on this page: https://dzone.com/articles/how-to-capture-java-heap-dumps-7-options

2020-12-19T20:12:14.240800Z

I haven't done this in a production setting before, so am not a good person to recommend which of these tools are best for your purposes.

Gleb Posobin 2020-12-19T20:14:20.241Z

Yes, clojure on JVM.

Gleb Posobin 2020-12-19T20:14:55.241200Z

I thought someone might recommend some tool they have used.

borkdude 2020-12-19T20:15:20.241400Z

You could also try VisualVM which is free

Gleb Posobin 2020-12-19T20:18:32.241600Z

I saw yourkit before, but don't have any money to pay, and probably would want to use it in the future too. I haven't seen VisualVM, it seems nice. Thank you!

clyfe 2020-12-19T20:19:20.241800Z

YourKit is free for open source projects use

Gleb Posobin 2020-12-19T20:25:53.242Z

I saw that, my project is not open source 😃

jmckitrick 2020-12-19T22:19:45.243100Z

What’s the simplest way to get a csrf token into html if I am not using html templating like selmer?

rakyi 2020-12-20T08:39:58.248200Z

You don’t have to put it in HTML. You can use cookie to send it to client and client can then add it to a separate header when making request to backend.

flowthing 2020-12-20T09:04:45.249700Z

Browsers don’t allow you to set headers for WebSocket handshake requests, IIRC.

flowthing 2020-12-20T09:06:05.250900Z

You could consider Origin header checking instead of CSRF tokens.

flowthing 2020-12-20T09:14:30.253400Z

But yeah, if you want to use the token, instead of embedding the token in the HTML, you can also serve it via a separate REST endpoint.

clyfe 2020-12-20T10:57:45.262800Z

From OWASP CSRF Cheat Sheet "CSRF tokens should not be transmitted using cookies."

rakyi 2020-12-20T12:08:35.265600Z

Don’t know why they categorically say that without explanation, when on the same page they explain Double Submit Cookie https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html#double-submit-cookie which does exactly that and which I had in mind. For example Django (Python framework) supports this.

rakyi 2020-12-20T12:13:20.266300Z

> Browsers don’t allow you to set headers for WebSocket handshake requests, IIRC. Can you point me to some docs about this? Didn’t find anything like that. IIRC we use CSRF token in header for WebSocket handshake requests and it works fine at work.

rakyi 2020-12-20T12:19:08.267500Z

From https://tools.ietf.org/html/rfc6455#section-4.1 > The request MAY include any other header fields, for example, cookies [RFC6265] and/or authentication-related header fields such as the |Authorization| header field [RFC2616], which are processed according to documents that define them.

rakyi 2020-12-20T12:33:56.267800Z

I was curious and after more googling it appears browsers actually don’t implement the spec, i.e. restrict the headers sent to servers. So I must have misremembered.

flowthing 2020-12-20T12:48:47.269800Z

Yes, I only learned that recently myself. Incidentally, I wrote a blog post on the topic of WebSocket endpoints and CSRF a couple of years ago: https://dev.solita.fi/2018/11/07/securing-websocket-endpoints.html

jmckitrick 2020-12-20T13:17:10.270100Z

Excellent resources, thanks! I’m just trying to update an older app using sente, and need this fix to plug a hole.

jmckitrick 2020-12-20T14:38:43.270500Z

I’m trying (force ring.middleware.anti-forgery/*anti-forgery-token*) and getting #object[clojure.lang.Var$Unbound 0x44a017ac "Unbound: #'ring.middleware.anti-forgery/*anti-forgery-token*"] as the result. That doesn’t make sense. Shouldn’t I be getting the value of the token itself? Is there some function that seeds the token at startup that’s missing?

jmckitrick 2020-12-20T15:20:22.271800Z

Well, what’s weird is I’m running an app from a generated template (Luminus) and right in the html it looks like this:

clyfe 2020-12-20T15:21:20.272Z

@rakyi I imagine the no-cookie note is because it's then hauled around on more requests (get), not just those you want to csrf protect.

jmckitrick 2020-12-20T15:23:34.272200Z

clyfe 2020-12-20T15:24:27.272600Z

is anti-forgery configured on in middleware?

jmckitrick 2020-12-20T15:26:18.272800Z

Yes. It’s enabled by default as part of the template.

jmckitrick 2020-12-20T15:27:13.273Z

And even if it weren’t, the variable should still have a value, no? Maybe not, but it’s irrelevant, since this feature and middleware are enabled by default in Luminus templates that use web forms.

jmckitrick 2020-12-20T15:31:14.273400Z

It appears I’m wrong… I disabled part of the csrf middleware for my local testing. But that shouldn’t prevent the token from generating, right? I’m rebuilding and testing again to see what happens….

jmckitrick 2020-12-20T15:33:05.273600Z

Same result as before. The page source shows the token as ‘unbound’

clyfe 2020-12-19T22:25:40.243200Z

What are you using?

jmckitrick 2020-12-19T22:26:24.243400Z

Just serving html, which pulls in a CLJS payload to start re-frame

jmckitrick 2020-12-19T22:27:03.243600Z

I’d rather not pull in selmer just to serve one file with just one injected element.

clyfe 2020-12-19T22:35:36.243800Z

How do you serve the html? via ring.middleware.file ? nginx public?

clyfe 2020-12-19T22:36:01.244Z

What do you use for csrf? ring-anti-forgery?

clyfe 2020-12-19T22:40:29.244200Z

Anyway, with re-frame you don't need the token in html, you can put it in, say, app-db, and acquire it with an ajax call. Subsequent ajax calls read it from there and add it as a header.

jmckitrick 2020-12-19T22:54:55.244500Z

Well, I’m serving the files with io/resource and I need the csrf token for the sente handshake, which I’m using for websockets. Sorry, I don’t use re-frame in this project… just reagent.