Clojurians Log v2

@cfleming: quick question. Sometimes the REPL slows way down, often when spooling lots of information to the repl window. Do you know what the technical reason is for that? Does the REPL not release the memory allocated for spooling the data to the screen?

meaning, would not printing things fix this issue? or is it more related to mapping over very large lists (30-50k records)

@erichmond: I’m not sure, I haven’t seen that. One problem I’ve had recently is that some of the regexes used for the stacktrace folding were very prone to backtracking, I’m planning to fix that soon.

a REPL fired up from the terminal doesn’t seem to have these same constraints

Do you have soft wrap on?

Ok

Yes, I do

sorry, I explained my issue horrible 😞

horribly*

Hmm, so several people have reported that soft wrapping slows things down a lot.

<— needs to think before he types

aha

ok

I will try turning off soft wraps

Could you try without that enabled and see if it helps? Everything will be unreadable obviously, but just to test.

Sure!

Also, to reframe my question into something MUCH clearer. The issue I am seeing is that I am mapping over 30-50k records, and printing out some checkpoint info (doing an ETL). If I do this in a normal repl fired up via the terminal, outside of IntelliJ+Cursive, this process runs fine.

When I try doing this inside IntelliJ+Cursive while using a remote REPL (my app is running on a vagrant VM, so I am remotely connecting to a headless REPL running in vagrant), execution eventually slows and essentially freezes, causing the editor to beachball, at which point I need to force quit IntelliJ and restart.

I was curious if you had seen this behavior before, as I could imagine it being a number of issues: 1) all the printing to the REPL is causing some kind of memory issue 2) realizing a seq that large is messing with the REPL 3) Sending that much data over a remote REPL is causing the slowdown.

s/being/could be/

Anyway, I will test without soft-wrapping. Thanks again!

@erichmond: Thanks for the clarification. I’ll see what I can come up with to test this. I might be able to use test.check generators to generate a ton of random data structures, print them out and see what happens.

I’m jammed up having to get this ETL done by today, but once it’s done, I can also go back and futz around

@erichmond: No problem, I’m travelling at the moment anyway

@cfleming: I was able to play around with this more. It turns out the issue is writing a ridiculously large string to the REPL will cause the REPL to slow down. If I then run the “clear repl” command, the REPL speeds up again

that actually makes sense to me, and is pretty easy to fix on my end (don’t return ridiculously large strings) heh.

How large is ridiculously large?

About 25MB give or take a few MB COUGH

smells the air … smells like user error

heh

Problem exists between chair and keyboard?

exactly

hehe yeah the same problem will happen in emacs if you do that >_>, or most any text editor, I’d imagine