Clojurians Log v2

does anyone know what the 0[0-9]+ portion of the intPat regex in LispReader.java is supposed to handle? this is right before the (N)? at the end. https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LispReader.java#L71 i am working on tree-sitter grammars for clojure and trying to understand these types of things in detail.

octal?

guess not with 0-9

I guess it's just bigints with N suffix

?

The octal bit is 0([0-7]+)

That's weird because it has a leading 0 and the N is optional. but 09N is a syntax error, and 010N is read as octal anyway. Maybe it's a way to parse all possible integer tokens and then the syntax error comes later when the parsed tokens are processed?

Yeah, looking at the stacktrace, I think the regex is used to determine "this is a number" and then the token is actually read and that's when illegal numbers are flagged...? LispReader line 352 throws the Invalid Number for 09 etc.

i read the alternation as among: 0, base 10 things with leading non-zero, hex, octal, num with radix, and mystery item the mystery item doesn't have its own capture group so may be it is for some kind of parsing as suggested (or some other purpose). iiuc, apart from the whole capture, there are only 8 capture groups and i only see the numbers 1 through 8 mentioned in matchNumber, so whatever the mystery item is for, it doesn't look like its value is directly reflected in any resulting number. thanks for the examination.

you could feed that regex to the test.chuck generator and see if it produces anything that doesn't match the same regex with the mystery portion removed

thanks for the idea!

With the mystery portion removed, it appears that the regex will no longer match a string like "09"

Maybe the mystery portion is there so "09" does match the integer pattern, because otherwise such a string would fail to match it, and then the reader would try to match it against the float pattern?

possibly something like that

And I think the float pattern would match "09", and such a string would be read as a floating point decimal 9.

Yeah, quick test by modifying Clojure source code to get rid of the mystery pattern shows that (read-string "09") return a Double type 9.0, but Clojure 1.10.1 throws NumberFormatException

The current exception behavior for reading strings of digits with a leading 0, and at least one digit 8 or 9 in them, of throwing an exception, seems far preferable to returning a floating point number of the number read as decimal, given that all other strings of digits with a leading 0 are read as an octal integer.

@andy.fingerhut Thank you for taking the trouble/time/effort to actually verify what I suspected!

Do you know if prepl is going to see more development in the near future? I noticed a few people got really exited when it landed in the Clojure code but I haven’t heard any official announcements about it

@janne What development is needed? It was designed to solve a specific problem, and it does that well.

Various tooling developers are starting to use it -- it's kind of a niche feature that will really only appeal to tooling developers.

Oh, could you elaborate what is the specific problem?

I have only used nREPL and I know some (like you) like to use the socket REPL. I was thinking that prepl was missing some features because it’s not used by that many people

prepl is designed to be used by code, not people. It's designed for tooling.

It has no prompt. It returns a hash map of the result value, the output (if any), the error (if any) for each evaluation. Which is what a program wants when interacting with Clojure.

Some tooling authors have had issues with it and would welcome work on things like interruptible evals, nesting, etc.

Yeah, I think the lack of interruptible evals is probably the thing that would concern me most as a tooling author.

nREPL’s “interruptible eval” is totally busted

Calls Thread/stop, which is deprecated for good reason

I wonder how @mauricio.szabo does it in Chlorine? I'll go and ask...

It relies on unrepl. When an evaluation is submitted, it is run in a future, and returns a function to call if you want to (try to) cancel the future.

that makes more sense

@ghadi Do you think that is something that could be integrated into prepl? I get the impression prepl is purely synchronous at the moment, yes?

No plans to work on prepl at this moment, but that doesn’t really mean anything. Could be picked up at anytime

(there's also the issue of evaluation produces really large -- or even infinite results -- which I believe prepl just tries to return in full?)

I guess both of those are "hard" problems to solve on top of the current prepl architecture?

@seancorfield not unless either 1) eval is parameterized or 2) the client always sends in a wrapper form for their eval form

(^ re: interruptibility, not re: printing)

What’s the alternative?

abandon the computation

or check Thread/interrupt status

how could it do that with arbitrary code?

you don't

Out of curiosity, what does "abandon a computation" mean that is different than calling some Thread stop/cancel/whatever operation?

Thread/interrupt requires cooperation from the evaluated code

I agree that it’s busted, but the only safe option is cooperative multithreading which requires the running code to buy-in (at least checking interrupt status or yielding)

abandon means just don't wait on the result of the code

thread/stop at least stops the cpu from spinning forever

it's busted in conception, not implementation

but it's also busted in implementation

please read the doc of Thread.stop

I understand why thread.stop is bad

I would also be sad if nrepl didn’t use it 🙂

It seems to me maybe roughly the same amount busted as "kill -9" on a Unix process that is one of potentially many cooperating processes? I understand those aren't identical, but seems they could have similar coupling issues that affect things in a similar way, e.g. JVM object locks <-> Unix file system locks.

the surface area of object monitors seems to me much bigger than unix file system locks

you can always clear a file system lock problem

yeah

but a jvm with thread monitors in a busted state is just broken and leaking/deadlock prone forever

and every object is suspect that that thread ever referenced

You can delete the lock file, but problems of system invariants being violated can persist past deleting it, just as for object locks. Agreed if surface area here means JVM object locks are used far more often than Unix lock files, but not sure I see a fundamental difference between the kinds of problems that result.

breaking core.async channels is a big risk too

how do you stop (deref (promise)) ?

oof even unrepl calls .stop on the thread

there isn't anything else

there is no other mechanism to stop a thread doing arbitrary things

there is abandoning the eval, or calling Thread/interrupt (which requires cooperation)

right

for eval, which is the most general of computations, there is no way to interrupt it other than calling thread.stop, as deprecated as that is

Maybe Thread.stop() is as useful/expedient in development situations, and has many possible sharp edges, just as def of a function in a running REPL is?

We don't think twice if we do it, because we know how to handle our chef's knives without cutting ourselves, but we are always worried others will be cutting themselves?

I said this in a thread earlier, but repeating here in case it is useful to anyone. I don't claim it is deep or anything: Maybe Thread.stop() is as useful/expedient in development situations, and has many possible sharp edges, just as def of a function in a running REPL is? We don't think twice if we do it, because we know how to handle our chef's knives without cutting ourselves, but we are always worried others will be cutting themselves?

The plan is to actually remove it in the jvm soon

Wow. For the JVM, that is really saying something.

I think people don’t realize how dangerous it is

^^

if you have a runaway computation initiated through the REPL, who cares? if you stop a Thread and your whole app breaks, that is worse IMHO

I can't count how many times have I hit Ctrl-C Ctrl-B (interrupt) in Emacs and everything breaks

Actually they did already remove destroy() and one of the stop() methods in Java 11

I sure hope they leave kill -9 in Unix 🙂

thanks for the investigation (and the previous tips for compiling clojure from source). i also built a modified clojure and was able to see for myself :thumbsup:

Thread interrupts work fine for a large amount of usecases

but not something like infinite seqs (reduce + 0 (range))

(although if you tried to print an infinite seq, you could handle that through an interrupt)

good to know. So without Thread.stop, a thread that doesn't respond to Thread.interrupt(), and hasn't been written to poll some state asking it to quit, is only stoppable via killing the entire JVM process?

I think that's right

no problem. You made me curious what was going on, and thankfully it didn't take long to find out.

I'm more interested in: how can I give input control back to the initiating REPL? how can I surface a long computation? than stopping the long computation

the answer is to trick out your eval function

wrap clojure.core/eval

so you are saying, when you accidentally evaluate code that is copying the same data to a new file over and over again in infinite loop, you are more interested in "surfacing" and making that computation "visible", not stopping it?

"stopping" it

like, I think it is one thing to explain and understand why Thread.stop is a problem, but is kind of nuts to tell people they are wrong for wanting to stop bad loops running while doing iterative developement in a repl