planck

Planck ClojureScript REPL
pepas 2019-02-10T21:03:27.014200Z

whoa!!!

root@raspberrypi:~# time echo ':cljs/quit' | planck 
ClojureScript 1.10.516

real	0m0.111s
user	0m0.100s
sys	0m0.010s
root@raspberrypi:~# time echo ':cljs/quit' | planck 
ClojureScript 1.10.516

real	0m0.062s
user	0m0.050s
sys	0m0.010s
root@raspberrypi:~# time echo ':cljs/quit' | planck 
ClojureScript 1.10.516

real	0m0.093s
user	0m0.080s
sys	0m0.000s

pepas 2019-02-10T21:03:37.014500Z

that's on a raspberry pi 3

borkdude 2019-02-10T21:04:14.014700Z

thatโ€™s pretty good right?

pepas 2019-02-10T21:05:02.015Z

that's really fast!

pepas 2019-02-10T21:05:34.015400Z

clojure startup time on the same machine is 16 seconds ๐Ÿ™‚

mfikes 2019-02-10T21:07:14.016400Z

Unfortunately, what you are seeing there is simply the time to shut down without really needing to wait for the ClojureScript (JavaScript) runtime to initialize.

pepas 2019-02-10T21:07:25.017Z

oh ha ๐Ÿ™‚

mfikes 2019-02-10T21:07:48.017800Z

This is part of what makes Planck appear to be fast: The C part of it is up immediately, allowing you to type.

borkdude 2019-02-10T21:08:02.018200Z

Thatโ€™s what I expected too. Try time plk -e '(+ 1 2 3)'?

pepas 2019-02-10T21:08:28.018400Z

root@raspberrypi:~# time echo '(+ 1 1)' | planck 
ClojureScript 1.10.516
2

real	0m17.065s
user	0m17.210s
sys	0m0.270s

pepas 2019-02-10T21:08:29.018600Z

ah

pepas 2019-02-10T21:08:57.018900Z

ah well, repl-drive development as they say ๐Ÿ™‚

pepas 2019-02-10T21:09:25.019200Z

is planck compatible with something like SLIME?

mfikes 2019-02-10T21:09:38.019400Z

Another test is time echo '(js/PLANCK_EXIT_WITH_VALUE 1)' | planck

mfikes 2019-02-10T21:10:06.019600Z

For Emacs integration, see http://planck-repl.org/ides.html

๐Ÿ‘ 2
pepas 2019-02-10T21:10:55.020400Z

Unfortunately I ran into an illegal instruction while building on an OLPC (i586)

### AOT compiling macro namespaces
+ mkdir -p planck-cljs/out/macros-tmp
+ planck-c/build/planck -sk planck-cljs/out/macros-tmp '-e(require '\''cljs.analyzer)' '-e(do (set! cljs.analyzer/*cljs-warnings* (assoc cljs.analyzer/*cljs-warnings* :undeclared-var false)) nil)' -e '(require-macros '\''planck.repl '\''planck.core '\''planck.shell '\''planck.from.io.aviso.ansi '\''clojure.template '\''cljs.spec.alpha '\''cljs.spec.test.alpha '\''cljs.spec.gen.alpha '\''cljs.test '\''cljs.pprint '\''cljs.analyzer.macros '\''cljs.compiler.macros '\''cljs.env.macros)'
./script/build: line 154: 21317 Illegal instruction     planck-c/build/planck -sk planck-cljs/out/macros-tmp -e"(require 'cljs.analyzer)" -e"(do (set! cljs.analyzer/*cljs-warnings* (assoc cljs.analyzer/*cljs-warnings* :undeclared-var false)) nil)" -e "(require-macros 'planck.repl 'planck.core 'planck.shell 'planck.from.io.aviso.ansi 'clojure.template 'cljs.spec.alpha 'cljs.spec.test.alpha 'cljs.spec.gen.alpha 'cljs.test 'cljs.pprint 'cljs.analyzer.macros 'cljs.compiler.macros 'cljs.env.macros)"
+ checkCmdSuccess
+ CMD_RESULT=132
+ '[' 132 '!=' 0 ']'
+ echo 'Build Failed.'
Build Failed.
+ exit 132

mfikes 2019-02-10T21:12:05.021800Z

The nice thing about what Planck is doing there, is if you start the Planck REPL, oftentimes it takes you a few seconds to actually enter the first form, so it is nice to have the textual UI up and running before you need to evaluate. The latest shipping ClojureScript (non-browser) REPLs now play the same trick.

pepas 2019-02-10T21:12:46.022500Z

ah neat

mfikes 2019-02-10T21:13:07.023Z

Yeah, if you hit an illegal instruction, then it is probably a compiler defect, and the only recourse is to pull out a debugger and get really low level.

mfikes 2019-02-10T21:13:26.023500Z

Meaning a gcc or clang compiler defect.

pepas 2019-02-10T21:13:30.023700Z

maybe something i686-specific, or SSE, etc?

mfikes 2019-02-10T21:14:10.024700Z

It could also be an illegal instruction in machine code generated by JavaScriptCore

pepas 2019-02-10T21:14:12.024800Z

is script/build --verbose verbose enough to show things like -march= and such?

pepas 2019-02-10T21:14:20.025Z

oh good point

mfikes 2019-02-10T21:15:21.025500Z

I bet you'd have to revise the build script to see more info

pepas 2019-02-10T21:15:43.025700Z

that should be pretty simple

pepas 2019-02-10T21:15:56.026300Z

its a bit of an odd beast, so I'm not surprised:

root@olpc# cat /proc/cpuinfo 
processor	: 0
vendor_id	: AuthenticAMD
cpu family	: 5
model		: 10
model name	: Geode(TM) Integrated Processor by AMD PCS
stepping	: 2
cpu MHz		: 430.954
cache size	: 128 KB
fdiv_bug	: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu de pse tsc msr cx8 sep pge cmov clflush mmx mmxext 3dnowext 3dnow
bogomips	: 861.90
clflush size	: 32
cache_alignment	: 32
address sizes	: 32 bits physical, 32 bits virtual
power management:

mfikes 2019-02-10T21:16:52.027Z

One nice aspect about Planck (for this sort of stuff), is that it is really just a small C wrapper driving JavaScriptCore. That makes it easy to mess around on small boxes.

pepas 2019-02-10T21:17:15.027200Z

yeah, very cool

pepas 2019-02-10T21:17:30.027500Z

thanks for writing it!

mfikes 2019-02-10T21:21:54.028800Z

@jasonpepas If it is indeed an illegal instruction caused by machine code generated by JavaScriptCore, some of those optimizations can be disabled by simply setting environment variables.

๐Ÿ‘ 1
mfikes 2019-02-10T21:25:00.030400Z

One interesting related flag that might speed up things on older hardware is to set JSC_useFTLJIT to true. I is currently defaulted to false in Planck to avoid crashing that was introduced in a recent version of JavaScriptCore.

mfikes 2019-02-10T21:26:21.031200Z

@jasonpepas For your illegal instruction problem, setting an environment variable JSC_useJIT=false would turn off all JIT compilation, and perhaps avoid the issue and allow the build to succeed.

borkdude 2019-02-10T21:31:31.031700Z

what are you going to use planck for on a raspberry?

pepas 2019-02-10T21:35:39.033500Z

Thanks @mfikes I'll give that a shot

mfikes 2019-02-10T21:35:47.033900Z

I'd really like to see this. One area that we haven't yet "reached" with ClojureScript is extremely tiny computers. For example, what if you wanted to prototype some small bit of hardware but drive it with ClojureScript. It seems like Planck / JavaScriptCore almost gets you there, but one huge issue I don't have an answer for is the fact that the core library is pretty huge.

pepas 2019-02-10T21:35:54.034100Z

@borkdude just little personal servers, like a wiki, etc.

pepas 2019-02-10T21:39:30.036500Z

@mfikes I was wondering if it would be possible / make sense to perform some sort of just-in-time eval for things like the core library? i.e. somehow insert placeholders into the environment which will trigger that code to get eval'ed the first time it is executed. Or does that not make sense in the context of something like clojure?

pepas 2019-02-10T21:39:43.036800Z

(or did you mean simply the disk size of the core library?)

mfikes 2019-02-10T21:41:25.037600Z

Oh, so if you ask the question: Is is possible to run a ClojureScript REPL in a device that, say, only has 256 MB of RAM, you start to encounter issues where the core library is too much to fit in there.

pepas 2019-02-10T21:41:50.037900Z

ok, gotcha, you just don't want the whole library in ram

mfikes 2019-02-10T21:42:21.038800Z

Yeah, there are nice tiny machines you can get with USB or other really simple connectivity, and it would be interesting to drive them with ClojureScript.

pepas 2019-02-10T21:43:11.040500Z

I'm tempted to ask if it would be feasible to implement something where forms could get kicked out of the environment and force them to be re-eval'ed from disk on next call, but I'm not sure that's actually any different than just loading everything into RAM and letting the OS use a swap file

mfikes 2019-02-10T21:43:16.040700Z

I think if you use :advanced you can produce a sufficiently small artifact that would work on those machines. (I think I did that with the Lego MindStorms brick at one point... but I can't recall). But having a REPL is the challenge with respect to RAM.

๐Ÿ‘ 1
mfikes 2019-02-10T21:43:50.041400Z

Yeah, avoiding loading the entire standard library, but lazily loading stuff via thunks might be a trick that would lead to it working.

mfikes 2019-02-10T21:44:42.042600Z

Planck does that, for example with parts of the analysis cache. It does that not to save RAM but to reduce startup latency in the case you don't need those portions of the cache.

๐Ÿ‘ 1
pepas 2019-02-10T21:44:50.042900Z

I wonder if that would speed up startup time as well, or is simply parsing the edn most of the cost?

mfikes 2019-02-10T21:46:01.044100Z

Yeah, I think that modern JavaScript engines even try to parse JavaScript in a way that defers as much as possible. So, it might not fully parse a big function until you call it. But I really don't know.

pepas 2019-02-10T21:46:15.044300Z

oh interesting

mfikes 2019-02-10T21:46:46.045Z

All, I know is: There are tiny machines that are very cool to do stuff with, and it would be nice to somehow cram ClojureScript into them ๐Ÿ™‚

pepas 2019-02-10T21:46:57.045200Z

definitely!

mfikes 2019-02-10T21:47:40.045800Z

I think I need to buy one and just try it out. ๐Ÿ™‚ They aren't too expensive I've heard. ๐Ÿ™‚

pepas 2019-02-10T21:47:59.046400Z

what's that, a lego mindstorms kit?

mfikes 2019-02-10T21:48:06.046600Z

Or I can do stuff like limit the RAM in a Unix shell.

mfikes 2019-02-10T21:48:23.047100Z

Oh, it is a way to essentially build programmable robots with legos.

pepas 2019-02-10T21:49:40.048200Z

yeah, actually I audited an EE course while working for UT and we built a little line-following robot using one of those

mfikes 2019-02-10T21:49:43.048500Z

It is essentially Lego's take on an Arduino

pepas 2019-02-10T21:49:49.048700Z

The visual programming language for it was pretty interesting

mfikes 2019-02-10T21:50:30.049700Z

Yep. I actually started a project to make an IDE for its language for the iPad. But never carried through with it.

pepas 2019-02-10T21:51:49.051200Z

Having already been through college and out in the working world, I had the benefit of being able to plan out a project and stay on task. I'll never forget that one of the teams made a robot which simple ran into a wall and sat there, while playing the imperial death march on its little piezo speaker. They had literally spent the entire project on making it play that song.

pepas 2019-02-10T21:52:08.051500Z

An iPad IDE would be fantastic!

borkdude 2019-02-10T21:59:42.052300Z

> All, I know is: There are tiny machines that are very cool to do stuff with, and it would be nice to somehow cram ClojureScript into them Yeah. Thereโ€™s also joker, which is a small Clojure interpreter built in Go. Iโ€™ve never really done scripting with that, but it might also run in tiny machines.

๐Ÿ‘ 1
borkdude 2019-02-10T22:01:37.053200Z

Itโ€™s probably very limited, not as far reaching as ClojureScript. I use it mostly as a linter, which is very cool imo.

pepas 2019-02-10T22:07:30.054800Z

I ran strace and I'm not sure if this is definitive, but it pauses at the futex( call for about 3 seconds before hitting the illegal instruction:

open("/root/.planck_keymap", O_RDONLY)  = -1 ENOENT (No such file or directory)
nanosleep({0, 1000000}, NULL)           = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0xbfbec148) = -1 ENOTTY (Inappropriate ioctl for device)
fstat64(0, {st_mode=S_IFREG|0600, st_size=8, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb771b000
read(0, "(+ 1 1)\n", 4096)              = 8
open("/root/.planck_history", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 4
fstat64(4, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb771a000
write(4, "(+ 1 1)\n", 8)                = 8
close(4)                                = 0
munmap(0xb771a000, 4096)                = 0
futex(0x843dce4, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
+++ killed by SIGILL +++
Illegal instruction

pepas 2019-02-10T22:07:42.055100Z

(somehow the build had completed, I suppose!)

mfikes 2019-02-10T22:13:28.059200Z

The build script creates a โ€œstage 1โ€ Planck executable that is then used to AOT compile shipping macros, to be included in a โ€œstage 2โ€ Planck executable. I bet this use of the stage 1 Planck failed during the build. But itโ€™s binary is there nevertheless for you to use, and evidently repro the SIGILL

pepas 2019-02-10T22:13:53.059400Z

oh interesting

pepas 2019-02-10T22:14:15.059800Z

so if the problem is in JSC it might exist in both stage 1 and stage 2

pepas 2019-02-10T22:15:26.060700Z

I tried JSC_useJIT=false but I killed it after a bout 5 minutes (not sure if it was stuck in a loop or if JS is really that much slower without JIT

pepas 2019-02-10T22:16:19.061Z

ah, gdb finally hit something

pepas 2019-02-10T22:16:22.061200Z

Reading symbols from ./planck-c/build/planck...(no debugging symbols found)...done.
(gdb) run
Starting program: /tmp/planck-2.20.0/planck-c/build/planck 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
[New Thread 0xb50c1b40 (LWP 22038)]
ClojureScript 1.10.516
[New Thread 0xb48c1b40 (LWP 22039)]
cljs.user=> [New Thread 0xb3effb40 (LWP 22040)]

Program received signal SIGILL, Illegal instruction.
[Switching to Thread 0xb50c1b40 (LWP 22038)]
0xb2a01475 in ?? ()
(gdb) bt
#0  0xb2a01475 in ?? ()
#1  0xb29fa1c4 in ?? ()
#2  0xb29ff088 in ?? ()
#3  0xb7dd55b4 in llint_entry () from /usr/lib/i386-linux-gnu/libjavascriptcoregtk-4.0.so.18
#4  0xb2a018bf in ?? ()
#5  0xb2ae6b62 in ?? ()
#6  0xb2aebe2b in ?? ()
#7  0xb7dd55b4 in llint_entry () from /usr/lib/i386-linux-gnu/libjavascriptcoregtk-4.0.so.18
#8  0xb7dd5601 in llint_entry () from /usr/lib/i386-linux-gnu/libjavascriptcoregtk-4.0.so.18
#9  0xb7dd12d8 in vmEntryToJavaScript () from /usr/lib/i386-linux-gnu/libjavascriptcoregtk-4.0.so.18
#10 0xb7b57f53 in JSC::JITCode::execute (this=0xb26c3840, vm=0xb4079000, protoCallFrame=0xb50bb524)
    at /build/webkit2gtk-u7EGwB/webkit2gtk-2.6.2+dfsg1/Source/JavaScriptCore/jit/JITCode.cpp:56
#11 0xb7b37c6c in JSC::Interpreter::execute (this=0xb408a5d0, program=0xb349fb00, callFrame=0xb369fa6c, 
    thisObj=0xb365fb60)
    at /build/webkit2gtk-u7EGwB/webkit2gtk-2.6.2+dfsg1/Source/JavaScriptCore/interpreter/Interpreter.cpp:928
#12 0xb7c701e9 in JSC::evaluate (exec=0xb369fa6c, source=..., thisValue=..., returnedException=0xb50bba90)
    at /build/webkit2gtk-u7EGwB/webkit2gtk-2.6.2+dfsg1/Source/JavaScriptCore/runtime/Completion.cpp:82
#13 0xb78e533f in JSEvaluateScript (ctx=0xb50bcd08, script=0xb307a960, thisObject=0x0, sourceURL=0xb307a970, 
    startingLineNumber=<optimized out>, exception=0xb50bbafc)
    at /build/webkit2gtk-u7EGwB/webkit2gtk-2.6.2+dfsg1/Source/JavaScriptCore/API/JSBase.cpp:66
#14 0x080779c5 in evaluate_script ()
#15 0x080746c7 in function_import_script ()
#16 0xb78e6fc8 in JSC::APICallbackFunction::call<JSC::JSCallbackFunction> (exec=<optimized out>)
    at /build/webkit2gtk-u7EGwB/webkit2gtk-2.6.2+dfsg1/Source/JavaScriptCore/API/APICallbackFunction.h:61
#17 0xb7b74a4b in JSC::handleHostCall (execCallee=execCallee@entry=0xb50bcd08, callee=..., kind=JSC::CodeForCall)
    at /build/webkit2gtk-u7EGwB/webkit2gtk-2.6.2+dfsg1/Source/JavaScriptCore/jit/JITOperations.cpp:666
#18 0xb7b778ef in JSC::linkFor (execCallee=0xb50bcd08, callLinkInfo=0xb327cfc0, kind=JSC::CodeForCall, 
    registers=JSC::RegisterPreservationNotRequired)
---Type <return> to continue, or q <return> to quit---

pepas 2019-02-10T22:18:02.062200Z

looks like it is definitely happening in JSC

pepas 2019-02-10T22:20:02.062600Z

hmm, there are no shortage of hits which go back to debian: https://www.google.com/search?q=libjavascriptcoregtk+illegal+instruction

pepas 2019-02-10T22:22:21.063100Z

this guy traced it to a movsd instruction: https://groups.google.com/forum/#!msg/linux.debian.bugs.dist/qTh7bOqmPsY/6emwKMV5BwAJ

pepas 2019-02-10T22:22:42.063500Z

looks like I'm hitting the same thing:

(gdb) display/i $pc
1: x/i $pc
=> 0xb2a01475:	movsd  (%ebx,%ecx,8),%xmm0

pepas 2019-02-10T22:23:15.063900Z

ah, damn, the patch which was attached to that message is now 404: https://anonscm.debian.org/cgit/pkg-webkit/webkit.git/tree/debian/patches/disable-jit-nonsse2.patch

pepas 2019-02-10T22:25:04.064100Z

looks like this might be it: https://sources.debian.org/patches/webkitgtk/2.4.11-3/

mfikes 2019-02-10T22:50:44.064400Z

Sweet. Nice work. ๐Ÿ™‚