Can I create a really tiny (in RAM & cpu usage) REST sever with clojure in some way?
I've done some benchmarks, httpkit can run very fast with 10M of RAM.
Everything fails around 8M
httpkit also compiles with graalvm so you could run this natively. See also here: https://github.com/kloimhardt/bb-web
I read that graalvm doesn't really use less memory at runtime. I haven't confirmed myself.
Let's put it to the test.
$ time clj -J-Xmx5m -e "(+ 1 2 3)"
6
clj -J-Xmx5m -e "(+ 1 2 3)" 2.88s user 0.17s system 232% cpu 1.309 total
avg shared (code): 0 KB
avg unshared (data/stack): 0 KB
total (sum): 0 KB
max memory: 108348 MB
$ time bb -Xmx5m -e "(+ 1 2 3)"
6
bb -Xmx5m -e "(+ 1 2 3)" 0.03s user 0.01s system 93% cpu 0.049 total
avg shared (code): 0 KB
avg unshared (data/stack): 0 KB
total (sum): 0 KB
max memory: 33488 MB
YMMV with longer running apps that do more stuff. Benchmark needed.
I think the comment I saw did not figure out how to set max memory on a graal binary. I'd love to test this myself.
This is really intriguing for me because it's a path to sneak in some clj at the dayjob without risking much for the company.
And I really, really dislike coding PoCs in Go with wildly changing requirements.
What would Xmx translate to for a native binary?
In my situation we run everything as some kind of container workload either via docker
or some other cgroupy thing - wonder if I could get graal to respect that in some way?
@gbson Just put -Xmx10m
as a command line arg
Wow, do you know how that works? 😄
that's pretty cool
yes, graalvm native images respect this: e.g.:
$ bb -Dmy.prop=123 -e '(System/getProperty "my.prop")'
"123"
(bb = babashka)
yes I am familiar with your borking great work!
I loev it
I wonder if this stuff works for Graal: -XX:+UseCGroupMemoryLimitForHeap
$ bb -XX:+UseCGroupMemoryLimitForHeap
error: Could not find option 'UseCGroupMemoryLimitForHeap'. Use -XX:PrintFlags= to list all available options.
I think native can go smaller because the program isn't allocated to memory by java anymore, so there's a little extra space
Would be interesting to see if it could be measured as a whole
Then again, java binary is probably larger anyway.
If you want to know how GraalVM wins in terms of memory, there’s a podcast with the main author
Ooh, that sounds interesting. Where is that?
Graal or native image, just to clarify?
actually I listened to this one: https://airhacks.fm/#episode_78 but the above one might also be interesting
it's about both native and JVM
Preliminary testing: openjdk is the fastest when you have a healthy amount of heap (eg 128M). At lower quantities (8/9/10) native dominates, even when using a fallback binary. However, I'm yet to determine if the cost gets eaten into the binary, so the overall memory usage is potentially the same.
Fallback native image is 340M memory with 5M heap, so I'll have to dig in I think :)
fallback isn't really native, it's just a thing with a jvm in it
best to disable that
@dominicm Recommended flags:
"-H:+ReportExceptionStackTraces"
"-J-Dclojure.spec.skip-macros=true"
"-J-Dclojure.compiler.direct-linking=true"
"-H:ReflectionConfigurationFiles=reflection.json"
"--initialize-at-run-time=java.lang.Math\$RandomNumberGeneratorHolder"
"--initialize-at-build-time"
"-H:Log=registerResource:"
"-H:EnableURLProtocols=http,https,jar"
"--enable-all-security-services"
"-H:+JNI"
"--verbose"
"--no-fallback"
"--no-server"
"--report-unsupported-elements-at-runtime"
Thanks, I'll admit I'm struggling right now 😅
I'll probably stop by #graalvm
Reporting back, the RSS is now closer to ~15M, now that I've solved the fallback issue.
RSS?
Memory in RAM as reported by Linux.
ah
On low amounts of memory, native image without fallback completely falls apart. 133 req/s with http-kit under 10M of memory for native, but under OpenJDK that's 8708 req/s. However, the fact is that it depends on how you're hosting. On a shared java host, you might prefer OpenJDK as the heap is more important than the libraries that are brought in. On a VM (ec2, openvz, kvm, etc.) you're more interested in overall memory, so you can give native-image an extra 20M and be fine.
Having said that, Linux only virtually loads the whole java binary, so in theory you might get away with it (albeit with a performance hit on actual low-memory machines where you end up in swap or back on disk).
I only need to serve a few requests per second and the IO is mostly waiting for other service calls. The payloads are tiny
Really tiny for a clojure app might be 128 mb. What is really tiny for your uses?
The competing idea is writing it in go
128 mb should be OK
I guess the other possibility would be ClojureScript, compile to JS and target Node perhaps?
There is also a #graalvm channel where folks work on compiling Clojure to native binaries that I believe have smaller requirements than running on the JVM, but I don't know how they compare to ClojureScript. Worth asking there.
@gbson you could make an exceptionally small single-file runtime using clojurescript on node with e.g. shadow-cljs and then compiling to a single binary with https://github.com/zeit/ncc - the one requirement would be that the end user has a node binary on their system. if that's not a blocker for you, i have done similar things before so let me know if you want any tips on setting it up.
and if you don't want to require a node binary and you know the target architecture you can pkg
to bundle your script + node together: https://medium.com/@tech_girl/deploy-your-node-js-application-as-a-single-executable-4103a2508dd7
liveview - wondering if there’s something like that in clojure, please? seems like this project has stalled https://github.com/prepor/liveview-clj
https://github.com/tatut/ripley (alpha, though)
@flowthing nice, thank you!
it is not staled. we are using it in production
@delaguardo looking at the examples, it seems like it’s sending the whole page html thru the websocket on every event. does that work for you the same way in production?
yes, it was designed to send entire page
@delaguardo hmm is there a reason for that? isn’t one of the main points of live view that it’s sending just the diff?
on client side it is using simple morphdom library to rerender dom nodes that needs to change
diff is tricky. you have to take in account the time between send action from client + render + send diff back. during that range client’s state can be changed bu user.
that could make this diff obsolete
that was the main reason to send entire page as a snapshot of current state
hmm. liveview actually designates dynamic “slots” in the template and then is sending just the values for these slots basically. not full html.
it’s also using morphdom library, just minimizing the payload to almost nothing
@gbson you could also consider fennel-lang, a single binary with the whole lua interpreter + fennel bundled would hover around 400Kb + size of your code (once "compiled" to lua)
yeah, I know about difference between phoenix’s liveview. the goal for that library was simplicity and predictability not minimization of payload. Also it was written in few days )
not even sure you can go that low with hello world in golang
also you could check my PR — https://github.com/prepor/liveview-clj/pull/1/files it adds ability to register event handlers on the client to react on changes coming from backend
then sure, you'll need to add some dependencies to that, and it's definitely not clojure, even if it has some ressemblance
@delaguardo nice!
this “feature” theoretically should give you ability to send diffs but without any helpers. If you will need something — let me know, will try to help
@delaguardo thank you!
@gbson Someone is experimenting with adding httpkit server + reitit to babashka here:
https://github.com/kloimhardt/bb-web
It can run with low memory (like 5-30mb). bb supports JVM flags like -Xmx
Not sure if this will be in bb mainline, but it could be added under feature flags, or it could appear as its own bb spin-off. Either way, in that repo there are some downloads available.
One thing I am considering is adding httpkit server to babashka for the next release, but not sure yet about a routing lib.
There's a build of babashka with httpkit server available here: https://github.com/borkdude/babashka/issues/556. You'd have to do your own routing with this. Please leave a note in this issue if you're interested.
So I have a CLI app that I’m running as an uberjar. Is there anything I can do, apart from :aot :all
, to make it start faster?
FWIW my long-term plan is to compile it using GraalVM, but I’m looking for a simpler solution that I can use for the time being
@wombawomba Does your CLI use any special libs beyond clojure itself?
Well, not really… I sometimes run other tasks in the same project, and those do have a ton of dependencies, which do end up in my uberjar. But the parts I use for the CLI only depend on some libraries I’ve written myself, which in turn have no essential dependencies apart from Clojure itself
@wombawomba Then I think babashka will be a great fit
It can even run uberjars, but also from source
Startup is milliseconds
of course you would say that 😉
I’ll give it a shot, thanks!
Yeah, well sorry, this is exactly the use case for it :)
@borkdude any suggestions on how to generate the --classpath argument for a lein project?
export BABASHKA_CLASSPATH=$(lein --classpath)
thx
sorry, without the hyphens, so lein classpath
yup
but if you just want your source on the classpath then -cp src
suffices
yeah, I tried that but it turns out I have a few dependencies that do need to be loaded (e.g. clojure.match)
anyway, now that I’ve build an uberjar, I’m getting:
----- Error --------------------------------------------------------------------
Type: clojure.lang.ExceptionInfo
Message: Could not resolve symbol: definterface
Location: clojure/core/match/protocols.clj:39:2
Phase: analysis
----- Context ------------------------------------------------------------------
35: (syntax-tag [this]))
36:
37: ;; markers
38:
39: (definterface IExistentialPattern)
^--- Could not resolve symbol: definterface
40:
41: (definterface IPseudoPattern)
ok, core.match isn't in bb.
ah, got it
(yet, there could be support for it, but I haven't heard many people ask about it)
so I’m not really using core.match, I just happen to require an ns that requires it (for a fn I don’t use) — is there a way to work around this issue?
@wombawomba one way: bb supports .cljc + :bb
so you can exclude parts using reader conditionals.
ok sweet, I’ll give that a shot
that looks interesting, bb + httpkit + reitit
I think direct linking helps with startup time. Also I have been playing around with openJ9 and shared classes and it significantly helps startup times.