Clojurians Log v2

Can you give an indication of what it was before and what it is right now on master?

the startup time

and in ms?

32ms on my Ryzen 3 down to 5ms. All approximations, of course.

impressive!

when this lands in a release, I'll try it out on my own machine and reflect this in a slide deck where I compare various Clojure scripting options

right now I've taken timings from a blog post by Stuart Sierra

if you have a macOS binary around with these speedups, I'd be happy to give it a spin

Not handy right now, but could get one to you in the next hour or so...but how?

as a zip file here on Slack maybe?

other way: my e-mail is <mailto:michielborkent@gmail.com|michielborkent@gmail.com>

I build binaries on CircleCI where I can get download links to the artifacts. I also sometimes just pass those links to people

Gmail has strict attachment size limits. Will try here.

Or, hey, could just post to my own web server and send you a link! 😀

sure! 🙂

Gimme an hour or less.

no hurry, the presentation is next week

Ooh, what presentation??

about babashka

Cool! Please share a link or something here. Is it a public event or similar?

it is the ClojureD conference

https://clojured.de/schedule/#michielborkent

it seems that the fast one is 5 MB bigger, is that because of more pre-compiled code?

Yes, that’s a 25-33% increase in size or so, pretty consistent across my 3 architectures IIRC.

===> multitime results
1: ./joker.fast -e "(+ 1 2 3)"
            Mean        Std.Dev.    Min         Median      Max
real        0.017       0.002       0.015       0.016       0.039

that's about what I get with bb too

for reference, I ran:

$ multitime -n100 -s0 ./joker.fast -e '(+ 1 2 3)'

so I guess I can just say: bb and joker have comparable startup time 🙂

Well that’s pretty good (for Joker) I guess! But bb has all the JVM libs available to it? Is there a list of “big” stuff missing in bb versus Joker, that we can compare to a similar list for Joker?

it doesn't have all classes, only those that we selected: https://github.com/borkdude/babashka/blob/7b30927a12608df6fe02fdd522c077fd9fe3eadd/src/babashka/impl/classes.clj#L15

At quick glance that seems more comprehensive than Joker…is there anything Joker has that bb doesn’t offer?

Also I would appreciate your input on this thread, starting with my comment the other night:

https://github.com/candid82/joker/issues/356#issuecomment-588955555

I think joker has a few things that bb has not, like a built-in http client, xml and yaml, which are on the list for bb.

we do have support for a user-land clj-http library

tl;dr: We can improve joker, we can improve bb, etc., but what is the 30K-foot overview of what the most promising combination of implementation language likely to be?

Does bb do multithreading better than Joker?

bb offers real threads like you're used to from clojure, it has pmap, future, agents, promises, etc

For classic scripting, that’s probably fine; but there are various use cases I could imagine for Joker, even as a scripting language, that bb would be better suited for simply because it has the real Clojure-style MT.

I think my life of adding stuff from the clojure eco-system inside bb is much easier: if I'm lucky, I can just stick a function inside a map and I'm done

this is basically the entire clojure.core namespace: https://github.com/borkdude/sci/blob/master/src/sci/impl/namespaces.cljc

That’s great! Offhand, do you have a sense of how GraalVM is doing in terms of stability, performance improvements, roadmap going forward, etc.? I was following the mailing list for it for awhile until early last year when things got real busy.

GraalVM has LTS releases and non-LTS releases, so in that regard they try to offer more stability I guess

they also have a Slack where you can ask questions to users and developers. they are quite responsive to issues on Github as well

As you’ll see from that discussion in the Issue on main programs in Joker, I’m very interested in whether Go, or GraalVM, or Rust, or whatever, would be enough-better, in terms of improvement over where we are right now, to justify a) continued work on Joker (in particular I have my gostd branch to auto-support the Go std libraries plus arbitrary 3p ones, but much more work is needed there), b) rewriting/rehosting Joker to another language, c) leaving Joker as a quick linter and moving to something like bb, or d) some other possibility I haven’t considered.

babashka is basically split up into two libraries: sci (the interpreter) and edamame (the code parser). other users can make something like bb but geared towards more specialized goals, like generating static websites. one such tool is called bootleg: https://github.com/retrogradeorbit/bootleg, it has hiccup, templating, etc.

(Right now, even without MT, Joker plus gostd would meet most of my needs beautifully.)

I didn't quite understand the thread about -main. I do support -main functions in bb just like the clojure cli does

bb also understands classpaths as produced by clojure so you can use some normal JVM libs without any change: https://github.com/weavejester/medley is an example of this

I’ll make some time to look over the bb ecosystem soon then, as it came to my notice many months ago (probably due to one of your tweets) and I’m intrigued by the idea of a “more-Clojure-like-Clojure” that starts up very quickly and doesn’t have GB of VM overhead like Clojure does for even a simple service.

That sounds great!

joker is an enormous inspiration for both clj-kondo and babashka so I do follow what happens in this ecosystem too. I hope we can benefit from each others work.

If Joker had gostd (i.e. ready access to the Go std libs), drop-in core libraries (which I think we can support without too much work), and maybe “true” MT, would you see it as being worth using versus where you see bb headed?

I'm not familiar with the Go ecosystem at all. I see it as a plus that I get to write Clojure while writing bb and also while using it 🙂

Oh, just saw your response about -main — I pointed to the part in that thread where I respond to @roman.bataev’s query about a potential Joker compiler and explain why I’m not sure that’s worth doing, even though I think about it a lot and would surely enjoy working on one. Could be in that “we can do it, but should we?” category.

the sci interpreter (which bb is based on) also runs on JS btw, there are already some projects leveraging this for e.g. Slack bots in a Google cloud function

ah, about compiling: I think it's nice to not have to compile, for scripting, like you do with bash. Just text, no steps in between

It’s funny, now that I’ve worked on Joker pretty much exclusively (during my free time) the past 18 months or so, my Clojure knowledge has probably declined somewhat since I’m not actually doing that much programming in it. I.e. most of my Clojure programming is Joker code in support of building and testing Joker, not real-world stuff. Haven’t reviewed my Clojure flashcards in over a year….

there is a place for everything I guess. CLJS does compilation to JavaScript which then can be quite fast on Node regarding startup, but the step in between is what it makes less attractive for scripts imo

Yeah, I think @roman.bataev is most interested in behind-the-scenes, versus “overt”, compiling. I.e. not compiling a script or lib/namespace down to an artifact, but JIT’ing interpreted code in the background. Whereas I’d like to have the static-compile-to-an-artifact feature more, I think!

for the latter I think GraalVM is very well suited

Oh, interesting! Somehow I thought GraalVM was primarily a JIT.

GraalVM comprises several parts: Truffle/Polyglot and native-image. I've only used native-image. What it does: translate .class files to native binary. Once it's compiled there is no JIT anymore.

Really gotta climb up the stack, now that the fast-init stuff is done (and after I get my taxes done 😉 ), and learn more about all these technologies at an architectural level soon.

if you have any questions, feel free to reach out. congrats on the startup!

Thanks, will do! And have a great presentation next week!

there's also a #graalvm channel and some docs on graalvm here: https://github.com/lread/clj-graal-docs

thanks!

We can probably take this to the International Unofficial Clojure Implementation Summit, together with Clojerl 😉