Clojurians Log v2

I just spent the last 30 years of my day trying to figure out why my app won't compile properly when using a reitit ring handler and updating clojure to 1.10.2-alpha1 fixed it.

congrats. seems 1.10.2-alpha1 has solved a major blocking issue for us GraalVM native-image tinkerers

Might be a long-shot, since it's an aleph specific question -but it does relate to graalvm and native-image. Has anybody run into a similar problem before? https://clojurians.slack.com/archives/C0G922PCH/p1591971729018900

@lukaszkorecki You should probably add reflection config for that class

as in this list? https://github.com/lukaszkorecki/graleph/blob/master/script/compile#L9 - admittedly I just dove head first without doing much research on how to make stuff work in Graal :-)

https://github.com/oracle/graal/blob/master/substratevm/REFLECTION.md

ohhh

Gotcha, thanks @borkdude - will keep Trying Things Until They Work ™️

Preferably you should solve a reflection issue using (set! *warn-on-reflection* true) in the library itself

but sometimes classes aren't simply used in all possible ways during image generation, in that case a reflection config helps

Right, that was my next step to see where the reflective calls are

is this related to your statsd setup btw? you were hinting at a blog, wonder if that will still happen or if you have moved on to other setups 🙂

The blog post is coming along, babashka will get a huge shout out, it's just very meaty and technical so it's taking a bit of time

This is the other end of the metrics pipeline - atm we're using CollectD as the statsd server

and that doesn't support tagged metrics, just plain old statsd protocol

> This is the other end of the metrics pipeline I figured yes, so now you're making your own server.

Yes, as an experiment for now

but if it works, It might be possible to have a datadog-like setup on the cheap

(DataDog is the only hosted metrics thing that supports tagged statsd metrics, as far as I know)

why should the server run as a native binary, cheaper memory usage?

Yeah, our monitoring node is composed of several daemons - Collectd, FluentD and carbon-relay (C, Ruby and Golang respectively). I could for now just run the JVM version of course. But if the project works as expected (statsd server with tagged metrics support, writing to graphite) - could be useful to open source it and make folks life easier in terms of distribution (single binary) and footprint

You're right though: for our own purpose we could just run the jvm version

for now

Anyway, I'll check the reflection stuff and see how it goes. This also helps me to evaluate if we can migrate some of our own services to run via native-image and reduce the cost even more

Exciting. Considering that UDP is so easy, the server can also be POC-ed in babashka itself probably. There are some from the ground up examples of how to do a multi-threaded webserver in bb. FWIW.

Definitely. It's just that I'm more familiar with Aleph - our own Statsd client's test harness uses Aleph as a stand-in statsd server

Baking a custom native image will probably be more performant though

ok cool. there's an overview here of working and not-working projects: https://github.com/BrunoBonacci/graalvm-clojure, also FWIW

That repo is great - it was a good jump off point, especially that it has aleph's http server confirmed to be working

i tried adding the class to "initialize at run time" list but it still got initialized at build time resulting in an error

there are no reflection warnings seen by lein check either

@vale sometimes there's reflection happening in the Java implementation of something. this is very common with JDBC drivers

in that case a reflection config helps

e.g.: https://github.com/babashka/babashka-sql-pods/blob/master/reflection-hsqldb.json

I'll set some time aside later today and report back

@lukaszkorecki i think i got it to run without error, should i put up a pr?

@vale 🙌 yes please!

@borkdude reflection config did fix the issue https://github.com/lukaszkorecki/graleph/pull/1/files

thank you @vale

hehe i'm also planning on using graalvm for a cheap and quick monitoring tool

for tufte mainly

tufte is a profiling lib - what does it have to do with monitoring?

to be able to see the stats in real time while production app is running

right

Where does the native-image part come in?

mainly for fun, i'm outdated when it comes to all the latest build tools and tech

also wanted something light/small to just sit on my app server and read from file/io to parse the logs and kill it whenever i want

is your log format json, edn or something alike?

was talking to peter yesterday about it, i was thinking of serializing using nippy to file/io every few seconds and reading from that

just want something super quick and dirty that will make it really easy understand how long my routes are taking and see the performance impact switching to reitit on my main project

if you want to have a quick and dirty scripting solution, you could also look at babashka, it's a native binary built with GraalVM, but it can execute clojure expressions. it doesn't support nippy yet though, although that has come up a few times. maybe I should add support for it... hmmmm

what other binary formats are there, cbor?

i don't know how to make a nice CLI display though, throwing a small dashboard together with reagent and chartjs would take me no time at all

otherwise i would use bb

hmm, there are libs to make TUIs in Clojure as well. might be cool to try that out with GraalVM

there was this one: https://multimud.github.io/clojure-lanterna/

Based on my experience BB is a great place to start, before rolling your own tool

its crazy powerful

any libs you'd recommend?

ah... will check that out

oh so you want to build a web-app compiled with graalvm

a small one yeah 😉

I do have one web-app example in bb: https://github.com/borkdude/babashka/blob/master/examples/notes.clj It's built from the ground up because bb doesn't have a built-in webserver (yet), so even the HTTP responses are parsed manually. But it works and starts fast, lol.

Rolling a custom GraalVM binary will likely be more performant and less gnarly to write.

Good luck.

btw aleph's http server compiles with graalvm native-image

what's the most performant http server these days? immutant?

confirming that nippy also works with graalvm:

$ ./bb "(require '[taoensso.nippy :as n]) (n/thaw (n/freeze [1 2 3]))"
[1 2 3]