Clojurians Log v2

We're making use of ProxySQL (https://github.com/sysown/proxysql), and it's a great tool, but for me at least, the code leaves something to be desired in terms of readability/extensibility/etc. ProxySQL is a protocol-aware proxy for MySQL, and provides some nice functionality like connection multiplexing, simple rules for handling queries and results, etc. Feels like something Clojure would be able to do much more concisely, between the data-transformation and concurrency capabilities. Posting here to see if anyone is aware existing work along these lines in Clojure (or the JVM), even if it's just around handling the MySQL wire protocol. Thanks.

We use that at work. Not sure why you think it should be rewritten in Clojure? It's very low-level and very specific (to MySQL). Why not use a perfectly good existing tool (ProxySQL)? @dave.dixon

@seancorfield The specificity to MySQL is one point, both because it locks us into MySQL and makes apps which don't use MySQL turn into special cases/gaps. There are several times we've looked into modifying or extending ProxySQL and decided that the LOE and risk would be too high with the current codebase. And not all features of MySQL are supported (notably cursors), which again become limitations/exceptions.

I'm not sure that it would turn out to be worth the effort to reimplement ProxySQL, but I'd like to get a better idea of what the effort would really be. Plus it's an excuse to get my hands dirty at the MySQL protocol level, which will be useful moving forward with our use of ProxySQL.

@dave.dixon A little anecdote about ProxySQL: we ran into a really obscure bug (I don't even remember the details any more) but we had to leverage the support contract our managed datacenter has with the ProxySQL folks and it multiple experimental runs with the ProxySQL team in Eastern Europe and the DBA team at our datacenter all monitoring the ProxySQL system with enhanced logging before they could isolate the bug. The ProxySQL folks were actually very excited that we had found this bug because it was a particularly complex edge case and they found it a fascinating problem to solve. They had a fix for us very quickly. All possible because of extended support contracts and entire teams of database experts getting involved.

Because of that, I would never even contemplate trying to write a replacement for ProxySQL -- it's a phenomenally hard problem to solve correctly and if you think the LOE/risk would be too high just to modify/extend the current codebase, imagine what the LOE/risk would be for replicating such a complex problem solution would be from the ground up and targeting multiple databases?

Well, that just makes me want to dig in even more 😜 And your assessment may be correct. But having spent quite a bit of time in the ProxySQL code, I have a growing suspicion that a lot of the apparent complexity is incidental. Not to say that the intrinsic complexity is low either, but that's part of what I'd like to learn. Some of that learning may be that it would be easier to handle the features we desire as an independent component. All those paths have the same starting point, I think, which is getting between client and server and dealing directly with the protocol.

Over the last week I worked on a new feature in babashka called pods. I published one sql pod right now: https://github.com/babashka/pod-babashka-hsqldb This allows you to create in memory and persistent HSQLDB databases from babashka scripts. I'm also planning a Postgres pod that's similar to this. Possibly also Windows support for both pods. After that, perhaps mySQL and sqlite pods, if I can get them to work with GraalVM. The database interactions use next.jdbc. For those that don't know babashka: it's a fast starting Clojure scripting tool that supports a significant subset of JVM Clojure features. You would typically use it for replacing bash or Python shell scripts. Feedback welcome!