sql

All things SQL and JDBC...
zilti 2020-12-17T12:03:39.492600Z

I ran into an error today. The query is as follows:

["WITH distinct_location AS(
  SELECT MIN(city.city_id) AS city_id, location::TEXT as location_str FROM choicelist.city
   WHERE city.name ~ ?
   GROUP BY location::TEXT
)
SELECT * FROM choicelist.city AS city
 INNER JOIN distinct_location USING (city_id)
 ORDER BY city.city_id;" "EU Only)"]
Which gives this very concerning error message: org.postgresql.util.PSQLException: ERROR: invalid regular expression: parentheses () not balanced This sounds an awful lot like sql injection...

synthomat 2020-12-17T12:08:52.493600Z

indeed, the input somehow leaked into the SQL query; what library is it?

zilti 2020-12-17T12:11:30.494300Z

It's generated by HugSQL

zilti 2020-12-17T12:12:40.495300Z

And the query indeed works fine as long as the parameter doesn't have unbalanced parentheses

zilti 2020-12-17T12:15:55.495700Z

So, the chain is HugSQL -> jdbc.next -> PostgreSQL JDBC

synthomat 2020-12-17T12:18:23.495900Z

have you looked into this? https://www.hugsql.org/#faq-sql-injection

zilti 2020-12-17T12:22:33.496400Z

Yes, and according to that, value parameters get deferred "to the underlying database library to perform SQL parameter binding to prevent SQL injection issues" which would be jdbc.next in this case

curtis.summers 2020-12-17T12:30:41.497400Z

In order to debug this, can you provide the original HugSQL query and the call to the generated HugSQL function?

zilti 2020-12-17T12:33:29.498400Z

Sure, the original HugSQL query is:

-- :name get-city-by-name :? :1
WITH distinct_location AS(
  SELECT MIN(city.city_id) AS city_id, location::TEXT as location_str FROM choicelist.city
   WHERE city.name ~ :name
   GROUP BY location::TEXT
)
SELECT * FROM choicelist.city AS city
 INNER JOIN distinct_location USING (city_id)
 ORDER BY city.city_id;
And the call: (get-city-by-name db {:name "EU Only)"})

zilti 2020-12-17T12:34:29.499Z

(or, for the sake of getting that intermediate output, (get-city-by-name-sqlvec {:name "EU Only)"}) )

curtis.summers 2020-12-17T12:38:40.499900Z

So, the error is in the regular expression syntax expected by Postgresql. invalid regular expression: parentheses () not balanced

zilti 2020-12-17T12:39:36.000300Z

Ok, so no injection going on then

zilti 2020-12-17T12:39:41.000600Z

That's a relief

curtis.summers 2020-12-17T12:39:44.000700Z

correct

zilti 2020-12-17T12:40:18.001500Z

I would've been very surprised by the way, this would be such a basic thing, pretty sure this is exactly what the parametrized queries are for in the first place

zilti 2020-12-17T12:40:54.002200Z

Now to find out why PostgreSQL wants to treat it as a regex... The database field type is CITEXT, a case-insensitive text field

zilti 2020-12-17T12:41:53.002500Z

Ah lol, of course, because of the ~ operator

zilti 2020-12-17T12:42:14.002800Z

Alright, thank you very much for helping me figure things out!

curtis.summers 2020-12-17T12:43:40.003Z

you’re welcome!