sql

All things SQL and JDBC...
practicalli-john 2020-08-28T17:11:13.127400Z

I get the error "No suitable driver found for" when trying to execute a create table statement on an heroku postgres database. I have postgres, next.jdbc and h2 libraries in deps.edn

seancorfield/next.jdbc    {:mvn/version "1.1.569"}
  com.h2database/h2         {:mvn/version "1.4.200"}
  org.postgresql/postgresql {:mvn/version "42.2.16"}
next.jdbc is required in the namespace using the alias jdbc
ns practicalli.database-access
  (:require [next.jdbc :as jdbc])
I created a data-source from a JDBC connection string, which is bound to an operating system environment variable. The jdbc connection string is of the form: <postgres://username>:password@host:port/database-name
(def data-source-postgres (jdbc/get-datasource (System/getenv "DATABASE_URL")))
I checked the environment variable is returning the right connection string. I then execute a simple table query
(jdbc/execute!
    data-source-postgres
    ["create table account-holders(
     id int auto_increment primary key,
     name varchar(32),
     email varchar(255))"])
When evaluating this code is where the error occurs. Is there something else I need to do to tell next.jdbc which driver to use? I checked the .m2/repository/ directory and Clojure CLI downloaded the postgres 42.2.16 driver jar when I ran the REPL, according to the jar file timestamp. I will try creating a db-spec hash-map version of the jdbc connection string.

practicalli-john 2020-08-28T17:11:41.127800Z

The h2 database works without any issues.

seancorfield 2020-08-28T17:13:13.128500Z

JDBC connection strings are supposed to start with jdbc: according to the spec.

seancorfield 2020-08-28T17:14:19.129700Z

Also, it may be that you need postgresql: rather than postgres: -- the :dbtype "postgres" is an alias for :dbtype "postgresql" and normally you need the full version, not the alias, for connection strings to work...

seancorfield 2020-08-28T17:14:23.129900Z

^ @jr0cket

seancorfield 2020-08-28T17:17:07.131100Z

next.jdbc does nothing to the connection string -- it passes it directly to DriverManager/getConnection so the error is coming directly from JDBC.

practicalli-john 2020-08-28T17:18:52.132300Z

Okay, I fixed that to be "jdbc:<postgresql://username>:password@host:5432/db-name" and have som progres, I get an java.netUnknownHostException instead 🙂

practicalli-john 2020-08-28T17:19:12.132800Z

Is the connection using SSL by default, or is that something I need to set ?

seancorfield 2020-08-28T17:20:38.133400Z

I don't know. This is not a next.jdbc issue and I don't use PostgreSQL.

seancorfield 2020-08-28T17:21:58.134Z

Your string above has host in the hostname position, so I would double-check that.

👍 1
seancorfield 2020-08-28T17:26:46.135800Z

I just tried it locally and it seems like it doesn't accept the //username:password@ part, so I think you either need ?user=username&amp;password=password in your URL, or you'll need to use the 3-arity of next.jdbc/get-connection to pass in username and password...

seancorfield 2020-08-28T17:28:27.136600Z

Or you could pass {:jdbcUrl "jdbc:<postgresql://host>:port/db-name" :user username :password password} as a spec to next.jdbc/get-datasource

seancorfield 2020-08-28T17:30:30.137700Z

I tested both of those approaches locally and they seem to work (I have a PG Docker image running on 127.0.0.1:5432)

practicalli-john 2020-08-28T17:30:35.137800Z

Yes, that sounds similar to what I found in stack overflow with java apps... It wont kill me to create a few more environment variables 🙂

seancorfield 2020-08-28T17:31:50.139200Z

This is why I tend to recommend using the db-spec format: {:dbtype ".." :dbname ".." :user ".." :password ".."} -- but I assume you're getting the DB URL directly from the Heroku environment?

seancorfield 2020-08-28T17:33:01.139400Z

(JDBC is such a pain!)

practicalli-john 2020-08-28T17:35:39.141600Z

Yes, Heroku provides a DATABASE_URL environment variable, however, I can define others. It seems JDBC_DATABASE_URL is the defacto naming for the connection string with separate username and password variables.

(def db-specification-postgresql
    {:jdbcUrl  (System/getenv "JDBC_DATABASE_URL")
     :user     (System/getenv "JDBC_DATABASE_USERNAME")
     :password (System/getenv "JDBC_DATABASE_PASSWORD")})

  (def data-source-postgresql
    (jdbc/get-datasource db-specification-postgresql))
This is connecting to the Heroku postgres and once I re-learnt how to write sql again I can successfully create tables 🙂 Thank you for your help. I'll push all the details on https://practicalli.github.io/clojure-webapps/ over the weekend in case anyone else uses this setup.

kwrooijen 2020-08-28T18:33:12.142400Z

Heroku also provides a JDBC_DATABASE_URL (I wasn't sure if you're creating it yourself or not)

kwrooijen 2020-08-28T18:34:08.143800Z

(Granted that you're using the Clojure buildpack)

seancorfield 2020-08-28T18:34:20.144200Z

@jr0cket When I checked the Heroku docs, it sounded like JDBC_DATABASE_URL already includes ?user=...&amp;password=... so you don't need that map format -- just use

(def db-specification-postgresql (System/getenv "JDBC_DATABASE_URL"))

☝️ 1
seancorfield 2020-08-28T18:35:57.145300Z

In the example map I gave above, I was assuming the simpler URL format (no query string), which is why I suggested :user and :password along with :jdbcUrl.

kwrooijen 2020-08-28T18:36:15.145700Z

Heroku might also rotate the database parameters (I believe). So it wouldn't be a good idea to redefine them

kwrooijen 2020-08-28T18:37:23.146Z

But I might be confusing that with something else

practicalli-john 2020-08-28T18:40:41.148800Z

@seancorfield the JDBC_DATABASE_URL does not show in the heroku dashboard as an environment variable, only the DATABASE_URL , but the plan is to create suitable ones (now its working from my local machine). I'll write some code to use it without setting it and see if its just hidden @kevin.van.rooijen yes the connection details can be rotated, I did read that, though it was manual process, but will double check 🙂

kwrooijen 2020-08-28T18:41:21.149400Z

The JDBC_DATABASE_URL isn't visible in the dashboard because it's only available for specific buildpacks

kwrooijen 2020-08-28T18:41:43.150Z

Try heroku run echo \$JDBC_DATABASE_URL

practicalli-john 2020-08-28T18:42:31.151Z

Ah, I did mean to look at the heroku Clojure build pack as I'd like to write one specifically for deps.edn projects (rather than the hack I currently use with the Leinginen build pack).

kwrooijen 2020-08-28T18:43:31.151600Z

I see, I'm not sure if it'll be visible if you create your own buildpack

practicalli-john 2020-08-28T18:43:37.151700Z

This works

kwrooijen 2020-08-28T18:44:29.152500Z

FWIW if JDBC_DATABASE_URL doesn't show up, there's also this for converting DATABSAE_URL to JDBC_DATABASE_URL: https://github.com/kwrooijen/clj-database-url

practicalli-john 2020-08-28T18:44:37.152800Z

Its on a long todo list, so probably not for another month or two but settings like this are something to look out for.

practicalli-john 2020-08-28T18:46:10.154300Z

Yes, it worked, nice trick. That saves me creating the env var remotely. Back to re-learning SQL now. Thanks.

2020-08-28T19:18:09.156200Z

@jr0cket If you haven't figured out the SQL problem with your create table statement, I believe the issue is that table names are identifiers which cannot have - in them unless they're quoted. (See https://www.postgresql.org/docs/7.3/sql-syntax.html#SQL-SYNTAX-IDENTIFIERS)

practicalli-john 2020-08-28T20:38:46.156300Z

I used a dbtool called dbeaver that generates correct SQL for postgres, but good to have the details of why, thanks.