sql

All things SQL and JDBC...
kwrooijen 2020-07-05T19:25:33.133400Z

Hello, I'm using next-jdbc, and I'm trying to modify values based on their column (or Clojure key name) when I query data. I've written a build-fn which changes column labels to clojure keys (kebab case / namespaced). Now I'd like to modify the values of these columns based on their key (some columns need to be modified when read). Maybe there's a better way to achieve what I'm trying to do, but here's my current setup:

(defn my-kebab-case-builder [rs opts]
  (let [kebab #(string/replace % #"_" "-")
        opts (assoc opts :qualifier-fn kebab :label-fn kebab)]
    (result-set/as-modified-maps rs opts)))

(defn my-column-modifier [rs rsmeta i]
  ;; This doesn't return the modified column label
  (.getColumnLabel rsmeta i)
  (when-let [value (.getObject rs i)]
    ;; Do something based on column label
    value))

{:builder-fn (result-set/as-maps-adapter
              my-kebab-case-builder ;; e.g. Change "updated_at" to :person/updated_at
              my-column-modifier)}
This works fine, except that the .getColumnLabel method doesn't get the new column label, created by my-kebab-case-builder. Is it possible to access this? I noticed that result-set/as-modified-maps creates a new MapResultSetBuilder https://github.com/seancorfield/next-jdbc/blob/71ea50eff845b3e6f2404c9f3388ad8ccb6581cc/src/next/jdbc/result_set.clj#L148-L161 and it passed a cols key, though I have no idea if it's possible to read this. I could work around this by changing my Clojure key to an "PostgreSQL label", and using that as an identifier, but that would feel like a workaround. Anyone have any tips?

seancorfield 2020-07-07T21:44:39.159700Z

@kevin.van.rooijen The latest next.jdbc on GitHub (develop branch) has a new, more generic builder-adapter that takes a column-by-index-fn that is called with the builder itself, the ResultSet, and the column index. That would allow you to write my-column-modifier as follows:

(defn my-column-modifier [builder ^ResultSet rs i]
  (let [col (nth (:cols builder) (dec i))]
    (when-let [value (.getObject rs i)]
      (if (= col :person/updated-at)
        .. ;; your custom processing
        ;; else call regular value processor
        (rs/read-column-by-index value (:rsmeta builder) i)))))
Note that with this new adapter, you're expected to call read-column-by-index if you want the default behavior for some subset of values.

seancorfield 2020-07-07T21:45:10.159900Z

Then you'd do:

{:builder-fn (result-set/builder-adapter my-kebab-case-builder my-column-modifier)}

kwrooijen 2020-07-07T21:48:26.160100Z

Interesting, I'll try this out tomorrow, thanks!

kwrooijen 2020-07-14T16:11:45.219700Z

@seancorfield Bit of a late response, but just tried it out. Works great! Thanks, this will clean up a lot of my code

seancorfield 2020-07-14T16:37:05.219900Z

Excellent! Thanks for following up on that for me.

seancorfield 2020-07-05T19:43:45.134800Z

@kevin.van.rooijen (.getColumnLabel rsmeta i) reads the column label directly from the underlying ResultSetMetaData Java object.

seancorfield 2020-07-05T19:45:45.135700Z

And my-column-modifier is called to read the column value, independently of how the column names are built.

kwrooijen 2020-07-05T19:48:21.136100Z

Ok, so there's no way for me to get the new column name in that case?

seancorfield 2020-07-05T19:49:59.136900Z

The column name comes from the Java object -- the question doesn't really make sense.

seancorfield 2020-07-05T19:50:21.137400Z

The builder is how the Clojure data structure is built from the underlying Java objects.

kwrooijen 2020-07-05T19:53:13.138400Z

Right, but I build the Clojure data structure based on the column name. Basically saying if a column is a specific keyword, I want to change it in a different way

kwrooijen 2020-07-05T19:53:54.139200Z

But it sounds like I'm better off modifying the clojure keyword to a string, underscoring it, and passing it to my-column-modifier

kwrooijen 2020-07-05T19:54:18.139400Z

Thanks for the info

seancorfield 2020-07-05T20:01:15.141Z

@kevin.van.rooijen Reading columns from the ResultSet is completely independent from creating the Clojure keys in the map. There is no Clojure data structure or keyword "column name" at the point you read the column value from the ResultSet.

seancorfield 2020-07-05T20:02:46.142200Z

You could write your own adapter, i.e., reify both the RowBuilder and the ResultSetBuilder and then you'd have access to your builder and therefore to the new column names you are creating.

kwrooijen 2020-07-05T20:03:24.142500Z

That sounds like an interesting option. I'll look into that, thanks

seancorfield 2020-07-05T20:04:42.143400Z

Start with as-maps-adapter and then you can get (:cols mrsb) inside the adapter and so you would have more control over how you read columns.

👍 1
seancorfield 2020-07-05T20:07:22.144700Z

Line 240 (`next.jdbc.result-set`) and your column reader could use nth on (:cols mrsb) to get the Clojure keyword you've constructed for that column.

seancorfield 2020-07-05T20:17:22.147400Z

It's worth bearing in mind that the current adapter still calls read-column-by-index on the result of calling column-reader -- so you could simplify that in your version -- but that is done deliberately so the ReadableColumn protocol is in play (but if you're writing your own builder or adapter from scratch, you can make that decision).

kwrooijen 2020-07-05T20:21:29.148100Z

I'm not really familiar with the internals, so I'll have to do some digging. But this'll give me a head start 🙂

seancorfield 2020-07-05T20:51:50.149Z

I think I could add a new type of adapter that allowed more control to the caller. I'll have to give it some thought.

seancorfield 2020-07-05T21:43:47.149200Z

https://github.com/seancorfield/next-jdbc/issues/129

adam 2020-07-05T22:21:13.150600Z

I am having trouble getting Postgres types to work with the IN clause:

(sql/query ds (honey/format {:select [:id :name :role]
                              :from   [:user]
                              :where [:in :role [(types/as-other "parent")]]}))
> ERROR: operator does not exist: user_role = character varying > Hint: No operator matches the given name and argument types. You might need to add explicit type casts.

adam 2020-07-05T22:25:30.152Z

I assume the type meta data is getting lost when passing the map to honey/format, but unsure what I can do about it

adam 2020-07-05T22:42:27.152600Z

Ah never mind, the following works:

(honey/call :cast "parent" :user_role)

seancorfield 2020-07-05T23:27:03.154500Z

@somedude314 Interesting. I'll create a ticket on HoneySQL to investigate whether metadata survives its transformations. It probably should if it can...

seancorfield 2020-07-05T23:29:44.155200Z

Actually, now I think about that, I think it is preserving the metadata (because it would get a different error if it wasn't).

seancorfield 2020-07-05T23:37:17.157900Z

That enum handling is a quirk of PostgreSQL so there may be contexts where it doesn't work -- it was added to support insert rather than select/in so it's entirely possible you have to use the more traditional cast approach in queries?