sql

All things SQL and JDBC...
dominicm 2020-11-05T12:48:43.220800Z

Yeah, we discovered that. I was just curious to know how others are handling exceptions in these cases, and relating them to the particular function calls.

seancorfield 2020-11-05T15:06:23.222Z

@dominicm sounds like a situation where you might want to use save points to get more control?

dominicm 2020-11-05T15:07:27.222500Z

I'll take a look and see if they would help. Never encountered those!

seancorfield 2020-11-05T15:59:09.223100Z

They weren't well-supported in clojure.java.jdbc but there are examples in the next.jdbc docs @dominicm

Darrell 2020-11-05T17:20:48.224200Z

Here’s a perhaps offbeat question: I have a need to query SQL Server 2005. Is that even doable with modern Clojure libs?

👀 1
seancorfield 2020-11-05T17:41:34.224700Z

@darrell As long as there's a compatible JDBC driver, yes.

seancorfield 2020-11-05T17:43:46.226600Z

I don't know whether the current Microsoft SQL Server JDBC driver will work with it (i.e., the versions published to Maven) but you can probably download an older JDBC driver from Microsoft's site somewhere that would work. I can't remember the oldest version of SQL Server that I tested clojure.java.jdbc against but I used to run it on a Windows XP VM so it was old (and I definitely had to download the driver manually back then).

seancorfield 2020-11-05T17:44:06.227100Z

But if you can find a compatible JDBC driver then both clojure.java.jdbc and next.jdbc should work with it just fine.

seancorfield 2020-11-05T17:45:00.227400Z

https://www.microsoft.com/en-us/download/details.aspx?id=2505 is the download page for the old drivers.

seancorfield 2020-11-05T17:46:31.228400Z

I would try the latest JDBC driver (from Maven) first tho'. That would be [com.microsoft.sqlserver/mssql-jdbc "8.4.1.jre8"] for project.clj or com.microsoft.sqlserver/mssql-jdbc {:mvn/version "8.4.1.jre8"} for deps.edn

Darrell 2020-11-05T17:49:01.229100Z

Thanks @seancorfield. I did try the latest but it didn’t work. It’s complaining about “The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: \“Unexpected rethrowing\“.”

seancorfield 2020-11-05T17:51:37.230100Z

Sounds like you'll need to download the older driver then. If you're using CLI/`deps.edn` you can just depend on the JAR via :local/root. lein is a bit more involved I think.

Darrell 2020-11-05T17:52:13.230400Z

Thanks! I’ll explore and see what I can get working.

Darin Douglass 2020-11-05T19:43:08.231300Z

@seancorfield is an explicit require of camel-snake-kebab.core required to use next.jdbc/snake-kebab-opts? looking at source it doesn't feel like it

Darin Douglass 2020-11-05T19:44:27.232500Z

i ask b/c i'm seeing an unbound fn error for #'camel-snake-kebab.core/->snake_case when running my uberjar'd app. it does not happen in-repl

Darin Douglass 2020-11-05T19:44:45.232900Z

putting an explicit require seemed to fix the issue

seancorfield 2020-11-05T20:23:27.234200Z

@ddouglass If you are requiring next.jdbc, and you have the CSK library on the path (in the JAR, in your case), then it should automatically require c-s-k.core for you. How did you build your uberjar?

Darin Douglass 2020-11-05T20:23:55.234400Z

lein uberjar

Darin Douglass 2020-11-05T20:24:15.235100Z

i've hopped into the docker image and looked at the jar in emacs and the .class and .clj files exist

seancorfield 2020-11-05T20:24:24.235600Z

Ah, I bet that doesn't put c-s-k on the classpath when it builds the uberjar...

Darin Douglass 2020-11-05T20:24:28.235700Z

it's a kafka streams app, so there may be some oddities thrown in there

seancorfield 2020-11-05T20:26:24.237200Z

Hmm, I haven't used lein for years. Can you try to create a small, self-contained repro case and put it up on github for me to look at?

Darin Douglass 2020-11-05T20:27:52.237700Z

via the jar

-rw-rw-rw-      1713   5-Nov-2020  19:49:22  camel_snake_kebab/core$__GT_snake_case_symbol.class
  -rw-rw-rw-      1744   5-Nov-2020  19:49:22  camel_snake_kebab/core$__GT_kebab_case.class
  -rw-rw-rw-      1715   5-Nov-2020  19:49:22  camel_snake_kebab/core$__GT_kebab_case_keyword.class
the app runs via java -jar , so it should be there.

Darin Douglass 2020-11-05T20:28:31.238500Z

i'll poke around a bit at it, i've a workaround in place anyway (explicit (require 'camel-snake-kebab.core)) so it's not too pressing

seancorfield 2020-11-05T20:32:36.239Z

I've managed to repro... I'll create a GH issue for this...

Darin Douglass 2020-11-05T20:33:00.239200Z

oh neat

Darin Douglass 2020-11-05T20:33:24.239800Z

what's the problem?

seancorfield 2020-11-05T20:38:02.240Z

https://github.com/seancorfield/next-jdbc/issues/144

Darin Douglass 2020-11-05T20:38:52.240300Z

👍

Darin Douglass 2020-11-05T20:38:54.240500Z

thanks

seancorfield 2020-11-05T20:44:07.241100Z

I think I have a fix for it. I'll try to cut a new next.jdbc release later today.

Darin Douglass 2020-11-05T20:44:13.241300Z

speedy boi

Darin Douglass 2020-11-05T20:44:16.241500Z

ty ty

dharrigan 2020-11-05T20:50:52.242Z

Ha! I'm precisely in that area right now - doing some work with resultsets and csk!

seancorfield 2020-11-05T20:51:57.242800Z

@ddouglass OK, change your dependency to 1.1.613 for seancorfield/next.jdbc and it should be fixed.

seancorfield 2020-11-05T20:52:58.243Z

Sorry about that 😞

Darin Douglass 2020-11-05T20:53:43.243500Z

not a problem, was funky when it died in the cluster but not in my repl

Darin Douglass 2020-11-05T20:54:55.244300Z

though i am confused on why requiring-resolve wasn't doing the trick?

seancorfield 2020-11-05T20:57:34.245200Z

It was requiring at macro expansion time (to determine whether the library was present), and in a REPL context that carries over to the evaluation time context.

seancorfield 2020-11-05T20:58:16.246100Z

But in an uberjar context, it does not. So I shifted the requiring-resolve into the expansion and just use require at expansion time -- which is what I should have done in the first place.

Darin Douglass 2020-11-05T20:58:52.246300Z

gotcha

dharrigan 2020-11-05T21:11:22.246800Z

I think I may have missed something, did the next jdbc channel disappear?

seancorfield 2020-11-05T21:45:58.247100Z

@dharrigan There never was one.

seancorfield 2020-11-05T21:46:08.247500Z

All the next.jdbc chatter tends to happen here.

dharrigan 2020-11-05T21:46:16.247800Z

waaaaaah! I must be going loopy in this lockdown!

seancorfield 2020-11-05T21:46:24.248Z

Join the club 🙂

dharrigan 2020-11-05T21:46:37.248300Z

I could have sworn there was a dedicated next jdbc channel

dharrigan 2020-11-05T21:49:53.248700Z

I have a question about next jdbc and plan then 🙂

dharrigan 2020-11-05T21:50:31.249100Z

So, as I've found out (and as you point out in the documentation) this is the case The row in the reduction is an abstraction over the underlying (mutable) ResultSet object -- it is not a Clojure data structure. Because of that, you can simply access the columns via their SQL labels as shown

dharrigan 2020-11-05T21:51:15.249900Z

thus, if I have a column such as tenant_id, then I need to access it like (:tenant_id row)

dharrigan 2020-11-05T21:51:26.250100Z

however!

dharrigan 2020-11-05T21:51:40.250500Z

(and let me formulate this...a moment)

dharrigan 2020-11-05T21:52:59.251300Z

I was (am!) experimenting with some camel-snake-kebab and I'm passing this into the opts of the plan: {:builder-fn rs/as-unqualified-kebab-maps}

dharrigan 2020-11-05T21:53:21.251800Z

when I do (log/info row), it spits out {:tenant-id "foobar"}

dharrigan 2020-11-05T21:53:24.252Z

and this confused me.

dharrigan 2020-11-05T21:53:54.252600Z

since, it appears that something is converting the columns to csk, when I log (which I suppose realises the row)

dharrigan 2020-11-05T21:54:21.253200Z

but if I try to then do (tenant-id row) I get nil, but (tenant_id row) returns a value

dharrigan 2020-11-05T21:54:37.253600Z

it feels like there are two representations of the data

dharrigan 2020-11-05T21:54:39.253800Z

.

seancorfield 2020-11-05T22:04:20.254200Z

Not quite sure what you're saying here?

but if I try to then do (tenant-id row) I get nil, but (tenant_id row) returns a value

seancorfield 2020-11-05T22:04:35.254500Z

Did you mean to have keywords there?

dharrigan 2020-11-05T22:07:18.255700Z

oops, sorry, yes (:tenant-id row) and (:tenant_id row)

seancorfield 2020-11-05T22:07:33.256Z

Inside the reducing function over plan, you do not have a realized result set -- the builder has not been called on the data -- you are interacting "directly" with the underlying ResultSet which has column labels (technically as strings).

dharrigan 2020-11-05T22:07:55.256300Z

ah right, so the builder is not invoked until the reduction happens

seancorfield 2020-11-05T22:10:50.256600Z

No, the builder is only invoked if you realize a row.

seancorfield 2020-11-05T22:11:06.257200Z

From the options page: * :builder-fn -- a function that implements the RowBuilder and ResultSetBuilder protocols; strictly speaking, plan and execute-one! only need RowBuilder to be implemented (and plan only needs that if it actually has to realize a row) but most generation functions will implement both for ease of use.

dharrigan 2020-11-05T22:11:07.257300Z

right, thank you. that clarified it

seancorfield 2020-11-05T22:12:29.258300Z

I'll see if I can make it clearer in the Getting Started guide section for plan. This seems to trip a lot of people up, if they don't have a good mental model of what happens with plan 😐

dharrigan 2020-11-05T22:12:34.258500Z

Another question, if I may...

seancorfield 2020-11-05T22:12:41.258700Z

You may indeed 🙂

dharrigan 2020-11-05T22:13:34.259700Z

Would a [{:keys [tenant-id]} row] (i.e., a keys) realize a row?

seancorfield 2020-11-05T22:14:03.260Z

That's a good question 🙂

dharrigan 2020-11-05T22:14:06.260200Z

Any operation that would require a Clojure hash map, such as assoc or anything that invokes seq (keys, vals), will cause the full row to be expanded into a hash map, such as produced by execute! or execute-one!,

dharrigan 2020-11-05T22:14:12.260500Z

does that suggest it does?

dharrigan 2020-11-05T22:14:45.260900Z

What i was trying to do is pass the (yet unrealized) row into a function

dharrigan 2020-11-05T22:14:48.261100Z

but destructure it

seancorfield 2020-11-05T22:16:14.262100Z

Macroexpanding it suggests that it will not realize a row and therefore you would need to use tenant_id because it's a column label.

dharrigan 2020-11-05T22:17:08.263100Z

Yes, just proved it

seancorfield 2020-11-05T22:17:13.263400Z

You can tell because if you do (let [{:keys [tenant_id tenant-id]} row] ...) if tenant_id has the value, no builder was involved. If tenant-id has the value, it was run through the builder.

dharrigan 2020-11-05T22:17:33.264100Z

so, the destructuring doesn't do the realization, i.e., thus not invoking the builder fn

seancorfield 2020-11-05T22:17:52.264700Z

Map destructuring uses get so no seq/`keys`/`vals` operation occurs.

dharrigan 2020-11-05T22:18:18.265100Z

right, so it's the keys function, not the :keys syntatic sugar

dharrigan 2020-11-05T22:19:38.266100Z

can I realize a row, thus having the builder function fire, without doing anything else (except return the result of that builder fn?)

seancorfield 2020-11-05T22:21:52.266400Z

Call rs/datafiable-row on the row.

dharrigan 2020-11-05T22:22:20.266700Z

right, just came across that

dharrigan 2020-11-05T22:22:23.267Z

cool

dharrigan 2020-11-05T22:22:43.267300Z

We talked ages and ages ago on plan

dharrigan 2020-11-05T22:22:57.267800Z

I've come back to revisit it (as the code is becoming more important)

seancorfield 2020-11-05T22:23:49.268100Z

In recent versions of next.jdbc, you can pass nil for the connectable and opts in datafiable-row

seancorfield 2020-11-05T22:24:11.268600Z

(which makes it easier to use when you don't care about datafy/nav stuff)

dharrigan 2020-11-05T22:27:08.268800Z

works like a charm

dharrigan 2020-11-05T22:27:33.269400Z

(rs/datafiable-row funkylicious-row nil nil)

dharrigan 2020-11-05T22:27:40.269700Z

invokes the builder

dharrigan 2020-11-05T22:28:15.269900Z

perfecto! thanks sean!

seancorfield 2020-11-05T22:31:35.270600Z

Remember that building a row is the expensive part you're normally trying to avoid when working with plan 🙂

👀 1
dharrigan 2020-11-05T22:32:18.270900Z

Yup, but I'm at that level where I need the data out of the row now 🙂

dharrigan 2020-11-05T22:33:06.271700Z

gotta get at it - eventually! 🙂