Clojurians Log v2

(the nested-tx and multi-rs branches have both been updated to match develop and I will continue working on those features for future releases)

seancorfield/next.jdbc {:mvn/version "1.0.476"} -- extends the "default options" behavior to the friendly SQL functions in next.jdbc.sql

Is there a way to iterate over the result set lazily in next.jdbc?

Judging from this the answer is no? https://stackoverflow.com/questions/58068904/how-to-adapt-the-ireduceinit-from-next-jdbc-to-stream-json-using-cheshire-to-a-h

@posobin There are two "lazy" things in play in questions like that: lazy streaming of results from the database, and lazy sequence processing of result sets. The former is possible via next.jdbc/plan. The latter is not -- because something has to manage all of the resources around connections and prepared statements etc.

In clojure.java.jdbc it was possible to provide a lazy :result-set-fn but the behavior was pretty much always wrong because the lazy sequence escaped the connection context and you nearly always got an exception from attempting to operate on a closed connection!

In next.jdbc, you can use plan and reduce/transduce/etc to process result sets that will not fit in memory, and you can terminate processing early via reduced (see docs for reduce and reduced in clojure.core).

You can use plan for results that would fit in memory too, and you can still control fetch size and other ways to lazily stream the result set. The processing itself must be eager, however. Also, note that persuading some databases to actually stream very large sets is tricky -- and the exact options needed tend to vary from database to database.

@posobin Can you explain your particular use case, if you're looking for a lazy sequence?