Clojurians Log v2

is there a good place to talk about libraries like sente? i have some architecture questions. i saw #ring but that doesn't seem to cover web sockets

@nbtheduke I don’t see any more specific channels so you could ask here. At least once folks see the question, they’ll know whether there’s a better channel for it…

(I haven’t used Sente for years but I can take a stab at it maybe)

i'm wondering about how to handle authorization in sente handlers. As an example, using Compojure for http requests lets me say: (wrap-routes admin-routes auth/wrap-authorization-required), and then in wrap-authorization-required, i can say (if (:admin user) (handler req) (response 401 ...))

I can’t remember whether Sente exposes header information but I seem to recall we tackled this somehow, back in the day. I think we passed some sort of auth token back to the server?

@nbtheduke The “official” Sente example app has a stub for login/auth https://github.com/ptaoussanis/sente/blob/master/example-project/src/example/server.clj

Oh, maybe I've completely misunderstood then. So the authentication happens when the websocket connection is made, not at each message sent?

A websocket is a socket, so a stateful bidirectional connection

If you have a clustered setup and you lose connectivity to one server and failover to another, you have to reestablish the auth'd status on the new server. Something to thing about.

I'm dipping my toes in the 'next-jdbc' waters, and ran into a small issue. It's probably a gap in my knowledge of streaming, but here goes....

I have a query using 'plan' that then can be reduced 'into []' with no issue.. it's a huge result set, which is what I expect.

But when I try to reduce that into something streamable, I'm getting an empty result..

So...

(ring-io/piped-input-stream
                                (fn [ostream]
                                  (let [^java.io.Writer w (<http://clojure.java.io/make-writer|clojure.java.io/make-writer> ostream {})]
                                    (get-big-data-streaming w))
                                  (.close ostream)))

This is in the handler ☝️:skin-tone-2:

And the function of interest:

(reduce (fn [^java.io.Writer w record]
              (let [m (datafiable-row record *db* {})]
                (.write w m)
                ))
            writer
            (jdbc/plan *db* sv))

We can safely assume that sv is a sqlvec that works and generates a large dataset outside of this streaming code

So the 'w' passed in is the Writer and locally is named writer

Hitting the endpoint takes about 4 seconds, the same as the non-streaming version, but returns no data.

@jmckitrick Not exactly the answer you’re looking for, but for anything next.jdbc-related you can also ask in #sql

ok

Hi. I am trying to implement a protocol for a type I am defining as part of clojure.core.matrix

mp/PFunctionalOperations


  (element-map!
      ([m f]
        (.modifyAll (.-array1d m) (UnaryFn. f)))
      ([m f a]
        (.modifyMatching (.-array1d m) (BinaryFn. f) (.-array1d a)))
      ([m f a more]
        (.modifyMatching (.-array1d m) (BinaryFn. f) (.-array1d a))
        (doseq [b more] (.modifyMatching (.-array1d m) (BinaryFn. f) (.-array1d b))))) 

; --------------------------------------------------------------------------------
; eval (current-form): (def m (create-vector [1 2 3]))
#'ojalgo-clj.core/m
; --------------------------------------------------------------------------------
; eval (selection): m
(1.0 2.0 3.0)
; --------------------------------------------------------------------------------
; eval (current-form): (.modifyAll (.-array1d m) (UnaryFn. inc))
nil
; --------------------------------------------------------------------------------
; eval (selection): m
(2.0 3.0 4.0)
; --------------------------------------------------------------------------------
; eval (current-form): (deftype Vector [^Array1D array1d] Object (toSt...
; (err) Syntax error macroexpanding clojure.core/let at (C:\Git\ojalgo-clj\src\clojure\ojalgo_clj\core.clj:18:1).
; (err) (.modifyAll (.-array1d m) (UnaryFn. f)) - failed: simple-symbol? at: [:bindings :form :local-symbol] spec: :clojure.core.specs.alpha/local-name
; (err) (.modifyAll (.-array1d m) (UnaryFn. f)) - failed: vector? at: [:bindings :form :seq-destructure] spec: :clojure.core.specs.alpha/seq-binding-form
; (err) (.modifyAll (.-array1d m) (UnaryFn. f)) - failed: map? at: [:bindings :form :map-destructure] spec: :clojure.core.specs.alpha/map-bindings
; (err) (.modifyAll (.-array1d m) (UnaryFn. f)) - failed: map? at: [:bindings :form :map-destructure] spec: :clojure.core.specs.alpha/map-special-binding

That looks like a syntax error related to something else than the matrix or UnaryFn stuff per se

As an aside, run! and the underlying reduce are faster than doseq

It seems like you accidentally put (.modifyAll (.-array1d m) (UnaryFn. f)) into a binding position but I don't see anything like that in your initial snippet

Thanks for the run! tip. So if I take out the arity overloading like so

(element-map!
      [m f]
        (.modifyAll (.-array1d m) (UnaryFn. f))) 

If I have

(element-map!
      ([m f]
        (.modifyAll (.-array1d m) (UnaryFn. f))))

(element-map!
   [p__18280 p__18281]
   (clojure.core/let
    [[m f] p__18280 (.modifyAll (.-array1d m) (UnaryFn. f)) p__18281]))

Which in turn leads to

; eval (current-form): (deftype Vector [^Array1D array1d] Object (toSt...
; (err) Syntax error macroexpanding clojure.core/let at (C:\Git\ojalgo-clj\src\clojure\ojalgo_clj\core.clj:19:2).
; (err) (.modifyAll (.-array1d m) (UnaryFn. f)) - failed: simple-symbol? at: [:bindings :form :local-symbol] spec: :clojure.core.specs.alpha/local-name
; (err) (.modifyAll (.-array1d m) (UnaryFn. f)) - failed: vector? at: [:bindings :form :seq-destructure] spec: :clojure.core.specs.alpha/seq-binding-form
; (err) (.modifyAll (.-array1d m) (UnaryFn. f)) - failed: map? at: [:bindings :form :map-destructure] spec: :clojure.core.specs.alpha/map-bindings
; (err) (.modifyAll (.-array1d m) (UnaryFn. f)) - failed: map? at: [:bindings :form :map-destructure] spec: :clojure.core.specs.alpha/map-special-binding

Now I remember, you have to do it like this https://github.com/nilern/monnit/blob/main/src/monnit/pair.cljc#L11-L26

An unfortunate gotcha

Aha! Thanks, I was looking at an extend-protocol example which works with the parens. Thank you very much for the help.

Hey team, I want to create a handy help to time and log other function calls.

(defmacro with-logging [k body]
  `(let [start-ms# (System/currentTimeMillis)
         res# ~body
         end-ms# (System/currentTimeMillis)]
     (log/infof {:k ~k :ms (- end-ms# start-ms#)})
     res#))

(defn get-foo [id] (with-logging :get-foo (db/get ["select * from foos where id = ?" id])))

Check out https://github.com/ptaoussanis/tufte

@stopachka one useful strategy is to capture the form/expression itself so you don't have to synthesize a name

(defmacro timing [expr]
  `(let [s# (System/currentTimeMillis)
         ret# ~expr
         time# (- (System/currentTimeMillis) s#)]
     {:ret ret#
      :time time#
      :expr '~expr}))

&env does not give the fn name as a local so you would need a defn-like macro You could use (.. Thread currentThread getStackTrace) but that might trigger deoptimization, not good for profiling

repl> (timing (Thread/sleep 5000))
{:ret nil, :time 5002, :expr (Thread/sleep 5000)}

> (source time)
(defmacro time
  "Evaluates expr and prints the time it took.  Returns the value of
 expr."
  {:added "1.0"}
  [expr]
  `(let [start# (. System (nanoTime))
         ret# ~expr]
     (prn (str "Elapsed time: " (/ (double (- (. System (nanoTime)) start#)) 1000000.0) " msecs"))
     ret#))

Also note the use of nanoTime instead of currentTimeMillis

that's super close to just a prepl

(defn t [f k] (let [n (System/nanoTime)] (k (f) (/ (- (System/nanoTime) n) 1e6))))

If you wanted to implement a user or content driven recommendation system in a JVM/Clojure/Postgres project what libraries would you look at? Cortex?

Hi folks! I believe I have seen the recommendation to use when instead of if when I only care about the truthy branch, i.e. (when cond someting) instead of (if cond someting) . I wanted to refer to it but could not find it. Am I just making it up?

I would ask in Zulip in https://clojurians.zulipchat.com/#narrow/stream/151924-data-science

I remember a recentish discussion on this as well. I found https://www.reddit.com/r/Clojure/comments/3x44qx/style_question_and_if_when/ but I think it was brought up more recently though I can't remember where

you aren't making it up: https://guide.clojure.style/#when-instead-of-single-branch-if https://github.com/clj-kondo/clj-kondo/blob/master/doc/linters.md#missing-else-branch

Thanks a lot! I was looking into the guide but just could not see it 🙂

No, actually the guide says > Use when instead of (if …​ (do …​)). I just skipped that because here I did not have a do 🙂

Fun fact: In Norwegian, do means toilet.

all the more reason to use it liberally through-out your codebase!

FWIW, Phil Hagelberg (technomancy) has pretty strong opinions about this: he thinks when should only be used for side-effecting things and that a single-branch if for pure code is just fine. I think he still holds that anyway. Certainly, there was a very pointed commit to Leiningen some years back that replaced when with if in a lot of places.

https://github.com/technomancy/leiningen/commit/f362d596f5879ad6cb6b9932124932388572959a

(I don’t agree with that but it sticks in my mind as a specific preference from a well-known Clojurian)

Thank you for sharing that!

@seancorfield The irony is I believe the Sonian team (where Phil used to work) would reject any PR that had a single branch if. @hiredman do you remember if that was the case?

I don't recall that, but I do recall lots of spirited debate

is there any extra overhead incurred by having the implicit do?

or does that get optimized away when it’s clear there is only one form?

do doesn't exist

(macroexpand '(when true 3))
;;=> (if true (do 3))

it exists in clojure, but doesn't exist in bytecode

that makes sense. I think

it is a mechanism for expressions, bytecode isn't expressions

I’m trying to figure out a good way to create a lazy list of vectors in this shape:

[[0 0 0]
 [1 0 0]
 [1 1 0]
 [1 1 1]
 [2 1 1]
 [2 2 1]
 [2 2 2]]

(take 10 (reductions (fn [a b] (update a b inc)) [0 0 0] (cycle (range 3))))

wow

that is very cool 😅

also, I always forget about cycle

but probably I wouldn’t have come up with this approach either way 😄

thanks @oscarlinusericsson!

Thanks! index space is the shit.

Is there a way to get the second to last error in your REPL?

negative

(def ^:dynamic *e2)
(binding [*e2 nil]
  (clojure.main/repl :caught (fn [e]
                               (set! *e2 *e)
                               (clojure.main/repl-caught e))))

Freaking awesome. It’d have been taking me years to come up with such solution @oscarlinusericsson

Nice