Clojurians Log v2

I've been working with some dynamic variables for a little project I'm using, and it seems like they aren't working for me. My setup is like so: I have a namespace which creates a dynamic variable, and defines a macro which includes binding it. In another namespace, I use said namespace (this works the same when I require it too), define a function, and call it from within the body of a usage of the macro, and while in that function the binding value isn't visible. However, whenever I use the cider debugger and debug either the function being called, or the usage of the macro, it works as intended.

Is there some gotcha with dynamic variables that would cause this? Or is there something else about my setup which is going wrong?

My guess would be you are binding the macro during macro expansion, but not in the expansion of the macro

behavior is identical when the usage of the macro is substituted for its macroexpansion

And what is that?

(restart-case (analyze-logs '("LOG: Hello, world!"
                              "LOG: This is a second log entry"
                              "LOG: "
                              "LOG: hey"
                              "blah"
                              "ERROR: "))
  ::exit (fn [] (throw (ex-info "exit" {}))))

(binding [*restarts* (merge *restarts*
                            #:user{:exit
                                   (fn [args__14292__auto__]
                                     (make-jump
                                      :semaphore.core/jump-target14766
                                      args__14292__auto__))})]
  (try
    (analyze-logs
     '("LOG: Hello, world!"
       "LOG: This is a second log entry"
       "LOG: "
       "LOG: hey"
       "blah"
       "ERROR: "))
    (catch
        semaphore.signal.Signal
        e__14300__auto__
      (apply
       (condp #(semaphore.proto/is-target? %2 %1) e__14300__auto__
         :semaphore.core/jump-target14766 (fn 
                                            []
                                            (throw
                                             (ex-info "exit" {})))
         (throw e__14300__auto__))
       (semaphore.proto/args e__14300__auto__)))))

Sent in a reply so as not to put a lot of code inline

Maybe you are constructing something like a lazy seq which is being realized outside the scope of the binding

No sequences are produced

actually wait

maybe that's it

That was it

No sequences are in the macro etc, but the return value produces a lazy value

Thanks!

Yep

@emccue im curious too! have u been already familiar with spring sec before u learnt clojure? @vemv let's not forget that one of clojure's core ideas is to allow leveraging existing jvm solutions.

I'm not going to pretend to have a real scientific justification but in general for stuff like passwords and hashing i'd rather have stuff that is more well audited. Its a comfort blanket sort of thing.

I tend to program in clojure as if everyone involved in the language other than me is going to die suddenly one day choking on the dust that was once a river

JDBC stuff is stable, http stack is stable, libraries that work with just data I can alter if needed

"Security" just feels like something that would require frequent/crucial updates from domain experts. I trust the spring community for that more

Which is hypocritical, since I do use buddy for stuff

IDK, a crypto fn is essentially a mathematical function and I don't expect math to change much over time :) FWIW I'm not java-allergic (in fact the opposite) but if I can pick a lib instead of a framework, that seems a win to me auditing is also a delicate topic. A framework (even more so one written in java) seems hard to audit; often surprisingly few people know the guts of the stuff

it's not all kittens and unicorns in clojure land of course, but a smaller lib in my language of choice seems vastly easier to assess (especially for code coverage)

> a crypto fn is essentially a mathematical function Yeah, but the time taken to compute that function is an important consideration. I remember reading this a while back https://security.stackexchange.com/questions/18197/why-shouldnt-we-roll-our-own and it i think influences my opinion

There is this offhand comment in there "a scheme isn't secure until it has been extensively attacked."

and at this point i'm arguing with vibes and impressions, but it does make sense to me

at least in the manner I use spring security it is a library - it has compile time dependencies on spring but I don't think much at runtime, and I just call the java classes directly without any of that factory/bean nonsense

(def ^PasswordEncoder password-encoder
  (PasswordEncoderFactories/createDelegatingPasswordEncoder))

(defn create!
  "Creates and returns a user record."
  [db email password]
  (sql/insert! db "\"user\"" {:email (string/lower-case email)
                              :password_hash (.encode password-encoder password)}))

(defn confirm-password
  "Takes a user and a password to check to see if it is the password for the user."
  [user password]
  (let [password-hash (:user/password_hash user)]
    (.matches password-encoder password password-hash)))

(side project, not work project so I can share code)

I respect all of that :) probably the world would be a safer place if there more lang-independent test suites around for crypto, csrf, etc. Like https://github.com/nst/JSONTestSuite but for this domain Else our security boils down to trust. There isn't a categorical difference between trusting Spring or trusting a high-quality :clj: lib

Yeah, which - there is a categorical difference between stuff like buddy and https://github.com/jcf/oauth-two

no offense to jcf or his code. I can't think of any obvious things he would get wrong or can even justify oauth being an issue

but it is marked as "This project is under active development, and has yet to reach 1.0. As such the API may change." in the readme, has 4 releases, hasn't been updated in 5 years, and has 17000 downloads on clojars

Good morning, I have a function that I think produces a lazy sequence:

(defn all-files []
  (let [grammar-matcher (.getPathMatcher 
                          (java.nio.file.FileSystems/getDefault)
                          "glob:*.{txt}")]
    (->> "./resources/"
         <http://clojure.java.io/file|clojure.java.io/file>
         file-seq
         (filter #(.isFile %))
         (filter #(.matches grammar-matcher (.getFileName (.toPath %))))
         (map #(.getAbsolutePath %))))) 

"/home/my_home/clojure/./resources/data/4/file1.txt"
"/home/my_home/clojure/./resources/data/4/file2.txt"
"/home/my_home/clojure/./resources/data/4/file3.txt"
"/home/my_home/clojure/./resources/data/4/file4.txt"
Execution error (NullPointerException) at (REPL:1).
null

maybe. might want to (clojure.repl/pst *e) when you get it to see the stack trace

(you'll want to be careful reading resources as files too - this won't work if you package this code+resources in a jar)

Never did the stack tracing, will try to figure it out, thank you.

I want to do a couple http/rest api calls. The api endpoint has rate-limiting. I am looking for a good way to structure the workflow. So say I start with 1 api call, then update an atom based on the result, then do more calls. Sometmes I have to iterate throuh paged results, etc. Is there some kind of example with core.async avaiable, where I could see how to structure a workflow like that?

https://github.com/brunoV/throttler the source here is pretty small and worth checking out even if you don't want to use the library

we have some work in progress towards adding something like this to core.async, @ghadi can probably drop a gist

Thanks @jjttjj and @alexmiller

I am using throttler. But I find thst he orchestration code I have is really bad. In javascript there are a ton of libraries that use promises to structure workflows. AndI would like to write clj + cljs code with core.async that has similar functionality.

The actual code that does the work is easy to write in clj + cljs.

But the coordination to structure a workflow is pretty difficult

Iteration is feedback

which is to say, if you have something like clojure.core/iterate, the way it works is it produces a seq by calling a function to get the first element, and then feeding that first element back into the function to get the rest

iteration is what you want for paginated apis, because each page is a page of results + some "next" value that you use to get the next page

with core.async there are sort of two ways to build an interative process

1. take a function and iterate it (lifting a function into a process) 2. take a process and iterate it

iterating a a function to turn it in to a process is pretty straightforward, it looks just like using iterate to build a lazy sequence, but instead of constructing a lazy seq you are sending to a channel

iterating a process involves adding a backward edge (like another channel), that takes data from the output and feeds it back into the input

I know anonymous functions with multiple arguments aren’t super popular here, but I hope someone enjoys adding this to their utils namespace:

(defn map-curry [f coll]
  (map #(apply f %) coll))

(map-curry #(hash-map %2 %1}) {:a 1 :b 2})
;; => ({1 :a} {2 :b})
;; Compare to:
;; (map (fn [[k v]] (hash-map %v %k))  {:a 1 :b 2})

;; Also works on tuples!
(map-curry #(+ % 1 (/ %2 %3)) [[1 2 3] [4 5 6]])
;; => (8/3 35/6)

where does the curry come in?

Thanks @hiredman this is what I need. Do you know where I might get some kind of example for this two approaches?

Seeing the construct #(apply f %) reminds me of a question. Rather than the anonymous fn syntax, I prefer the higher-order-fn approach of (partial apply f). However, that used to come with a performance penalty. I see that https://clojure.atlassian.net/browse/CLJ-1430`partial`, but I thought there was also something about getting it close to anonymous function performance in a release at one point? I’m not seeing it though, so maybe I was wrong. I’m wondering if these two constructs are going to be about the same in performance, or if anonymous functions are still faster. Does anyone know please?

I do not. https://github.com/clj-commons/useful/blob/master/src/flatland/useful/seq.clj#L129-L148 is a seq version of unfold(or iterate), you can write something similar that sends output to a channel instead of cons'ing up a seq

in terms of performance the problem is apply not partial ; )

The question is in terms of: (partial some-fn) vs. #(some-fn %)

The only reason I mentioned apply was because that was the construct in the above message from Max

I’m influenced by the fact that Rich has said he considers partial to be less idiomatic than an anonymous function.

I hadn’t heard that. Thank you

(e/qb 1e5
  ((partial tmpfn) nil)
  (#(tmpfn nil)))
;; => [7.33 4.55]

Do you mean?

(e/qb 1e5
  ((partial tmpfn) nil)
  (#(tmpfn %) nil))

An almost verbatim reproduction of the test from that issue:

Clojure 1.10.2
user=> (require '[criterium.core :refer [bench]])
nil


user=> (let [f (partial + 1 1)] (bench (f 1 1)))
Evaluation count : 575498700 in 60 samples of 9591645 calls.
             Execution time mean : 98.891101 ns
    Execution time std-deviation : 1.024939 ns
   Execution time lower quantile : 98.103690 ns ( 2.5%)
   Execution time upper quantile : 100.146501 ns (97.5%)
                   Overhead used : 5.615028 ns

Found 4 outliers in 60 samples (6.6667 %)
	low-severe	 3 (5.0000 %)
	low-mild	 1 (1.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
nil


user=> (let [f (fn [a b] (+ 1 1 a b))] (bench (f 1 1)))
Evaluation count : 6352825620 in 60 samples of 105880427 calls.
             Execution time mean : 3.946144 ns
    Execution time std-deviation : 0.466162 ns
   Execution time lower quantile : 3.811868 ns ( 2.5%)
   Execution time upper quantile : 3.993066 ns (97.5%)
                   Overhead used : 5.615028 ns

Found 1 outliers in 60 samples (1.6667 %)
	low-severe	 1 (1.6667 %)
 Variance from outliers : 77.1883 % Variance is severely inflated by outliers
nil

@quoll A silly mistake, but the result is basically identical

In my case, the result is quite different. :) 99ns vs 4ns.

This is why I don’t trust myself benchmarking things like this 🙂

Thank you for the comparisons

I imagine, the results may also vary greatly between JVMs. But I'm too lazy to actually check it. Especially since I almost never use partial myself.

@p-himik probably because you're declaring the function in let, and #{ wastes time declaring the function on the fly with each iteration of the bench

That lambda ends up being compiled as a class, once. It's not recompiled on each execution.

partial is useful when you have a multi-arity function that you want to curry.