Hi team, I’m trying to make concurrent calls to apis to gather paginated results, I’m thinking of using atom, future and loop for that, my take so far is as follows, but not sure how do I ensure not breaking out of loop before all future calls finish? Any advise please? Thanks
(let [panda-mappings (atom {})
range 200]
(loop [offset 0]
(future (swap! panda-mappings assoc offset "val"))
(if (> offset range)
(println @panda-mappings)
(recur (+ offset 50))))
(println @panda-mappings))
I second looking at pipeline-blocking.
However the way you use swap!
and future
can’t work. If you want to use futures you must use them to perform the actual fetch and you have to store the future inside the map. Then once all futures started you have to wait.
(loop [futures {} [offset & offsets] (range 0 250 50)]
(if offset
(recur (assoc futures offset (future (DO-FETCH offset))) offsets)
; done, let's wait
(into {} (map (fn [[o f]] [o @f])) futures)))
nice @cgrand, thanks for the idea of putting all the future
in one collections and wait
my attempt was
(let [panda-mappings (atom {})
range 200
futures []]
(loop [offset 0]
(conj futures (future (swap! panda-mappings assoc offset "val")))
(when (< offset range)
(recur (+ offset 50))))
(when (every? realized? futures)
(println @panda-mappings)))
and it works 🙂 , will need to learn your example a bit more, seem a bit cryptic for me the map part…, anyway, thanks
many people find it counter-intuitive, but when I do side effects collecting results across a collection, I prefer reduce
or into
over loop
since they bake in the traversal pattern
(into {}
;; wait on and deref futures via transducer
(map (fn [[k v]]
[k @v]))
;; eagerly start futures before further processing
(mapv (fn [o]
[o (future (DO-FETCH o))])
(range 0 250 50)))
the whole thing turns into a few very common idioms (something I aim for)
cool @noisesmith, really like this idiomatic way, thanks. One more question please, when I try doing (map (fn [[k v]] @v))
instead of (map (fn [[k v]] [k @v]))
, to get all results without the indices to the map, I get “Vector arg to map conj must be a pair” IllegalArgumentException, I’ve been mucking around without luck, any thoughts?
you can only conj a key/value pair to a hash map
replace the {} with #{} or [] and it works, if you didn't need the keys
results is
I’m trying to get something like this instead…
{100 val, 200 val, 150 val, 250 val, 0 val, 50 val}
clojure.core.async has some ideas about how to communicate like this. you could check out clojure.core.async/pipeline-blocking
you mean future is not the right tool for this kind of problem?
there are many tools for this. but core.async can give you good coordination tools for your problem of waiting until all the requests finish
pipeline-blocking will actually use thread underneath it but with async channels to communicate results and pending results
Am following a Clojure Tutorial - Anyone has an idea why (shutdown-agents) causes .... : "Exception updating the ns-cache #error"? I can skip this but am just wondering o:
(defn agent-ex []
(def tickets-sold (agent 0))
(send tickets-sold + 15)
(println)
(println "Tickets " @tickets-sold)
(send tickets-sold + 10)
(await-for 100 tickets-sold)
(println "Tickets " @tickets-sold)
(shutdown-agents))
(defn -main [& args]
(agent-ex)
)
That's not a message from Clojure itself, must be something from your tooling. If something in your tooling is using agents or futures, shutdown-agents could be breaking the tooling itself possibly
Right! Maybe NREPL itself then o:
looks like it
Right! Thanks for your help! 🙂
@zackteo def
should only be used at the top-level. If you want a local binding, use let
instead.
And, yeah, most REPL tooling will break if you call (shutdown-agents)
since it will kill off stuff inside the tooling, not just your program.
The safer idea is to move the (shutdown-agents)
call into -main
so it's
(defn -main [& args]
(agent-ex)
(shutdown-agents))
that makes your agent-ex
function safe to call from the REPL.I'm trying to use Double/isNaN
as a predicate in some
, but clojure interprets it as a static field. Is this the idiomatic way of dealing with such things?
rules-engine.thrashing02=> (Double/isNaN ##NaN)
true
rules-engine.thrashing02=> (some Double/isNaN [##NaN 2])
Syntax error compiling at (form-init5367077877367404867.clj:1:1).
Unable to find static field: isNaN in class java.lang.Double
rules-engine.thrashing02=> (some #(Double/isNaN %) [##NaN 2])
true
#(Double/isNan %)
is the idiomatic way
i believe the reason is that methods are not first class objects on the JVM.
I really appreciate the built-in predicates for string?
, vector?
, etc, rather than having to go through another function, like Python's isinstance("sheep", str)
, especially in combination with other functions like some?
or map
.
Ok, thanks. Only quirk I have is that this is already in an anonymous function, which I cannot nest, so I have to do something else. But this makes sense, thank you.
make either the outer or the inner a (fn [thing] ...)
version. probably the outer since #(Double/isNan %)
is quite simple
yep, that's what I did:
(fn [it] (Double/isNaN it))
and yep, it's simple, so the fn
construct works well.
Hey
it might be worth making a nan? function in core
(def first-test '(not x))
(def second-test '(not (not a)))
(def third-test '(not (not (and a b))))
(def fourth-test '(not (not (not (not c)))))
(defn not-elmination
[expression]
(cond
(= first-test expression) set '{}
(= second-test expression) set '{a}
(= third-test expression) set '{(and a b)}
(= fourth-test expression) set '{(not (not c))}
:else set '{})
)
I am trying to return sets here and im getting this error "Map literal must contain an even number of forms"Anyone know why?
#{}
is the set literal, {}
is the map literal
set
there makes no sense, what are you trying to do
So if the user enters '(not x), the cond checks and i want to return a set {(not x)}
ok, set '{(not x)}
makes no sense, you want #{'(not x)}
'#{(not x)}
also works, it'll quote the entire set instead of the specific element but in this case they're the same thing
So i should be returning #'{a} for example
no
#{'a}
or '#{a}
those are equivalent to #{(quote a)}
and (quote #{a})
respectively
#'{
is not valid syntax
the syntax for a set is #{ elements.. }
so you can use '
either in front of the whole set to quote all the values in the set, or in front of an element in the set
Ok thanks for the information, I will apply that to my code rn
hmm now im getting "cond requires an even number of forms"
(def second-test '(not (not a)))
(def third-test '(not (not (and a b))))
(def fourth-test '(not (not (not (not c)))))
(defn not-elmination
[expression]
(cond
(= second-test expression) #{'a}
(= third-test expression) #{'(and a b)}
(= fourth-test expression) #{'(not (not c))}
:else set '#{})
)
@ntrut Parentheses are important in Clojure and mean "function call" (in general) so set '{}
is two unrelated expressions but (set '{})
would be a function call -- and would call set
on '{}
:else set '#{}
is wrong
(because it is three separate expressions)
Sorry, its hard getting used to clojure, just recently starting learning it in one of my classes
No worries. Lisps are always a bit tricky to get used to.
And it also depends what languages you're used to.
In Java or similar: obj.method(arg)
In Clojure (method obj arg)
Defiantly not used to this, I code in Java/C/Pyhon
New territory for me here
What is wrong with my else?
:else expresssion
but you have :else a-function an-empty-set
(also, you can just use #{}
as an empty set. there are no forms in it that you are quoting)
Yep using #{} fixed my else problem and now my program works
thanks people
Regarding the infer-externs
cljs compiler option: Are you intended to move the generated externs in inferred_externs.js
to a file that you use for advanced compile? Or is it supposed to be enough to use type hinting and infer-externs
when you use advanced compile?
@mitchell_clojure If no one answers here, you might try #clojurescript -- I'm not sure how many active folks here are familiar with ClojureScript compilation issues.
Thanks Sean! Hard to know sometimes if I am missing something simple sometimes
if i have an defrecord instance, is there a simple way to create a new one with just one value substituted, sort of like update-in does for maps?
you can use update
records are maps
or assoc
into it
user=> (defrecord Point [x y])
user.Point
user=> (def r (->Point 1 2))
#'user/r
user=> (update r :x inc)
#user.Point{:x 2, :y 2}
user=> (assoc r :x 10)
#user.Point{:x 10, :y 2}