Clojurians Log v2
Clojure programming
Channels
Apps
clojure-dev
Issues: https://clojure.atlassian.net/browse/CLJ | Guide: https://insideclojure.org/2015/05/01/contributing-clojure/
dominicm 2019-09-21T05:56:29.040700Z
http://clojure.github.io/clojure-contrib/dataflow-api.html links to http://dev.clojure.org/display/community/Where+Did+Clojure.Contrib+Go which is 403
alexmiller 2019-09-21T13:53:05.040900Z
alexmiller 2019-09-21T13:53:14.041100Z
is a migrated version
alexmiller 2019-09-21T14:46:42.041300Z
docs updated
ikitommi 2019-09-21T17:48:56.044400Z
Hmm.. clojure.core/merge seems quite slow. Is this worth investigating or a known issue?
➜ ~ clj -Sdeps '{:deps {criterium {:mvn/version "0.4.5"}}, :jvm-opts ["-server"]}'
Clojure 1.10.0
user=> (require '[criterium.core :as cc])
nil
user=> (cc/quick-bench (merge {} {}))
Evaluation count : 2712492 in 6 samples of 452082 calls.
Execution time mean : 221.103323 ns
Execution time std-deviation : 1.892978 ns
Execution time lower quantile : 219.218989 ns ( 2.5%)
Execution time upper quantile : 223.419465 ns (97.5%)
Overhead used : 1.964440 ns
borkdude 2019-09-23T14:17:10.048Z
@ikitommi is reduce into any faster?
borkdude 2019-09-23T14:18:53.048200Z
maybe this calls for a library with a faster version of merge 🙂
borkdude 2019-09-23T14:51:25.048700Z
quite the speedup
borkdude 2019-09-23T14:52:08.049100Z
maybe include coercing from and to a Clojure map since that's what you want eventually I guess?
ikitommi 2019-09-23T15:03:22.054500Z
yeh, this is not the way…
;; 230ns
(let [m1 {}
m2 {}]
(cc/quick-bench
(clojure.lang.PersistentArrayMap/create
(.toMap (.merge (Map/from ^java.util.Map m1)
(Map/from ^java.util.Map m2)
Maps/MERGE_LAST_WRITE_WINS)))))
ikitommi 2019-09-21T17:49:28.044500Z
merge flamegraph
2019-09-21T18:50:50.044900Z
A merge of an empty map in to an empty map is basically the worst case to micro benchmark, what would make it faster is checking for empty maps at the beginning and doing nothing, which wouldn't speed up any real world use cases
slipset 2019-09-21T19:17:03.045100Z
I guess you could gain some speed up by introducing multiple arities for the low arity cases.
slipset 2019-09-21T19:17:47.045300Z
(merge {} {}) is (conj {} {}), no need to start the full reduce1 machinery for that.
slipset 2019-09-21T19:29:01.045900Z
user> (cc/quick-bench (merge {} {}))
Evaluation count : 2421150 in 6 samples of 403525 calls.
Execution time mean : 250.990304 ns
Execution time std-deviation : 9.293609 ns
Execution time lower quantile : 243.127947 ns ( 2.5%)
Execution time upper quantile : 266.075269 ns (97.5%)
Overhead used : 2.076286 ns
Found 1 outliers in 6 samples (16.6667 %)
low-severe 1 (16.6667 %)
Variance from outliers : 13.8889 % Variance is moderately inflated by outliers
;; => nil
user> (defn unrolled-merge
([m1] m1)
([m1 m2]
(conj (or m1 {}) m2))
([m1 m2 & ms]
(#'clojure.core/reduce1 conj (merge m1 m2) ms)))
;; => #'user/unrolled-merge
user> (cc/quick-bench (unrolled-merge {} {}))
Evaluation count : 7737186 in 6 samples of 1289531 calls.
Execution time mean : 75.288592 ns
Execution time std-deviation : 1.673650 ns
Execution time lower quantile : 73.961836 ns ( 2.5%)
Execution time upper quantile : 78.025617 ns (97.5%)
Overhead used : 2.076286 ns
Found 1 outliers in 6 samples (16.6667 %)
low-severe 1 (16.6667 %)
Variance from outliers : 13.8889 % Variance is moderately inflated by outliers
;; => nil
user> slipset 2019-09-21T19:30:35.046100Z
https://ask.clojure.org/index.php/2770/enhance-the-performance-of-map-merges?show=2770#q2770
slipset 2019-09-21T19:34:50.046400Z
The ticket is here:
slipset 2019-09-21T19:34:51.046600Z
slipset 2019-09-21T19:36:09.046800Z
The approach taken is to introduce merge1 which doesn’t use reduce1, and then reimplement merge later when the proper reduce is available.
ikitommi 2019-09-21T20:37:03.047200Z
In my case, it was a real-world merge of small maps. But the time it takes was about the same as with empty maps, just to highlight the issue. Last comments from 2017: "This ticket needs help. Step 0 is writing a benchmark harness that exercises maps of various sizes, and the various polymorphic arguments below.".