Clojurians Log v2

btw i think i found the problem

(apply concatv rest)

Oh, interesting... Does that introduce laziness somehow?

(at least that is "one" problem)

well, it's a recursive call

OH! Yeah, that sounds so obvious once you say it out loud! 🙂

let me explain. sorry, i noticed that even if I remove the :fn I still get the crash with the original impl of concatv. but with this loop/recur one (inspired by the Stuart Sierra blog on concat) that crash won't occur:

(defn concatv
  "Strict version of concat."
  [head & tail]
  (loop [seqs (cons head tail) acc []]
    (if (empty? seqs)
      acc
      (recur (rest seqs) (into acc (first seqs))))))

Seems weird to have head & tail when you only cons them together and never use them again.

oh, could just be [& args] ?

good point

You might want to consider whether (concatv) should work and what it should produce (I'd say yes and [] respectively).

i'm just trying to match behavior of the original concat (which seems to agree w/ you) 🙂

cljs.user=> (concat)
()

and yea that concatv does similarly:

cljs.user=> (concatv)
[]

I just tried it in Clojure and got an error

Oh, it works after changing to & args and (loop [seqs args ...

hmm, mine was from lumo

yea, sorry

didn't paste in here

(defn concatv
  "Strict version of concat."
  [& args]
  (loop [seqs args acc []]
    (if (empty? seqs)
      acc
      (recur (rest seqs) (into acc (first seqs))))))

The version with [head & tail] requires 1+ arguments 🙂

haha, yea 2+ or 0+ makes more sense than 1+

here is a "final" version that works (tested up to 75 iterations at a time):

(s/def ::any-coll
  (s/with-gen
    (s/coll-of any?)
    #(s/gen (s/coll-of any? :max-count 125))))

(s/fdef concatv
  :args (s/cat :args (s/* ::any-coll))
  :fn #(let [r (:ret %1)
             args (-> %1 :args :args)
             sumhash (fn [c] (apply + (mapv hash c)))]
         (and
          (= (count r) (apply + (mapv count args)))
          (= (sumhash r) (apply + (mapv sumhash args)))))
  :ret (s/coll-of any? :kind vector?))

(defn concatv
  "Strict version of concat."
  [& args]
  (loop [seqs args acc []]
    (if (empty? seqs)
      acc
      (recur (rest seqs) (into acc (first seqs))))))

so yea i think i was kinda confounded earlier by the fact that there was a potential stack overflow in the code under test. so even when i got the test code "right" the code under test could screw me

but it was somewhat non-deterministic on both sides. bit of an entangled heisenbug

🙂

two entangled heisenbugs rather

Complected, even 🙂

haha exactly

Did you consider (reduce into [] args)?

small update: so ... there turned out to be 3 problems. my hunch about s/* was also correct. some of the crashes were emanating from it as well. here's a "more final" version (at least until I get another random, intermittent failure some time in the future):

(defn arbitrarily-partition [coll freq]
  (let [signals (take (count coll) (cycle (cons true (repeat freq false))))
        shuffled (shuffle signals)
        zipped (map vector shuffled coll)
        partitioned (partition-by first zipped)]
    (for [c partitioned]
      (map second c))))

(s/def ::coll-of-colls
  (s/coll-of (s/coll-of any?)))

(s/def ::distinct-coll-of-colls
  (s/with-gen
    ::coll-of-colls
    #(gen/let [elems (gen/set (s/gen any?) {:max-elements 150})
               freq (s/gen (s/int-in 1 10))]
       (arbitrarily-partition elems freq))))

(s/fdef concatenate
        :args (s/cat :args ::distinct-coll-of-colls)
        :fn #(let [r (:ret %1)
                   args (-> %1 :args :args)
                   sumhash (fn [c] (apply + (mapv hash c)))]
               (and
                (= (count r) (apply + (mapv count args)))
                (= (sumhash r) (apply + (mapv sumhash args)))))
        :ret (s/coll-of any? :kind vector?))

(defn- concatenate
  "Strict version of concat."
  [args]
  (loop [seqs args acc []]
    (if (empty? seqs)
      acc
      (recur (rest seqs) (into acc (first seqs))))))

(defn concatv
  "Strict version of concat."
  [& args]
  (concatenate args))

Nope, does it work?

Also, how much memory will it use? the loop/recur one is tail optimized so should use a constant amount of memory