Clojurians Log v2

any dockerized helloworld clojure app examples out there?

Hiya folks! I’m working on Windows newline bug in rewrite-clj, and while digging in, I’ve found that clojure.tools.reader push back reader read-char and peek-char seem to treat \r\n differently. To demonstrate, I’ll setup a reader like rewrite-clj does and dump peek and read behavior:

(defn peek-behaviour [s]
  (let [rdr (-> s 
                r/string-push-back-reader
                r/indexing-push-back-reader)]
    (map (fn [ndx]
            [ndx :peek (r/peek-char rdr) :read (r/read-char rdr)])
          (range (count s)))))

(peek-behaviour "o\nhey")
;; => ([0 :peek \o :read \o]
;;     [1 :peek \newline :read \newline]
;;     [2 :peek \h :read \h]
;;     [3 :peek \e :read \e]
;;     [4 :peek \y :read \y])

(peek-behaviour "o\rhey")
;; => ([0 :peek \o :read \o]
;;     [1 :peek \return :read \newline]
;;     [2 :peek \h :read \h]
;;     [3 :peek \e :read \e]
;;     [4 :peek \y :read \y])

(peek-behaviour "o\r\nhey")
;; => ([0 :peek \o :read \o]
;;     [1 :peek \return :read \newline]
;;     [2 :peek \newline :read \h]
;;     [3 :peek \e :read \e]
;;     [4 :peek \y :read \y]
;;     [5 :peek nil :read nil])

@lee Maybe non-significant difference, but I think rewrite-clj uses clojure.tools.reader.edn under the hood. Probably uses the same functions though.

I do not know the specifics of that issue, but I do know that trying to maintain compatibility with Clojure's reader, maintaining column and line info, handline \n vs \r\n differences, and some corner cases involving look-ahead then unread 1 character used in some cases, can be notoriously confusing to handle in clojure.tools.reader (from personal involvement in examining test cases submitted by others in earlier JIRA issues, and trying to figure out why they occurred).

oh, and contributing to the difficulty is that there can be different 'layering' of JVM IndexedPushbackReader vs. BufferedReader vs. different combinations of those.

It seems tools.reader has some normalize? option to normalize \r\n to \n, in read-char and unread-char, but perhaps this normalization isn't needed in peek-char as peek-char is often just used to determine if the next char is some special character and not some whitespacy thing

Thanks @borkdude & @andy.fingerhut! The lack of symmetry on read-char vs peek-char for newlines is a bit surprising at first glance, but maybe there’s a good reason for it that I just don’t know yet.

@lee @bronsa can probably give the definitive answer

@lee So why does rewrite-clj have problems with this?

It is the way it currently parses comments. It is the only element is parses to end of line. If a comment ends in \r\n it gets confused due to the peek-char/read-char mismatch. I can work around current tools.reader behaviour, I’m sure.

why doesn't it use read-char until it gets a newline?

I will switch to exactly that. It tries to reuse one of its utility functions to parse to end of line.

Just thought the odd behaviour was worth raising.

Hello guys! I have a collection of commands that I need to proccess and each command causes an update in my "state" so I used a reduce to apply every command using an empty state as first val, but one thing I need is to print every intermediate state and I can't do that with a reduce because I'll only have the last state at the and, also I can't use a map because one command should be applied to the previous state result 😕 and I'm struggling with putting the print inside the function that reduce calls because looks like a bad practice causing a side effect inside the function with business logic

Is there an idiomatic way of doing this?

Sounds like the job for reductions, I think.

wow that's perfect! Exactly what I wanted

Thanks!

since you only need the intermediate values for a side effect, also consider a middleware / wrapping pattern -

(defn logged [f]
  (fn [& args]
    (print-args args)
    (apply f args)))

then (reduce (logged state-f) init coll)

nice one too thanks!

I'm curious why reduce-kv for a priority-map calls the reducing function with item as the key and priority as the value (https://github.com/clojure/data.priority-map/blob/master/src/main/clojure/clojure/data/priority_map.clj#L395-L397) instead of the map's key and value. Calling reduce will be passed the map's key and value. I'm assuming it has something to do with this particular data structure's origins, but I cannot find any documentation pointing me to in in the clojure.data.priority-map ns.

my guess would be that's a good way to ensure that you get all of the same priority items at the same time. otherwise you might get a low priority, a high priority, a low priority of the same precedence as the previous one, etc

reduce ensures order

priority->set-of-items is a map i think?

if its reducing over an unordered thing then i don't see how this implementation does

I mean calling reduce on a priority map, not that internal map.

ah that's a sorted map

i'm not following. calling reduce on the priority map is implemented as that above, calling reduce on the sorted map priority->set-of-items

but i guess its a good decision that when reducing you're given every item of equal priority at the same time

Calling reduce-kv is implemented as above, not reduce

oh i missed that distinction

Since PersistentPriorityMap doesn't implement IReduceInit, it must be calling coll-reduce

that seems like a good point

the reduce and reduce-kv will be quite different

Right. Seems odd, right?

yeah

certainly when there's a custom reduce-kv way implemented

Yep. Makes me think it was intentional, which led me to my guess of having to do with priority map CS origins or something

i see in the reducers ns there's some stuff where the reduce-kv is used if present

(defn reduce
  "Like core/reduce except:
     When init is not provided, (f) is used.
     Maps are reduced with reduce-kv"
  ([f coll] (reduce f (f) coll))
  ([f init coll]
     (if (instance? java.util.Map coll)
       (clojure.core.protocols/kv-reduce coll f init)
       (clojure.core.protocols/coll-reduce coll f init))))

but i don't remember seeing anything like that in the reduce pathway

although its pretty complicated so maybe i missed it

Not sure I'm following what you're looking for.

i was wondering if there was some code pathway that checked if it wasn't ireduceinit but it was IKVReduce that maybe it would use that pathway

reduce on priority map will hit https://github.com/clojure/data.priority-map/blob/2adf0d7efb2be75b936bf2ff1e3e400d74ab64aa/src/main/clojure/clojure/data/priority_map.clj#L391 which returns a SeqIterator which calls https://github.com/clojure/clojure/blob/140ed11e905de46331de705e955c50c0ef79095b/src/clj/clojure/core/protocols.clj#L33. Not relevant tho haha

Main question is why the author added a custom reduce-kv impl that does not pass what I'd expect for the value. Is it intentional? Bug?

e.g.,

(reduce-kv
    (fn [acc k v]
      (assoc acc k v))
    {} m)
=> {:b 1, :a 2}
(reduce
    (fn [acc [k v]]
      (assoc acc k v))
    {} m)
=> {:b [1 :bananas], :a [2 :apples]}

It seems like the correct implementation of reduce-kv would be

(reduce-kv (fn [a priority items]
             (reduce (fn [a item] (f a item (item->priority item))) a items))
  init priority->set-of-items)

This part of the readme describes approximately how reduce-kv is working > In addition to supporting all the functions a sorted map supports, a priority map can also be thought of as a queue of [item priority] pairs. To support usage as a versatile priority queue, priority maps also support conj/peek/pop operations.

So is it expected that reduce-kv on a priority map should work like that queue? IMO, that seems incorrect -- use the seq interface or queue fns.

what is m two code blocks ago?

(def m
    (clojure.data.priority-map/priority-map-keyfn-by first compare
      :a [2 :apples] :b [1 :bananas]))