Clojurians Log v2

a lot of functions have the following args spec:

(s/def ::filter-fn-args
  (s/alt :transducer (s/cat :f ::ss/ifn)
         :seq (s/cat :f ::ss/ifn :coll ::ss/seqable)))

This spec is used e.g. for filter , keep, remove and partition-by

filterable? 😛

Ah, no, misread the spec.

the idea is you walk over some collection with a fn

and the function can return a transducer or seq

I think “pred” is a more meaningful name than “f”

yeah, I usually follow the names of the arguments of the actual function that’s being spec’ed

I’m actually looking for a better name for ::filter-fn-args which is not good if you want to re-use this for keep, remove and partition-by. Maybe I should just inline it.

functor-transducer-args

?

that might be close.. but a functor cannot transform the outer data structure, which is what filter (leaving out things), keep and partition-by do

the common things in these is that a function is applied to all elements in a sequence (the elements themselves may be transformed, or not)

bleh

ok, ::bleh

🙂

::fn-over-seq … ooor just inline and not re-use it

while these functions all have the same signature, you are detecting repetition, not abstraction

good point

there is nothing to say that one of these functions won’t gain an arity or something in the future

repetition is not always a bad thing imho

ok, you confirmed my inclination to inline it

and to some degree I think there is clarity to just having them each be separate but the same

💡

@slipset I tried to look up the “category theory” term for filter that works on more than just a list, and now I finally get what MonadPlus is about 😄, thanks https://stackoverflow.com/a/3011500/6264

(well sort of, but now I have a clue)

interesting remark in #clojure just now by @alexmiller > expectation for sequence functions should be that take something seqable and return something seqable there was a change recently in speculative which spec’ed the return value of some sequence functions more strictly: https://github.com/borkdude/speculative/commit/fcadecebb411cfa18af360991b2e4a6c1d8468b2 should we revert it?

(the fn specs were restored in a later commit)

next/rest are a little different case

and I think their returns are constrained by ISeq

but on the seq-or-transducer, I think that should be seqable

because you can not rely on map giving back a seq?

this is not about rely on, but about commitment

what’s the difference?

and seqable is a less constrained commitment (regardless of whatever it actually does now)

less constrained, but sufficient

right. for optimization it might return a vector or whatever, you never know right

there are some subtle cases where being able to return something seqable, means you don’t have to force the creation of a seq object, and in some cases the perf difference of this is important

ok, good to know. I think I’ll document this in the guidelines for future specs

btw @alexmiller do you know such an example from the top of your head?

I ran into it when I was doing the 1.7 transducers work on range, repeat, cycle, iterate, etc but I don’t remember the specific case

I can see arguments that map, filter should commit to seqable - But from my view they have a distinct "return-type" compared to take-last. Probably wrong but... map can't return nil and I would think that is true for all "transducable"(?) functions. take-last on the other hand can return nil.

soo... (s/and seqable? some?)?

seqable? includes nil

if map returned nil I wouldn’t say it was wrong

Hmm... Can't say I see how it would return it

I’m not saying it does now

I’m saying conceptually, if it did, I think that would be acceptable

As in, your code should not break if it did.

Although the docstring says that it returns a lazy-seq. Not all docstrings cover the types very precisely though, so that’s not something to go by 100% of the time.

Pipeline with map functions. Last function in pipe does a (when % (println "still something"). Probably bad structure by me, but I often check for something.

this is actually the place where you should use seq

(when (seq %) (println ...))

that is bad code when working with sequences

ok if working with collections

because map cannot be expected to return something?

It might be general code, perhaps I don't even know if it is a sequence or not?

you were talking about a pipeline with map functions. that should at least return a seqable?.

(assuming no Exceptions happened)

Say that we have heterogeneous collection. I do some kind of transformation of all elements. (map my-multi-method coll) int -> #(* % %) seqable -> #(map inc %)

can you give a full example? I have trouble understanding this one

https://gist.github.com/bonega/17f8d77e34fcfcb66951effb10f6407a

ok and now your point?

I might want do checks later if (when (nth x coll) (do-something)). But I have no idea if this is a "real" nil or not

what is the goal of the check?

I mean, what is your definition of a “real nil”

In that gist you posted, I would expect you would want to return a vector, not the result of map

So I would use mapv or ‘into’ to []

Or vec

vector -> vector makes sense

@alexmiller is your point that empty vectors are less often interchanged with nil than empty sequences?

Perhaps I have too strict(wrong) of a view regarding what kind of thing map computes over.

still I find this interesting: https://clojure.org/reference/lazy

Changed: Sequence fns (map, filter etc) return seqs, but not nil You’ll need to call seq on their return value in order to get a seq/nil seq also serves as test for end, already idiomatic (when (seq coll) ...) allows full laziness doesn’t support nil punning since sequence fns no longer return seq/nil

but it is not a reference doc though

🖕

also it is about what changed, not about the future. So your point still stands

I think we should stick to the general contract of seq functions which is seqable in and out. Like, you’re also not going to cast the return value of something which implements IFoo because you know it’s a Bar under the hood?

but it would be better if we had this commitment in a reference doc somewhere, so we could point to it

does it exist somewhere?

nil and empty vector are different things

Esp in a use case where you are using positional vectors like this one

But with sequences nil “puns” as an empty sequence

I am interested in knowing a counterexample. What kind of code are we writing today that is hindered by disallowing nil? Wouldn't that just be a conditional (if (map..)) that have no chance of executing the later branch? Then when clojure possibly change in the future, that branch might start executing. The removal of nil-punning as described was a breaking change. That breaking change should probably also be reflected in eventual specs.

My reason for reverting the change is that we don’t want to spec implementation details. () or nil as a return value from a sequence function is an implementation detail, not part of the contract

Also see https://clojurians.slack.com/archives/CDJGJ3QVA/p1547228993128100?thread_ts=1547227519.119700&cid=CDJGJ3QVA

But it is something that changed for users when writing programs. Code that used to do (if (map ...)) stopped working when the change was made. We shouldn't write program today expecting nil as return, at best they will break in the future. A lot of things are implementation details.

map stopped returning nil by design and would have broken all code dependent on that behaviour

Are you referring to the historic document that is not reference documentation?

yes

today the contract for sequence functions is seqable? to seqable?, whether map returns () or nil is not something you should be thinking about

any opinions on this @mfikes or @slipset?

(if (map a-fn coll) (do-something) (do-else))

Of the top of my head, I’ve thought map returns (s/nilable seq?)

That is correct, it is part of a contract though? - yeah minus nil

Actually, perhaps nix the nilable part

that’s the question. also: should we consider different return specs for different seq functions, or treat them under the same contract

Is the rough idea seqable? in, seq? out?

@mfikes given that () or nil may be treated as the same value (punning), the question is: is this important enough to spec differently

Trying to think of things that might return nil…

@mfikes start reading here: https://clojurians.slack.com/archives/CDJGJ3QVA/p1547221885081900

map, remove, filter can't return nil - I think it is a bug to write code that relies on them returning nil. If that changes in the future, so will your execution

you are turning things around: you should not rely on them to return () instead of nil, that should be un-important

since empty seqs can be nil-punned

so you should not rely on them returning a more specific value than seqable?

there are no such things as empty seqs

I think

’()

isn't that an emptylist?

(seq? '()) ;; true

anyhow, I am attacking this mostly from ide tooling angle. I would like to be able to mark code doing (if) on the return of a map as incorrect

Another example where this is relevant:

(re-seq #"\w+" "")

@andreas862 as @alexmiller pointed out, you should always call seq on a sequence if used as a truthy value

Sure, but that is why my tooling should mark the bare if-statement

Yes, but what if that changes in the next version of Clojure, so that nil isn't possible anymore? Nil punning on that stops working. That should probably have an effect on the spec then

You are again turning it around. You should never nil pun on a result of a sequence function…

Yes, and we can have tools that helps with enforcing that

Or...

hm

Sure, but those tools should be able to leverage seqable? and nothing more than that

is (when (re-seq ...)) a bug?

yes

depends. you should call seq on it if you want to confirm if it’s non-emty

it’s irrelevant if it returns nil or an empty seq when it didn’t find anything

interesting, got any example for re-seq returning an empty-seq instead of a nil?

that should not matter

you’re not supposed to look at the implementation and then figure out that it’s never going to

Another example where you shouldn’t rely on seq? “Returns a sequence of values”

(vals nil) ;;=> nil
(vals {}) ;;=> nil

My argument has always been about the ones that I classified as not nil-punnable. map, filter, remove etc. I did know about some others returning nil

But have to say that I do probably use some nil return directly

have a lot of code to fix

probably re-seq, not sure about the rest

you were depending on re-seq being able to return nil, despite the docstring?

I think in that case you should use re-find

if you want to check if there is at least one match

not that sure about re-seq, have to check. But there is a big risk that I would use one of them if I found that it seemed to leave nil for empty.

We already discussed this in the very start of speculative: https://github.com/borkdude/speculative/issues/45 That’s why we made it seqable? in the first place. Sorry I forgot about this @andreas862. Should have remembered when I merged the change.

a non-nil seqable isn't invalidated by that issue.

seq? goes out the door though

For now I reverted back to #45. Any changes to this we should discuss in that issue or at least keep the most important notes there, so we can fall back on that.