Clojurians Log v2

morning....

Good morning.

Our Clojure library, to-jdbc-uri, has tests and 100% code coverage. So far, all contributors, whenever they added a feature, they also added tests. I think Clojurians test more or are willing to test more than we give them credit for.

@pupeno: well spoken

i might be unpopular with this but I don’t think testing and code coverage has a huge value by definition. you always have to consider the context. of course for an open sourced library it makes sense to have unit tests and high coverage. so I think/hope Clojurians aware of this (perhaps obvious?) contextual nature of testing and do extensive testing when it makes sense...

@benedek: all numbers are useless when you consider it gaming them.

not sure i follow @pupeno, sry

@benedek: why do you think testing and code coverage has no value by definition?

i think it is very dependent on the context. I would possibly rephrase this and say that you need a fast/short feedback loop when doing development. in certain situations a big and well maintained test set gives the fastest feedback loop. and other situation such a test set is just too ‘expensive’ to create/maintain

sry for the shameless self promotion but this blog post is kinda about this: http://benedekfazekas.github.io/2015/01/16/cheap-webapp-regression-testing/

OK. Never mind then.

You had something else in mind.

how so?

@pupeno: @benedek I suspect you are both talking of the same thing...

you just managed to word it very differently.

I’m just tired of the constant devils advocate. Whenever I say “Woohoo! I increased code coverage form 78% to 85%” someone points out how code coverage can be gamed and thus it’s useless. I find the same with BMI (Body Mass Index). Every time you use that concept someone will point out how it’s useless because for high performing athletes with less than 8% body fat and more muscles than an elephant, so it’s useless for everybody.

sry @pupeno did not want to offend

I’m not offended :simple_smile:

I see more people not testing when they should than the other way.

I don't just assume tests provide a feedback loop although this is the reason I use TDD when I don't have a REPL. What about tests as regression coverage during refactoring and as documentation of requirements? I have lots of first hand evidence of the later saving my bacon when trying to learn a new context.

re regression coverage: exactly my point in the blog post. you deffo need something to catch regressions most of the times. only a unit test test might not be the cheapest option

I agree. In my last role as CTO I had a discussion with my senior dev's around deleting unit tests once we had used them for feedback and providing that we could prove coverage in integration/functional tests.

@pupeno: that is abs. a fair point. not contradicting that. but I guess we have to consider that coverage/test set comes with a price that is why sometimes ppl just decide not to invest in it

However, I used to insist on rigorous test coverage with a less experienced team.

@benedek: something that I notice is that people say going from 0 to 100 with tests is so much slower, and they ignore that tests make going from 100 to 200 faster. They are an investment.

@pupeno: I agree. But to @benedek's point we have to pay interest on that investment during change so we need to be careful to not double or triple cover the same functionality.

Sorry, I been burned by developers writing code that has a lot of coupling and no tests and now we have to treat the whole thing as a fragile tower that we have to stay away from because if we touch one thing, it might all collapse without us realising so.

I have never found using high test coverage is slower long term than low test coverage...ever. As the time spent rewriting tests is never as bad as time spent debugging!

However, I think, as with all things there's a balance to be struck.

I think the balance needs experience and judgement.

Therefore my default position will always be high test coverage at all levels

I also find that coverage is able to shift the team’s focus from real problems, real user value sometimes. At a place I worked a couple years ago when the lead architect went to holidays he gave the stretch task to the team to increase the coverage from 85% to 95%. I felt it was a wasted effort tbh.

@agile_geek: I abs. agree with the balance thing.

As with all things, setting targets drives behaviours.

I measure coverage and use it to have a conversation. I never use it as a target.

If I want a target, what about features delivered or better ROI?

@pupeno: don’t get me wrong. you deffo need something to catch regressions. no objection there from me

I found when I had just two very senior dev's we could make decisions about when it was prudent to remove unit tests.

In the other team with two more Junior dev's we had rigid adherence to TDD!

With Snr's leading the way

In to-jdbc-uri (https://carouselapps.com/to-jdbc-uri/), we have just a single pure function that if it fails, your whole app just doesn’t start, that gets different arguments in production than in development, so it might fail in prod while not in dev. It’s very easy to test and having a full test coverage allows people to add more use cases without worrying about breaking the current one.

As I said I'd always default to paying the price of changing tests over paying the price of releasing something I'm not happy with. Start with good test coverage and make decisions based on measurements. However, code coverage is not an indication of quality tests!

@agile_geek: for me, code coverage is a multiplier. code coverage multiplied by quality of tests = the quality of your test suite. No matter how high one is, if the other one is low, your result is low.

I wrote a Web App two years ago and recruited a snr dev to take over dev from me. I had 2 hours to show her the basics of the project and how to build it then was off site for 3 days. I came back and she was making effective changes. I asked how she got on and she replied "I was stuck for an hour but then I read and ran your tests..." (readability is almost my primary measure of test quality)

@agile_geek: that’s why it would be better to measure the quality of tests by using those tools that automatically break your code and see whether your tests catch the error, but in my experience they are far from trivial to use and in most cases, reasoning about the test gives me a good enough estimate of the quality of the test. Specially in cases as simple as this: https://github.com/carouselapps/to-jdbc-uri/blob/master/test/to_jdbc_uri/core_test.clj

P.S. Sorry for butting in but I get passionate about testing! I think all programming should be taught by writing tests as the first thing after the basic syntax.

I am talking at Sunderland Uni in March about this very subject!

@benedek: @pupeno fancy a face to face chat in new year about this?

I could do with input to that talk from all perspectives

@pupeno: that abs. makes sense

@agile_geek: I think writing tests first is a good exercise, I’m not an advocate of doing it 100% of the time because I believe sometimes we are in exploratory mode. Sometimes we are playing with the clay to see how it behaves, not trying to build a statue to specification. But I’m saying this to you, in this context, because I know you won’t use this as an excuse to write 0 tests. For developers that are just looking for an excuse to not write tests, I’d rather not give them some based on nuances they are going to ignore.

@agile_geek: face to face, yes.

With the caveat that I think everybody here is probably super smart and we can sit down and think up of thousands of use cases in which a general rule doesn’t apply and keep a good conversation going forever while it might be a better use of our neurons to sit down and code and/or test.

@pupeno: I agree wholeheartedly (except I'm not smart, I've just been professional s/w dev for 28 years and made a lot of mistakes.) I really want some ideas of how to get across to post grads the importance of tests (we can discuss the drawbacks with a more mature perspective as well)

@agile_geek: oh, you are smart. And yeah, I’d be happy to talk about it and contribute whatever I want.

😊

agree with @pupeno both about his caveat and also I am happy to contribute

Cool - I have your emails so I'll drop you a Doodle poll with possible dates and we'll do a chat over coffee or drinks in a pub in Jan?

sounds good to me

thanks for the good chat @pupeno @thomas @agile_geek . hope i was not too annoying 😉

All opinions in this community are welcome and valid! I never hear ppl not speaking from some perspective of authority hard earned in experience and s/w battle scars!

@benedek: me too 😉

I appreciate the conversation as well. some good stuff in here.