jobs-discuss

Job hunting, interview process and anything related to the experience of a job writing the Clojure language.
2019-05-21T12:07:44.015500Z

Does anyone know of any studies that measure the correlation between generative testing and software quality?

2019-05-21T12:08:15.016200Z

The case for general code coverage as a metric seems to be very weak. But I wonder if it would be a different story for generative tests.

valtteri 2019-05-21T12:31:42.020700Z

IME problem with code coverage is that people mean different things when talking about it. Some people and tools think that “if I call each function in my tests at least once I have 100% test coverage” which is a quite different than “calling each function with all the possible parameter invariants”.

valtteri 2019-05-21T12:31:57.021100Z

Generative testing definitely helps you to go towards the latter

1👍
vemv 2019-05-21T12:36:36.025300Z

> IME problem with code coverage is that people mean different things when talking about it. Related: coverage should be split depending on the test suite type curl localhost can get you 65% coverage, which demonstrates the limited accuracy of aggregating test types under a single metric. Assuming one categorizes tests according to e.g. unit, integration, functional etc, then each category should generate its own coverage I don't recall a tool built with that in mind, but haven't used a lot of them tbh

2019-05-21T13:17:34.026300Z

Is there any data I can present to my higher-ups besides my own opinion to show the value of coverage (even different kinds of coverage)?

2019-05-21T13:18:10.027100Z

It’s hard to find empirical evidence that any testing is beneficial, though anecdotally I would swear by it.

valtteri 2019-05-21T13:24:19.029500Z

Testing is not always beneficial. Good and sensible testing usually is. I’m afraid it may be hard to find studies supporting that because what is ‘good’ and ‘sensible’ depends on the context.

lilactown 2019-05-21T13:29:30.033200Z

Hillelogram on twitter has some great threads on this

valtteri 2019-05-21T13:29:51.033500Z

And don’t get me wrong, I am a huge fan of testing. 🙂 It’s just.. complicated. I’ve written and I’ve seen other people write very bad tests that just add waste to the project. OTOH I’ve worked in a project where we developed and maintained a huge web-store with zero tests. It was one of the worst experiences I’ve had.

valtteri 2019-05-21T13:32:26.035700Z

I think you probably should try to take a step back and ‘sell’ quality as a value to your higher-ups. All though that’s really hard if it’s not baked in to corporations core values somehow… And that’s a smell. 🙂

2019-05-21T13:32:57.036300Z

Well, how do you measure quality?

2019-05-21T13:33:21.037100Z

I’m not convinced measurability is important, but then again I’m not a business person.

valtteri 2019-05-21T13:33:36.037500Z

How much time you spend fixing bugs instead of developing

alexlynham 2019-05-21T13:33:43.037700Z

quality is measured in incidents of defects

alexlynham 2019-05-21T13:33:51.038Z

production downtime

alexlynham 2019-05-21T13:33:56.038300Z

issues filed by users

alexlynham 2019-05-21T13:33:57.038500Z

etc

lilactown 2019-05-21T13:34:17.039100Z

quality is measured in how much work it takes to improve your system

lilactown 2019-05-21T13:34:21.039400Z

🙂

alexlynham 2019-05-21T13:34:23.039500Z

(there are other metrics ofc but these are eaaaaasy to track)

neupsh 2019-05-21T13:34:41.040Z

I add Maintainability in there as well.

lilactown 2019-05-21T13:35:18.040600Z

you can have a system that works 100% but can never be changed. I would consider it a bad quality system

valtteri 2019-05-21T13:35:30.040900Z

Unless… It doesn’t need to be changed 😉

1👍
lilactown 2019-05-21T13:36:22.042100Z

well, in my experience, the only thing I can count on is any system I build will need to change at some point

valtteri 2019-05-21T13:37:15.044300Z

That’s probably 90% of the cases yes.

lilactown 2019-05-21T13:37:17.044500Z

whether it needs to be moved from AWS to GCP, or needs to support a new feature, or bugs need to be fixed

orestis 2019-05-21T13:37:26.045200Z

I’m reading “Software Design X-Rays” which tries to analyze code history and show which parts of the system are the ones that have the technical debt, which is mostly the ones that change often by multiple people. It’s a very interesting read so far.

1👍1
jumar 2019-05-23T11:03:17.076300Z

Btw. you can try the CodeScene tool which is based on the ideas from the book for free (open source projects) here: https://codescene.io/ Or look at the showcase: https://codescene.io/showcase

orestis 2019-05-21T13:38:21.047900Z

OTOH, in my previous job I was doing software interactives for museums — write once, never change. That was an interesting valley — if it works and looks good (try to test that, on a tight budget), you could move on 🙂

lilactown 2019-05-21T13:38:40.048500Z

IMO all measurements are proxy metrics for how easy it is to change

1✔️
valtteri 2019-05-21T13:38:43.048700Z

I remember seeing a tool that claimed to do something similar with git. Analyzing history and seeing which files were always edited together and possibly tangled.

lilactown 2019-05-21T13:38:46.048800Z

that’s pretty interesting @orestis 🙂

orestis 2019-05-21T13:40:19.049800Z

git log --format=format: --name-only | egrep -v '^$' | sort | uniq -c | sort -r | head -15

3👍
orestis 2019-05-21T13:40:40.050400Z

that’s straight from the book — gives you a sorted list of the most frequently changing files

orestis 2019-05-21T13:40:49.050700Z

remove the head at the end to pull them all out 🙂

valtteri 2019-05-21T13:43:15.051300Z

👆good stuff

valtteri 2019-05-21T13:46:26.051400Z

Does the book have a spell for analyzing which files are always edited in same commit?

orestis 2019-05-21T14:38:45.052600Z

Perhaps! I’m only in the beginning...

orestis 2019-05-21T14:45:58.056200Z

Any data points on how long a new hire takes to get on-boarded fully in a new environment that might have new languages, frameworks, etc?

orestis 2019-05-21T14:46:46.057500Z

I’ve seen some comments that new hires can learn enough Clojure to get productive in 2 weeks, but that seems an extremely short period of time.

2019-05-21T14:49:00.057700Z

ah looks like the same author as https://pragprog.com/book/atcrime/your-code-as-a-crime-scene

orestis 2019-05-21T15:02:09.059200Z

Yeah the x-rays expands on that. I haven read the crime scene one!

2019-05-21T15:17:21.060200Z

This is great, thanks!

bja 2019-05-21T15:49:22.060800Z

I've seen 1-2 weeks be the norm (for java devs)

bja 2019-05-21T15:50:05.061500Z

That presumes you have an existing codebase and relatively stable dev practices (someway to get the codebase up and running with a handful of commands)

bja 2019-05-21T15:50:31.061900Z

it usually takes a month or two for idiomatic code to be the norm

bja 2019-05-21T15:50:52.062400Z

at least from my observations of about a dozen devs

bja 2019-05-21T15:52:46.063600Z

java + python/ruby experience tends to be the shortest path (outside of actual clojure or lisp experience) to productive

bja 2019-05-21T15:53:42.064600Z

I've only ever compared one experienced CL user, but that person had dabbled in clojure in the past

bja 2019-05-21T15:54:43.065600Z

I kinda suspect that dynamic language workflow experience is slightly more crucial than java experience, but that also is probably project-dependent.

bja 2019-05-21T15:55:10.065900Z

and everyone has hacked python, ruby, or javascript these days anyway

alexlynham 2019-05-21T16:27:53.066100Z

depends on the codebase

alexlynham 2019-05-21T16:28:28.066800Z

probably a few weeks to get some commits in, but then multiply some amount of familiarisation time for each drastically different part of the system that is encountered

seancorfield 2019-05-21T16:38:03.068100Z

I added Zulip Mirror Bot so the discussions here get archived to Zulip and become searchable. I was looking for that link to the long post about why companies can't always hire remote workers outside their country (or even outside their state). Does someone still have that link?

dpsutton 2019-05-21T16:39:12.068600Z

it was a twitter message from a hashicorp person

dpsutton 2019-05-21T16:41:22.068800Z

https://news.ycombinator.com/item?id=17022563

seancorfield 2019-05-21T17:07:34.069Z

Thank you @dpsutton

1
mccraigmccraig 2019-05-21T17:21:34.073100Z

that HN post chimes with our early experiences with attempts to employ remotely in other EU (we're in the UK) countries - we found we couldn't understand how much it was going to cost us to employ someone in another EU country, and that it was memorably rather more than we expected. as a small company, with limited resource to investigate, it is much lower risk to stay within familiar territories

seancorfield 2019-05-21T17:29:17.073600Z

@justalanm See just a few posts above ^