The devil of regression testing


"No sympathy for the devil; keep that in mind. Buy the ticket, take the ride...and if it occasionally gets a little heavier than what you had in mind, well...maybe chalk it off to forced conscious expansion"

 Hunter S. Thompson (Fear and Loathing in Las Vegas)

On our mailing list where we discuss all sorts of important development issues and organise pub trips, a mate of mine came up with an interesting problem to do with testing - It went a bit like this ..

“I've just been talking to someone who I has attempted agile without any QA or automated tests at all. Unsurprisingly they've seen an increase in regression issues and are now looking to retrofit done automated testing.”

As the resident tester it got me thinking. What is the cost of regression testing in an agile project ? - well the cost in my opinion is actually 0, zip, nadda, snake eyes !!

Right about now everyone who has faced the cost of regression testing is thinking I am smoking crack? or maybe hill billy drunk on the 4th of july? - I’m not and here is why.

Lets say you have a project you estimated at 30 points - whatever - you figure as you are going along you can do roughly 5 points a week including getting it manually QA’d in a test environment and showed with great fan fare to your product owner  - Happy days!! - you will be out of there cashing the check in 6 weeks... 

or will you.. 

What was your ACTUAL velocity??

Easy I hear you all say - 5 points!! 

The problem is, it wasn’t really - The 5 points a week was stuffed with tech debit, Now I’m not talking the stupid tech debit i’ve cited before where you just sweep all the hard stuff into the tech debit backlog to temporarily inflate your velocity (and we all know how thats going to end) I’m talking about serious tech debit that wont bite you for a month or so. The question is actually what did you DO for that 5 points? 

Write code that solved the problem?

Or write code that solved the problem with unit tests, integration tests, platform tests, automated UI tests that prove every build and deployment you haven’t broken anything??

In most cases its the former  - And here is where logically (though not financially I’m sorry to say) it doesn’t cost. What is actually happening is your velocity gets corrected. You weren’t doing 5 points, you were actually doing maybe 3 but incurring the tech debit of poorly tested code allowed the team to inflate their velocity to an unrealistic 5. The over all correction theoretically will be 4 weeks in this case - but anyone who has tried to go back and retrofit tests to poorly tested code will know its harder than writing them at the time. so in this case it will take longer. 

I guess if you can guarantee some other poor stooge will have to deal with it later you *could* give them crappily tested code to work with but once the smell has pervaded the building you wont be working back there any time soon and your rep wont be the greatest.


July 5 2011
blog comments powered by Disqus