Every now and then some ingenious project manager thinks of a way to deliver faster by negotiating to skip testing. At first glance, this looks like a win-win deal: customers get software faster or cheaper, as long as they “understand” that there will be problems. Developers, on the other hand, get more time to focus on new features. In reality, that is just one big lie, and a very dangerous one.
No testing = not working
The problem, like a lot of similar issues, comes from basic misunderstandings. “No testing” simply means a completely different thing to customers than it does to developers. Customers accept that there will be some bugs to be flushed out later. But they expect the system to generally work. For developers, “no testing” has a literal meaning as in zero testing - a get out of jail free card, which allows us to ship something as soon as it compiles.
Without a doubt, this approach guarantees major problems in delivered software. Even with good unit test coverage, skipping final integration testing is a big gamble on whether the components will work together correctly. And the odds say that something will fail. I lost count of times when a system delivered without testing was not able to perform key business functions. In the eyes of developers, the problem might have been a misplaced bracket or five lines of untested code – easily fixed when spotted. Most of the code is there, so programmers think that the work was done. In the eyes of customers, such a system is completely useless. It does not matter that hundreds of thousands of lines of code were written if the damned thing breaks during a login.
Three quotes to remember
Programming and testing together is faster than just programming – Kent Beck, Extreme Programming Explained
Nobody ever remembers a late delivery, just a bad one – Bruce Robinson, How to get ahead in advertising
Build half a product, not a half-ass product – 37 Signals, Getting Real </div>
“No testing” deliveries are completely pointless and should be avoided at all cost. They do no good. No buts, no exceptions. Customers will find the software unusable. Because of all the complaints that are sure to come, developers will have to jump into a panic-mode bugfixing race straight after the delivery. And, technically, that is the testing part of the project that had to be done. There will have to be a bugfix release later, so the software was not even delivered earlier. The first deployment is just a waste of time, and there are no benefits from it. Skipping testing does not speed up deliveries, it makes them even later. Kent Beck put this nicely in XP Explained: “Programming and testing together is faster than just programming”.
Better late then crap
When pressed to complete his task faster, Richard E. Grant’s character in How to get ahead in advertising has a great reply: “Nobody ever remembers a late delivery, just a bad one”. Being late is bad, and causes a lot of stress - especially among project managers who start looking for loopholes. But when compared to delivering crap, delivering late is definitely the lesser of two evils. Agreeing to skip testing in order to speed up development is no better than closing our eyes and sitting on a time-bomb, waiting for it to explode.
When things go bad, and there is no chance to meet the deadline, the best thing to do is to split the delivery. So one part will be delivered on time, with less features. That gives customers something to play with, or even start working. The rest of the features can get delivered a bit later. In my experience, cutting the scope is much tougher to agree on than “No testing”, but a much better way forward. As the opening paragraph of chapter 5 of Getting Real by 37 Signals advises: “Build half a product, not a half-ass product”.
If you find yourself in a situation to have to speed up the delivery, my advice is to sit down with customers, explain the situation, and ask them to choose top priorities from the list of unfinished tasks. Then deliver what you can, but do it properly.