Lisa Crispin presented a keynote titled “Limbo lower now: An agile approach to defect management” today at the Agile Testing Days 2010 in Berlin today. She compared defect management to Limbo dancing where the bar is progressively lowered. “We want to lower the bar and reduce defects”, said Crispin.
The traditional view for defects is that teams want to track and keep a record of them to perform analysis such as defect rates and root cause analysis. Many of the traditional advantages of defect tracking systems do not apply to agile development, said Crispin. Metrics, traceability, knowledge base of past defects and prioritisation are handled better with other tools in agile teams, such as unit tests, continuous integration and task cards. This makes the use of defect tracking tools on agile projects a bit controversial.
In the lean and agile view on software development, defect queues are queues of rework, and waste (Poppendieck). The lean approach is to write a test to confirm a defect, fix it and the test will prevent it from re-occurring. Teams using these processes treat bugs as technical debt.
As some potential uses for traditional defect tracking systems in agile teams, Crispin mentioned coordinating distributed/large teams and giving customers access and visibility over system issues. “If you’re using post-its or cards on the wall, people in other location can’t see that”, said she.
Offering an alternative perspective to bugs, she quoted Antony Marcano saying that defect reports are missed features and misbehaviour of the system. Bugs can point to missing features, so a defect tracking system can help identify them, but defect tracking systems are not necessarily the best tools for the job.
Instead of logging bugs, she advised writing tests to confirm the bugs. This is a good opportunity for the team to practice writing good tests, which they can apply later for test-first development. (This is the idea behind Defect Driving Development approach to teaching TDD). “Tests that reproduce bugs can be used as a good communication tool between developers and testers”, said Crispin.
Crispin said that choosing the right way to track or solve defects should be a team problem, that each team should understand their real needs and choose a solution by team consensus. Crispin advised teams to experiment. Some good ideas to experiment with are:
- Try starting out without a defect tracking system
- Try to turn bugs into stories
- Set rules such as "no more than 10 bugs at once"
- Try to fix all bugs immediately
- Make bug cards visible
- Prevent bugs in the first place
Crispin’s team uses a defect tracking system for production bugs. For problems caught in development they just communicate the issues and not track them. “A problem caught in development is not a bug” said she, as the team is still working on it. When they find such a bug it is usually fixed straight away. If the team decides not to fix it immediately, it is put on a task card.
Not all bugs need to be tracked, though. Crispin gave an example of when her team spent an iteration fixing low priority bugs, and the business users were surprised by that – they preferred to have new functionality built in and did not care that much about low priority bugs. Tracking bugs that nobody cares about is waste in the process. On the other hand, she said that sometimes they push back and explain to customers that some lower priority bugs should be fixed because the team will be able to move faster later.
Related to metrics, Crispin said that her team values trends and goals more than actual metrics. She gave an example as of a goal: “no more than 6 bugs in production over the next six months”. They use the goals to measure how they are doing and decide whether to change the process.
