Jeff Patton from Thoughtworks held a very interesting session at XpDay last month in London, focusing on a common misconception that causes “agile” projects to fall into the same trap that the waterfall ones typically do.

Incremental is not iterative

Using a very interesting combination of pop music and rock star images, Jeff Patton told a story of a failed agile project in his XpDay keynote “Embrace Uncertainty”. The project started off nicely, almost by the book, with customer involvement and stories split into iterations, based on what functionality is to be delivered in what release. After they got something delivered to play with, customers changed their minds (as they so often do) and new stories and features were introduced into the plan. After a few deliveries, the scope kept growing and growing instead of reducing. From the developer perspective everything worked as planned – customer was expanding the scope and developers are there to oblige, because that is the essence of agile practices. Spice Girl Mel B was used for the role of a developer writing user stories and losing all sight of the big picture (while “So tell me what you want, what you really really want” was playing in the background). For the customer, the thing simply did not work - iteration after iteration, they were not any closer to having the project done.

The problem was that the concept of iterating had been lost and iterations were replaced by increments – making the project fall into the same trap as waterfall projects do. Iterations were aimed to deliver blocks of functionality. Jeff demonstrated that visually using the Mona Lisa painting (powerpoint slides from the keynote have still not been released, so the pictures below are not the same ones as Jeff used, but they show the point):

Although both these plans aim for the same thing, the incremental plan does not really reduce the risk of delivering something unsuitable to the client. The big picture appears only at the very end. Because increments are done in detail, a lot of effort is wasted when a piece needs rework (and the initial releases are almost certain to fall into this category). Iterative development offers a chance to see the picture from the start, and guide the development towards the full picture in steps. Not carving stuff in stone from the start allows us to change them easier later on, and we know that we’ll need to do that. Jeff gave the following rule of thumb to check quickly if your plan is iterative or incremental: “it’s not iterating if you do it only once”.

Dealing with uncertainty

However, iterative planning is harder to sell. Stakeholders want the project delivered by a fixed date on a fixed budget, and an uncertain iterative plan does not help with that. The XpDay keynote also offered three strategies for dealing with that:

  • Follow the money: focus the project on business goals, not on features nor user stories. Prioritise goals before prioritising stories - that will make it easier to decide what features need to be implemented completely by when. I wrote about this subject last year (see The magic of goals), discussing how focusing on goals makes everybody happier.
  • Don't choose your solution too early: defer writing user stories till last possible moment. Focus on what users need to accomplish at that moment. This is intended to allow easier rework and reduce wasted effort. Lean software development has a similar principle "Defer Commitment", which calls to Maintain Options (Think of code as an experiment – make it change-tolerant) and Schedule Irreversible Decisions at the Last Responsible Moment (Learn as much as possible before making irreversible decisions). See Lean Software Development: An Agile Toolkit and Implementing Lean Software Development: From Concept to Cash.
  • Build up feature quality iteration by iteration: this helps to stay within budget, deliver all features in each iteration but not necessarily all on the same level of quality. This technique is especially effective with customers that cannot prioritise features and insist on having everything delivered. Jeff discussed an example of a bus - it's pointless to ask the client whether brakes or transmission are more important. But the good thing with software is that you don't have to deliver everything at top quality from the start. We can first build a bus with low quality transmission and high quality brakes, then improve transmission in the next iteration.