The waterfall trap for “agile” projects

Jeff Patton from Thoughtworks held a very interesting session at XpDay last month in London, focusing on a common misconception that causes “agile” projects to fall into the same trap that the waterfall ones typically do.

Incremental is not iterative

Using a very interesting combination of pop music and rock star images, Jeff Patton told a story of a failed agile project in his XpDay keynote “Embrace Uncertainty”. The project started off nicely, almost by the book, with customer involvement and stories split into iterations, based on what functionality is to be delivered in what release. After they got something delivered to play with, customers changed their minds (as they so often do) and new stories and features were introduced into the plan. After a few deliveries, the scope kept growing and growing instead of reducing. From the developer perspective everything worked as planned – customer was expanding the scope and developers are there to oblige, because that is the essence of agile practices. Spice Girl Mel B was used for the role of a developer writing user stories and losing all sight of the big picture (while “So tell me what you want, what you really really want” was playing in the background). For the customer, the thing simply did not work – iteration after iteration, they were not any closer to having the project done.

The problem was that the concept of iterating had been lost and iterations were replaced by increments – making the project fall into the same trap as waterfall projects do. Iterations were aimed to deliver blocks of functionality. Jeff demonstrated that visually using the Mona Lisa painting (powerpoint slides from the keynote have still not been released, so the pictures below are not the same ones as Jeff used, but they show the point):

Delivery 1 Delivery 2 Delivery 3
Incremental plan
Iterative plan

Although both these plans aim for the same thing, the incremental plan does not really reduce the risk of delivering something unsuitable to the client. The big picture appears only at the very end. Because increments are done in detail, a lot of effort is wasted when a piece needs rework (and the initial releases are almost certain to fall into this category). Iterative development offers a chance to see the picture from the start, and guide the development towards the full picture in steps. Not carving stuff in stone from the start allows us to change them easier later on, and we know that we’ll need to do that. Jeff gave the following rule of thumb to check quickly if your plan is iterative or incremental: “it’s not iterating if you do it only once”.

Dealing with uncertainty

However, iterative planning is harder to sell. Stakeholders want the project delivered by a fixed date on a fixed budget, and an uncertain iterative plan does not help with that. The XpDay keynote also offered three strategies for dealing with that:

  • Follow the money: focus the project on business goals, not on features nor user stories. Prioritise goals before prioritising stories – that will make it easier to decide what features need to be implemented completely by when. I wrote about this subject last year (see The magic of goals), discussing how focusing on goals makes everybody happier.
  • Don’t choose your solution too early: defer writing user stories till last possible moment. Focus on what users need to accomplish at that moment. This is intended to allow easier rework and reduce wasted effort. Lean software development has a similar principle “Defer Commitment”, which calls to Maintain Options (Think of code as an experiment – make it change-tolerant) and Schedule Irreversible Decisions at the Last Responsible Moment (Learn as much as possible before making irreversible decisions). See Lean Software Development: An Agile Toolkit and Implementing Lean Software Development: From Concept to Cash.
  • Build up feature quality iteration by iteration: this helps to stay within budget, deliver all features in each iteration but not necessarily all on the same level of quality. This technique is especially effective with customers that cannot prioritise features and insist on having everything delivered. Jeff discussed an example of a bus – it’s pointless to ask the client whether brakes or transmission are more important. But the good thing with software is that you don’t have to deliver everything at top quality from the start. We can first build a bus with low quality transmission and high quality brakes, then improve transmission in the next iteration.

Image credits: Malene M./SXC

I'm Gojko Adzic, author of Impact Mapping and Specification by Example. My latest book is Fifty Quick Ideas to Improve Your Tests. To learn about discounts on my books, conferences and workshops, sign up for Impact or follow me on Twitter. Join me at these conferences and workshops:

Specification by Example Workshops

How to get more value out of user stories

Impact Mapping

11 thoughts on “The waterfall trap for “agile” projects

  1. Pingback: Exploration Through Example » Blog Archive » Jeff Patton on a trap for Agile projects

  2. Pingback: Denis au fil du web » The waterfall trap for “agile” projects

  3. It seems that incremental provides information while iterative provides protection.

    As an aside, this

    “customer was expanding the scope and developers are there to oblige, because that is the essence of agile practices”

    is a fallacy and unfortunately a common one.

  4. Hi Jason,

    It seems that incremental provides information while iterative provides protection.

    In my experience, iterative provides information better and sooner than incremental.

  5. Don’t you think the reason they kept getting it wrong was because they weren’t gathering requirements? One thing that cannot be skipped in analysis is time. Sometimes people just have to let ideas sink in a bit before they synthesize them. So if you start coding too soon, especially where the project is a new concept to people, you can really wast a bunch of resources. Agile solves a lot of problems but it doesn’t solve helping people learn or synthesize ideas faster. Check out

  6. Pingback: Incremental Is Still Waterfall : Laran Evans

  7. Pingback: Internet Alchemy » links for 2007-12-06

  8. Pingback: From Zero to Agile · No waterfall trap

  9. Iterative may be harder to sell, if you can’t make your customers see that iterative development actually brings faster results and faster ROI. You find possible mistakes on early stages of the product development and eliminate the situation when you need to go back and remake the whole product from the very beginning. Agile and Scrum methods have a lot of advantages.

  10. Pingback: Increment on the long term, iterate day by day « Software Engineering Slave

Leave a Reply

Your email address will not be published. Required fields are marked *