Poppendieck said that structured programming and top-down programming tried to divide software in layers around hierarchy and execution in late sixties (see Edsger Dijkstra’s work on structured programming). In the early seventies, David Parnas promoted decomposition along the lines of responsibility and likeliness of future change. In mid-seventies, Vinton Cerf and Robert Kahn designed TCP/IP by introducing a clear separation of concerns and commonality/variability analysis. In the second half of seventies software projects were being divided by process (Barry Boehm), leading to watefall development. Early eighties brought evolutionary development (Tom Gilb), building up value incrementally. (I was a bit surprised by the nineties completely missing from the story.)

Taking the best ideas from all these divisions, she said that divisions by responsibility and business value are the best.

Poppendieck compared waterfall to a “defect injection process”, saying that if specifications written first, then test and code based on those specifications by different people, the chance of them matching is very small, and there is nobody really to blame for that. She proposed that specifications should be written in the form of tests straight away, and then the code written according to these tests, to eliminate one potential cause of mismatch (specifications-tests) and ensure that the other (specifications-code) is easy to verify. This effectively is the idea behing agile acceptance testing.

see other articles from Agile Testing Days

Here is a situation I have seen a few times more than I would have wanted: the team and the clients split the whole project into several phases that will be shipped every few weeks; early deliverables mostly provide the plumbing for the later ones, and have only enough user-interfacing functions for the plumbing to be tested functionally. The clients set up and play with each new release. They dutifully give their feedback and participate in testing each time. When the whole thing comes together on the end, it turns out that the system still requires a lot of changes to solve the problem that it was intended for — the iterations and customer feedback failed to provide the directions.

The problem hides in way that the project was split: early deliverables mostly provide the plumbing for the later ones and the whole thing comes together on the end. So, until very close to the end of the project, the deliverables could not really be used in production. Unless the software is really being used, we cannot expect authoritative feedback. The customers may be able to give us their general gut-feel and comment on functional correctness, but usability and feasibility can only be measured once the software is really actually used by people in the course of their daily business. Technical integration issues as well as usability (human-integration) issues are expected and typically not problematic to solve, but we must not let them pile up. Early feedback should allow us to solve some of those problems during the whole project, instead of finding them out on the end.

One more reason why deliverables should be used in production is that the customer’s business will rarely stand still during the course of the project, making the end result effectively a moving target. If the deliveries are not really used until the end of the project, then we have no idea whether they are in touch with the present business requirements. On the other hand, if the deliverables are used every day, we will know that straight away – and most likely the clients will ask for adjustments early.

So how do we avoid this problem?

Here are three ideas to avoid the problem:

• Focus on user stories, not on technical use cases
• Plan deliveries so that they are complete in the sense that they can really be used in production
• Divide and conquer: Split a big project into several mini-projects, and then focus on delivering those in a sequence

Focus on user stories

User stories have emerged over the last several years as the preferred way of planning scope for agile projects. Focusing on user stories is a great way to split the project into parts that actually bring value to the customer, since every user story should have a clearly defined customer benefit. So planning deliverables in terms of user stories instead of technical use cases is a great first step.

A pitfall that we must avoid is to then batch stories based purely on technical dependencies. Although it makes perfect sense to have early deliverables provide the technical infrastructure for latter ones, and that type of dependency must be taken into consideration when planning deliverables, ordering purely based on that leads to the exact problem that we are trying to avoid. This is effectively incremental, not iterative development.

Have production-worthy deliveries

Instead of focusing on technical order, it is better to try plan deliveries so that every shipped version of the software can really be used in production. Although it is not always possible to adhere perfectly to this idea, it is a good general rule and we should try to keep the deliveries complete in that sense. While working on the delivery plan, just ask the clients this simple question: “pretend it’s magic and in a few months we give you these stories — would that piece of software be complete in the sense that it can actually be used in production?”. If the answer is no, go back to the drawing board.

Doing production-worthy deliveries several times during the project might require writing some throw-away code, but that is a small price to pay for early feedback (to programmers) and getting business benefits early (to customers). For example, several years ago I worked on a trading system that was supposed to replace excel-based trading and distribution management. One of the major problems that we were supposed to solve was optimising transport routes and providing transport plans to local centres. So our first delivery was a piece of software that allowed traders to input their trading plans (from the excel sheets), produced an optimised transport plan, and allowed them to export it back to excel. The first and third part of that delivery were thrown away later, but we got good feedback on a fairly complex part of the system and solved tons of edge cases that were not identified during initial planning. Traders were happy because this saved them a few hours of work every day, and they got that several months before the whole system was in place.

Break a big project into mini-projects

Reg Braithwaite wrote that his Golden Hammer for building software iteratively is to divide it into separate products. This makes excellent sense for building larger systems where the software will perform a lot of functions for a lot of people on the end. By dividing one single large product into smaller products we force ourselves to have cleaner interfaces and looser coupling between software components. This can allow us to develop them independently, and will also reduce the risk of the whole thing missing the target on the end. Each of those mini-products would be a deliverable, and it should be usable on its own.

I recently worked on a highly scalable e-commerce web site. We split the project into three separate products: an e-commerce integration platform, high-performance caching system and the actual of web site. The integration platform was delivered first, with a low-fi card processing web that was used for some 3rd party integration. It went live a year before the rest of the system, helping us flush out any problems with transaction processing. We had new requirements about this part of the system every few months, driven by the fact that people were actually using it. So when the project was close to the end, this integration platform was in touch with reality. The caching system went live next, and it was used to support a mobile content distribution application. Both the low-fi card processing site and the mobile app were not part of the initial plan, but we were able to slot them in nicely. The web site front end then built upon the foundations of those two systems that were in production for months. Thinking about the whole project in terms of separate products both helped us deliver it quicker (because we had real production feedback for parts of the system early on) and also increased the value of what we were doing (since the same components were used to support different applications and smaller projects that started after that big one). Divide and conquer approach paid off big.

Deliveries and iterations

I know that some of you, reading this, will now start to complain how iterations should be two to four weeks long and that the business problems are often larger than what can be solved in such a short period of time. I intentionally avoided using the word “iteration” in this discussion, because I do not think that every iteration needs to be delivered.

If an iteration result is not complete in the sense that it can really be used every day, then the iteration delivery will just sleep on the UAT servers after testers play with it for a while. To get quick feedback on such iterations, I propose giving clients access to the local integration environment and letting them play with the software there. You can avoid packaging and polishing and they do not have to update their local installations.

If an iteration produces software that can really be used every day, then ship it and make the clients use it. It is definitely useful to get feedback early, but having something go live, even if it causes rework or throwing away parts of software later, is much much more valuable than software that is only tested. It has more value for the customer, since they are getting a part of their problem solved earlier. It also has more value for the development of the whole project, since it brings real, authoritative feedback from everyday use.

Image credits: Emin Ozkan and Sachin Ghodke/SXC

Incremental is not iterative

Using a very interesting combination of pop music and rock star images, Jeff Patton told a story of a failed agile project in his XpDay keynote “Embrace Uncertainty”. The project started off nicely, almost by the book, with customer involvement and stories split into iterations, based on what functionality is to be delivered in what release. After they got something delivered to play with, customers changed their minds (as they so often do) and new stories and features were introduced into the plan. After a few deliveries, the scope kept growing and growing instead of reducing. From the developer perspective everything worked as planned – customer was expanding the scope and developers are there to oblige, because that is the essence of agile practices. Spice Girl Mel B was used for the role of a developer writing user stories and losing all sight of the big picture (while “So tell me what you want, what you really really want” was playing in the background). For the customer, the thing simply did not work – iteration after iteration, they were not any closer to having the project done.

The problem was that the concept of iterating had been lost and iterations were replaced by increments – making the project fall into the same trap as waterfall projects do. Iterations were aimed to deliver blocks of functionality. Jeff demonstrated that visually using the Mona Lisa painting (powerpoint slides from the keynote have still not been released, so the pictures below are not the same ones as Jeff used, but they show the point):

	Delivery 1	Delivery 2	Delivery 3
Incremental plan
Iterative plan

Although both these plans aim for the same thing, the incremental plan does not really reduce the risk of delivering something unsuitable to the client. The big picture appears only at the very end. Because increments are done in detail, a lot of effort is wasted when a piece needs rework (and the initial releases are almost certain to fall into this category). Iterative development offers a chance to see the picture from the start, and guide the development towards the full picture in steps. Not carving stuff in stone from the start allows us to change them easier later on, and we know that we’ll need to do that. Jeff gave the following rule of thumb to check quickly if your plan is iterative or incremental: “it’s not iterating if you do it only once”.

Dealing with uncertainty

However, iterative planning is harder to sell. Stakeholders want the project delivered by a fixed date on a fixed budget, and an uncertain iterative plan does not help with that. The XpDay keynote also offered three strategies for dealing with that:

• Follow the money: focus the project on business goals, not on features nor user stories. Prioritise goals before prioritising stories – that will make it easier to decide what features need to be implemented completely by when. I wrote about this subject last year (see The magic of goals), discussing how focusing on goals makes everybody happier.
• Don’t choose your solution too early: defer writing user stories till last possible moment. Focus on what users need to accomplish at that moment. This is intended to allow easier rework and reduce wasted effort. Lean software development has a similar principle “Defer Commitment”, which calls to Maintain Options (Think of code as an experiment – make it change-tolerant) and Schedule Irreversible Decisions at the Last Responsible Moment (Learn as much as possible before making irreversible decisions). See Lean Software Development: An Agile Toolkit and Implementing Lean Software Development: From Concept to Cash.
• Build up feature quality iteration by iteration: this helps to stay within budget, deliver all features in each iteration but not necessarily all on the same level of quality. This technique is especially effective with customers that cannot prioritise features and insist on having everything delivered. Jeff discussed an example of a bus – it’s pointless to ask the client whether brakes or transmission are more important. But the good thing with software is that you don’t have to deliver everything at top quality from the start. We can first build a bus with low quality transmission and high quality brakes, then improve transmission in the next iteration.

Image credits: Malene M./SXC

No testing = not working

The problem, like a lot of similar issues, comes from basic misunderstandings. “No testing” simply means a completely different thing to customers than it does to developers. Customers accept that there will be some bugs to be flushed out later. But they expect the system to generally work. For developers, “no testing” has a literal meaning as in zero testing – a get out of jail free card, which allows us to ship something as soon as it compiles.

Without a doubt, this approach guarantees major problems in delivered software. Even with good unit test coverage, skipping final integration testing is a big gamble on whether the components will work together correctly. And the odds say that something will fail. I lost count of times when a system delivered without testing was not able to perform key business functions. In the eyes of developers, the problem might have been a misplaced bracket or five lines of untested code – easily fixed when spotted. Most of the code is there, so programmers think that the work was done. In the eyes of customers, such a system is completely useless. It does not matter that hundreds of thousands of lines of code were written if the damned thing breaks during a login.

“No testing” deliveries are completely pointless and should be avoided at all cost. They do no good. No buts, no exceptions. Customers will find the software unusable. Because of all the complaints that are sure to come, developers will have to jump into a panic-mode bugfixing race straight after the delivery. And, technically, that is the testing part of the project that had to be done. There will have to be a bugfix release later, so the software was not even delivered earlier. The first deployment is just a waste of time, and there are no benefits from it. Skipping testing does not speed up deliveries, it makes them even later. Kent Beck put this nicely in XP Explained: “Programming and testing together is faster than just programming“.

Better late then crap

When pressed to complete his task faster, Richard E. Grant’s character in How to get ahead in advertising has a great reply: “Nobody ever remembers a late delivery, just a bad one”. Being late is bad, and causes a lot of stress – especially among project managers who start looking for loopholes. But when compared to delivering crap, delivering late is definitely the lesser of two evils. Agreeing to skip testing in order to speed up development is no better than closing our eyes and sitting on a time-bomb, waiting for it to explode.

When things go bad, and there is no chance to meet the deadline, the best thing to do is to split the delivery. So one part will be delivered on time, with less features. That gives customers something to play with, or even start working. The rest of the features can get delivered a bit later. In my experience, cutting the scope is much tougher to agree on than “No testing”, but a much better way forward. As the opening paragraph of chapter 5 of Getting Real by 37 Signals advises: “Build half a product, not a half-ass product“.

If you find yourself in a situation to have to speed up the delivery, my advice is to sit down with customers, explain the situation, and ask them to choose top priorities from the list of unfinished tasks. Then deliver what you can, but do it properly.

Image credits: Peter Huys/SXC.

Whatever the reason, from my experience the developers (and the development managers) are the ones who have to stop the vicious cycle – and start delivering better. We have to acknowledge that ‘just get it done as soon as possible, and we’ll test it on production‘ does not mean the same thing for us and for the customers. For most customers, that means that they can live with a few minor problems, but for developers this is typically a green light to shoot fireworks all around. Someone has to take the responsibility for closing this gap, and I am afraid it is not going to be the customers.

And ‘I told you so‘, when the customer comes back unsatisfied with the delivery is not really an option – not if you want to keep them. Even if they are reasonable more than the average, and you win that argument, you will be losing much more – as your reputation will suffer. Somewhere in the back of their minds, customers will feel cheated, even if they do not say that out loud, and you just lost a credit point.

Sprinting is futile if you crash

Sprinting to add raw, unpolished functionality almost never pays of, and it can blow up in your face. I learned this the hard way three years ago, while working on an on-line casino project². We developed much faster than we initially estimated, and the software was ready for delivery before the layout design was finished. We decided to deliver the software early, hoping to make a good impression on the clients, but the exact opposite happened – clients hated the mock-up layouts so much that they wanted to cancel the entire project. It took a lot of effort, mostly painful politics, to convince them that the project should continue.

As a different example, when I worked on an ERP system, we underestimated the effort required for the first iteration. There were two options: we could either deliver all the planned functions, but the software would probably be unstable and quite buggy; or, we could deliver about half of the planned functions but polish them. The clients wanted the system up and running as soon as possible (as always), and they pressed for functionality, asking for earlier delivery even if it throws fireworks. However, knowing that it was really important to make a good first impression on a new client, we decided not to pursue the functionality, but to cut it in half and do it properly. The result was quite good – they were happy with the first cut, not so happy about the prolonged schedule, but the way the system worked and looked like gave them confidence that we knew what we were doing. Had we delivered more functionality, like they asked, I’m pretty sure that we would have lost the contract soon after the first delivery, or at least have to go through a political hell of explaining who requested what, who was warned and who acknowledged warnings. On the end, there is only one first impression – if you miss that one, it’s a lot harder to establish trust and confidence afterwards.

Why bother, after all?

Hmm… How to start explaining the benefits of happier customers? More work, better relationships, more money… And sometimes you might really get away with murder. Mistakes do happen, and in a long term engagement some big problem is bound to come up. That is the moment when you really need the customer to think ‘well, these guys were pretty darn good so far, and what they suggest seems reasonable, so let’s give it a chance‘ and not ‘those stinking %^$&#* have gone too far, and now they want to shut down the production for 10 hours. memo to self – call the others and let’s start planning a migration on Monday‘. To have a lucky break, and customers’ support, you need to have credibility – yet delivering unpolished software again and again slowly slices pieces of credibility from the reputation of development organisations. Once lost, the credibility is hard to recover. Save it for the rainy days, when you will really need that ‘get out of jail free‘ card – don’t throw it away with unpolished releases.

It’s all in prioritisation

If you deliver regularly, and often, most customers will not mind too much if the project actually takes longer then initially estimated. Even if they do mind, it’s the lesser of two evils. If there is not enough time (or from a different, less fatalistic, perspective – there is too much work) it’s mostly better to deliver less, but better polished, than more, no matter what the customer will tell you in his Ursa Minor Beta language. (‘Mostly’ means that there are exceptions, of course – and again developers and development managers are the ones that have to recognise them). Pressing for non-realistic schedules does no good, and it can actually turn the project into a death march³. Agile development techniques can help in this situation, as the client should be involved in planning. Even if you are not using agile methods for development, talk to the customers and get their priorities – most of the time they will be happy to get high-priority features now and get the rest a bit later, if it all works good and their goals are met⁴. I’ll borrow a quote from Getting Real⁵: It’s better to make half a product than a half-assed product.

Finishing touches

No fireworks on production:
The software must not scare users by crashing, or printing stack traces and raw SQL errors – it’s much better to deliver less functionality but properly polished than more functionality that blows up. From my experience, the first option tells the customer that the project might be more complex than initially expected, but the second gives them the impression that developers are incompetent since the software crashes.

No mock-up layouts:
Avoid showing mock-up layouts, especially if the layout is an important part of the customer experience. On the end, users only see the user interface, so it better be good.

Check positive branch manually:
Automated testing is great, but sometimes it can blind you. Make sure that the basic functionality of the system (at least the positive branch) works and looks OK from the GUI – check it by hand and with your own eyes.

No surprises:
User interface should behave, as much as possible, like standard applications. There are, of course, exceptions when an innovative user interface is the main selling point of the product – but most of the time a standard interface will do much better.

Automate installation and upgrades:
Build an automated installer, which will make sure that everything is done properly, and not require clients to manually copy and edit files. My favourite bad example is the web upgrade where the developer just sent some files along with a note ‘Please remember to configure all the parameters like we talked about over the last few weeks‘. The automated installer will make the client happier, and reduce support problems. It does not matter if your clients are technical enough to install everything by hand – they will be happier to just run a script. Creating a simple installation script will take you 30 minutes, and your clients will surely appreciate that. Nullsoft Installer⁶ is a great free tool for building automated installers, if you do not have the budget for commercial tools.

Provide documentation:
Release notes, a simple HTML-based help system and supporting documentation add a more serious note to the software, and should be prepared for anything except the most trivial projects. Even if everything is plainly clear and the system is intuitive (which it should be), these documents will make your team look more professional. It’s the same difference as the one between sending a letter printed on a plain paper or on a letter-headed one.

Footnotes

1. Ken Schwaber explains various meanings of a ‘project being done‘ in the presentation A Canary in a Coal Mine
2. My post-project conclusions about the on-line casino are described in the post Joys and Perils of Mass-Market Java Games
3. see Edward Yourdon’s excellent book on Death March projects
4. see The magic of Goals, Focused Projects and Better Requirements
5. see Fix Time and Budget, Flex Scope from Getting Real
6. see NSIS Project on Sourceforge

Image credits: Wax115/SXC, Kavitha/SXC, Nasa/GIMP Savvy