As usual, Dan North did a fantastic keynote today at Oredev. Amazon should really start paying his salary — I ended up buying several books while he was still talking. North’s central idea was that patterns for effective software delivery derive from embracing uncertainty and that the delivery landscape changes significantly once we understand that.

North started by repeating the key ideas from his Oredev 2007 keynote, “Fear leads to risk, risk leads to process, process leads to hate… and suffering and Gantt charts”. Many elements of delivery processes in organisations are there to address risks because people are afraid of uncertainty. “We are terrified of uncertainty - we would rather be wrong than uncertain,” said North. Many of this process elements are wasteful and counter-productive, because uncertainty is a fact of life and more structure in a process does not address that properly. As an example, North mentioned the pointlessness of rigour in requirements and long term planning, citing a recent research which states that requirements have a half-life of only a few months today (the time it takes for half of the written requirements to become obsolete).

Dealing with uncertainty is at the core of the ideas promoted by the agile manifesto, but according to North that was widely misunderstood and many teams focused on practices expecting them to eliminate uncertainty. The original XP book has a tag line Embrace Change, which suggests that people should prepare for change but teams tried to predict where the change will happen and plan for it. According to North, uncertainty isn’t just about expecting change, it’s assuming that some unexpected bad things will happen during the project and that we can’t even know about them when we start. We can’t plan for them — if we could they wouldn’t be unexpected. “Embrace change was wrongly named…,” said North, “your ability to survive is directly related to handling the unexpected. Embrace uncertainty; expect the unexpectable and anticipate ignorance.”

As a potential technique to deal with uncertainty, he suggested Deliberate Discovery: if we know that some unexpected bad things will happen, we can optimise the delivery process for discovery instead of optimise it for structure. “The premise is assuming this will happen, what would you do differently?” asked North, adding that “we can actively reduce ignorance - instead of saying ‘I want to do estimation’, get people in the room and do stuff that is likely to produce the learning.”

He called the strategy double-loop learning — not just aim to improve the process, but learn to improve how we improve the delivery process.

He also suggested rolling wave planning as a way to embrace uncertainty of scope: “Get started and get data.”

For embracing uncertainty of technology, he suggested not splitting the deliverables in spikes and features any more and imposing rigour in testing and delivery on features. He said: “For features we have certain rules. spikes are hacks to learn and promise to throw the code away. what if you didn’t choose? what if i’m always going to start code by sketching. When it stats taking shape, stabilise it, make it testable, introduce rigour.”

For embracing the uncertainty of effort, he suggested investigating Beyond Budgeting, summarising it as “Measure stuff, spend money as if it’s your own.”

For embracing uncertainty of structure, North suggested using Real Options. He said: “Structure is an illusion of control, you can’t predict the future, so you can relax a bit. Use options, which have value and expire, commit deliberately and never commit early unless you know why.”