Dan North gave a very inspiring talk at the Oresund Developer conference today, titled Our obsession with efficiency. His presentation focused on perceptions of efficiency and pitfalls of applying wrong efficiency models to software development. efficiency. Most efficiency is measured as budget/revenue, effort, time or activity and improvement efforts focus on these things, but according to North this is wrong.

“We get what we measure”, said he, adding that the system is going to game the metrics. “Does finding a lot of bugs really mean that you have good testers, or does it mean that you have crap code?”, asked North. He also mentioned several examples of how people game the system in software development, including commenting out failing tests to increase test pass rates and generating tests without assertions to increase test coverage. Improving the efficiency in these parts leads to inefficiencies on the global level. “When you focus on efficiency of a part, you lose sight of the big picture. […] Efficiency is the enemy of effectiveness.”, said North.

Likewise, things that seem very inefficient can produce great effect on the global scale. As some examples of that, North mentioned cross-functional teams, pair programming, parallel spikes and the theory of constraints. Using a less controversial topic, he said that experimentation and innovation are very inefficient if considered on their own because they lead to lots of work that will necessarily be thrown away, but the global results of that are very effective.

North suggested that teams shouldn’t care about effort, but about the effect. Functional points, velocity, man hours and similar are easy to game and can lead to local optimisations that hurt the big picture. It is crucial to measure the correct thing and North suggests measuring cycle time for end-to-end customer satisfaction or cost of the end-to-end process, including opportunity costs and factoring in time.

So how can we improve the effectiveness? North suggests these pointers:

  • Understand that you're working in a system, and that it is likely that there is a system above that system and a system above that one
  • Identify the goal of that system - sometimes it's just its own survival
  • Step outside to observe the effects - how well does its goal align with yours?
  • Think about what's stopping the system from giving you what you want, and improve that.