A lot of the confusion about software quality is that it means different things to different people, and the best definitions so far are zen-like, eg Weinberg’s ‘value to someone (who matters)’. They don’t describe things like technical code quality, which I intuitively know matters but doesn’t directly provide value.

Laking a holistic definition of quality, teams measure things like bug trends, code coverage etc. What gets measured gets optimised, so we end up overly optimising things beyond the point that it makes sense. Air and food are a necessity, but more air and food than we need do not really improve the quality of life. Similar to that, technical correctness and performance are necessary but going beyond a certain point gives us diminishing returns. As with any local optimisation, there is a potential that we can hurt the whole pipeline by working on the wrong thing. As I was explaining this comparison to a client, it hit me that it might be worth trying to build a parallel between Abraham Maslow’s hierarchy of needs and software quality.

The famous Maslow’s pyramid lists human needs as a stack from physiological – necessary for basic functions (such as food, water), safety (personal security, health, financial security), love and belonging (friendship, intimacy), esteem (competence, respect) to self-actualisation (fulfilling potential). The premise of the hierarchy of needs is that when a lower level need is lacking, we disregard higher level needs. For example, when a person doesn’t have enough food, intimacy and respect, food is the most pressing thing. Another premise is that satisfying needs on the lower levels of the pyramid bring diminishing returns after some point. Our quality of life improves by satisfying higher level needs more. Eating more food than I really need brings obesity. More airport security than needed becomes a hassle. The key idea of the pyramid model is that once the basics are satisfied, we should work towards satisfying higher level goals.

Maybe software quality isn’t as simple as a zen-like sentence, and maybe measuring bugs actually does provide some value. Maybe we should stop trying to make it simple and instead model quality on different levels. I tried modelling this with a client – and this is their particular case – but it might apply to others as well.

The lowest level of quality are physiological needs – if these things aren’t satisfied, software is completely useless. For this particular situation, we identified two things: that the software has to be deployable and it has to satisfy the minimum functionality so that “it works”. Activities such as TDD, functional testing (automated + manual), post-deployment testing and similar help us prove that this category of needs is satisfied. Measuring bug counts, code coverage and so on also works on this level. And similar to the human physiological needs, enough is enough and after some point there is no more value to be gained. Newer versions of Microsoft Office have thousands of features that nobody ever uses, because even Word 95 had enough. Investing in more features, developing, testing, maintaining them, is an overkill.

Once the software “works”, the second thing we need from it is to “work well”. Looking at parallels between human security needs and what we need from software, this is where a lot of what people typically call non-functionals goes in. Performance, reliability, security etc. Activities such as architectural design and performance optimisations come into this level, and performance testing, penetration analysis, stress tests etc might prove that we have satisfied enough. And similar to human security needs, enough is enough. Recent examples of Apple Itunes store security questions demonstrate that more features than needed in this space piss people off, or waste money. Building a system that can handle millions of concurrent users when in the next year or so we’ll only have thousands is a horrible waste of time. I lost a lot of my own money on this a few years ago – we goldplated a system architecturally instead of shipping more stuff that makes money.

Providing the software performs well and is secure enough, the next level above is love and belonging – and now we cross over to the users of the software. Case in point is Twitter. Famous for its fail whale, twitter’s second-level qualities are often just good enough, but that hasn’t stopped them from building a huge community of loyal users. Activities such as user interaction design, graphical design, community engagement and similar support software in fulfilling the needs on the level of love/belonging. Usability testing proves it. Of course, different types of software need different levels of this. In-house admin software requires a lot less love because people are forced to use it. On the other hand, user devotion and interaction can make or break consumer products.

So far this is nothing revolutionary. If I look back at most of my client engagements, these three levels are where most of the investment in specifications, development and testing was, and roughly proportional to the levels as well. Most of the investment goes in building in functionality and testing it. Performance and things like that are sometimes planned for, often reactively built in, and tested less frequently. I rarely worked with teams that were serious about usability and invested a lot of time or money designing for it and testing it. The pyramid model told this concrete client that I worked with that maybe they should start shifting their investments a bit. But the real surprise came out when we started looking at the top two levels.

Usability on its own can be an overkill. No doubt about it. In Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation, Tim Brown presents Nokia Ovi as a key success story to showcase design thiking.

Barely a year later, Nokia announced Ovi, a new service offering that
could be accessed through any of its multimedia devices. Design
thinking had enabled Nokia not only to explore new possibilities but
also to convince itself that these possibilities were sufficiently
compelling to move away from its strongly entrenched and previously
successful approach. The timing was right. Today Ovi is one of the
operating business divisions of the company, and Nokia – a technology
leader – has reinvented itself as a service provider.

Oh, the irony. If ever there was a IT equivalent of harakiri, this was it. Nokia – a technology
leader – has “reinvented itself as a service provider” but nobody wants their service. Don’t get me wrong, I am not saying that user interaction design is bad, or that design thinking is wrong – just that it’s not the end. It is a need to be satisfied, but brings diminishing returns after a point. There are two more levels on the pyramid above it.

The key thing missing from the Nokia Ovi story is the fulfilment of the potential. Usability marks potential, and if nobody is using it do we care? The level above usability is usefulness. I did a study with a client recently and looking at the log files we determined that roughly 60% of the features aren’t used enough to justify investment in further maintenance. Maybe instead of investing a lot of money in functional testing we can invest in measuring the usefulness of software? One team I interviewed for
Specification by Example: How Successful Teams Deliver the Right Software use automated tests as a target for development, but then disable most tests after they pass the first time. Instead, they protect against problems by delivering in small batches and measuring production use. They define quality more at the fourth level than the first level, which enables them to be much more productive. If the indicators expected by the business users aren’t there, the feature is taken out. Think of this as a business-bug.

Finally, the fact that someone uses a feature doesn’t necessarily mean that the feature was right for the business in the first place. There is one more level above – corresponding to self-actualisation. Does the software achieve what it was originally intended to? Does it save money, earn money or protect money? Or whatever the key business goals were originally. If not, then it doesn’t really matter that people use it, that it is usable or performant or that all unit tests pass. This is the space where activities such as Effect Mapping and Feature Injection come in, along with the ideas of Build-Measure-Learn cycles and Actionable Metrics from The Lean Startup to prove them.

The top level is really where “more is better”, with perhaps a gradual transition to “good enough is good enough” on the two levels below, with the lowest two levels definitely falling into the “good enough” category. Yet from what I see most software teams invest, build and test only at the lowest two levels, gold-plating things without a way to explain why that is bad. Breaking things down in a visual model such as the one with the five levels of the pyramid here helped me get one team to start thinking better about what they really want to achieve. Your turn next!

• Forrester research on water-scrum-fall
• Article summarising the research
• Specification by example
• Effect mapping
• Feature injection
• User story mapping

I’ve evolved a new technique for splitting user stories over the last few months shamelessly stealing repurposing Jeff Patton’s User Story Mapping and ideas described by Craig Larman and Bas Vodde in Practices for Scaling Lean & Agile Development. I think it works particularly well in situations where a team cannot find a good way to break things down and is insisting on technical divisions. It has the visual playful aspect similar to innovation games and it’s easy to remember. I call it the User Story Hamburger.

Inexperienced teams often can’t get their heads around splitting stories into smaller stories that still deliver business value. But they will happily break a story down into technical workflow or component tasks. I like the idea of User Story Maps which show the big picture under a breakdown of a business workflow. We can do the same on a much lower level, for tasks making up a user story, keeping the team in their comfort zone. Then we use this breakdown to identify different levels of quality for each step, and create vertical slices to identify smaller deliverables. Here is how to create the hamburger:

Step 1: Identify tasks

As a group, identify technical steps that would be involved in implementing a story on a high level. Breaking it down into technical/component workflow is OK. The tasks become layers in a hamburger bun – meat, lettuce, tomato, cheese – throw in some bacon for fun as well.

For example, if we’re working on a story to contact dormant customers by e-mail, the steps might be: query db for dormant customers; send e-mail to customers; retrieve delivery report; remove bounced addresses from the system; mark non-bounced as ‘recently contacted’.

Step 2: Identify options for tasks

Split the team into several small groups and ask them to define quality for each task, or what would make a particular task good. Then they should write down several options on different levels of quality on post-it notes.

For example, speed of execution and accuracy of results might be a measure of quality for database query for dormant customers. Two possible options could be to make a slow and relatively inaccurate query on all customers compared with overnight transaction reports, which won’t pick up intraday changes. Another option to increase accuracy would be to have a nightly job making customers dormant and remove the dormant mark on every transaction, which would enable us to be 100% accurate and faster. The first option would work only if we send e-mails once a month, the second would work at any time.

Another example might be the volume we can send, the content personalisation and spam-regulation compliance for sending e-mail. We have an option of sending things manually, slow and low-volume. Second option is to use an automated process and send generic e-mails, with a manual unsubscribe process. A third option would be to use an automated process and send personalised e-mails, with manual unsubscribing. The fourth option would be to send personalised e-mail automatically and enable people to unsubscribe automatically. Yet another option would be integrating with a third party service that does all that.

Step 3: Combine results

Create a single hamburger on a big board. Ask representatives from each team to bring post-it notes and fill in the layers of your hamburger, briefly introducing what each note it. Identify duplicates and throw them away. Align task options from left to right based on the level of quality.

Step 4: Trim the hamburger

As a group, go through tasks and compare the lowest quality options with things next to them, based on how difficult it would roughly be to implement each option. Mark that information on the post-its. It might be worth breaking things down into relatively same-size technical tasks to do some simple comparisons. Think about throwing away lower quality items that would take more or less the same to implement as a higher quality option.

Also decide what is the maximum needed level of quality for each task. For example, intra-day bounced e-mail removal won’t really bring much more value than overnight bounced e-mail removal. Take items over the line out – that’s what will be left in the box after you eat the hamburger.

Step 5: Take the first bite

Now that you have a hamburger, decide how deep you’ll take the first bite. Discuss what is the minimum acceptable level of quality for each step. For example, manual sending might not be acceptable at all, because of the low volume. But sending e-mail once a month might be acceptable. If the lowest quality option is more-less the same size as the higher quality one, you might go deeper straight away. For example, sending generic e-mail with manual unsubscribing might be more or less the same effort as integrating with Mailchimp. On the other hand, a fast realtime update of customer activity might be much more difficult than a on-demand slow database query. For some steps, eg removing bounced addresses, doing nothing might be a valid level of quality initially.

Step 6: Take another bite… and continue

From there on, any further bite should provide more quality, regardless of what you add. Eg implementing initial bounced e-mail removal enhances the quality. So does more frequent e-mail sending. Use complexity comparisons between tasks to decide how deep you want to take further bites.

This method works very nicely because it is visual, and it gets people thinking about alternatives while still staying in their comfort zone. It also works nicely with ‘bite-size chunks’ analogies. And you can easily explain why releasing just a technical task doesn’t make sense because no sane person would eat only the lettuce. On a final note, make sure not to do this while people are hungry.

If this sounds interesting, you can practice many other collaborative techniques for delivering the right software iteratively at my upcoming two day workshop in London in mid-March.

My two-day hands-on Specification by Example workshop is finally coming back to London. Since 2007, more than 2000 people have benefited from this training. The workshop is aimed at testers, business analysts and developers and based on my books Specification by Example and Bridging the Communication Gap. Through facilitated exercises and discussion, you will learn:

• how to extend specifications with examples to create a single source of truth for testing and development
• how to avoid functional gaps and inconsistencies in specifications and tests
• how to run specification workshops to facilitate collaboration
• good practices for designing specifications with examples and acceptance tests for agile teams
• how to create a living documentation system to facilitate change and improve your process long-term
• how other teams, from small web startups to large distributed teams in investment banks, apply specification by example in their contexts

For more information and to sign up, see the event page on EventBrite.