I’ve evolved a new technique for splitting user stories over the last few months shamelessly stealing repurposing Jeff Patton’s User Story Mapping and ideas described by Craig Larman and Bas Vodde in Practices for Scaling Lean & Agile Development. I think it works particularly well in situations where a team cannot find a good way to break things down and is insisting on technical divisions. It has the visual playful aspect similar to innovation games and it’s easy to remember. I call it the User Story Hamburger.

Inexperienced teams often can’t get their heads around splitting stories into smaller stories that still deliver business value. But they will happily break a story down into technical workflow or component tasks. I like the idea of User Story Maps which show the big picture under a breakdown of a business workflow. We can do the same on a much lower level, for tasks making up a user story, keeping the team in their comfort zone. Then we use this breakdown to identify different levels of quality for each step, and create vertical slices to identify smaller deliverables. Here is how to create the hamburger:

Step 1: Identify tasks

As a group, identify technical steps that would be involved in implementing a story on a high level. Breaking it down into technical/component workflow is OK. The tasks become layers in a hamburger bun – meat, lettuce, tomato, cheese – throw in some bacon for fun as well.

For example, if we’re working on a story to contact dormant customers by e-mail, the steps might be: query db for dormant customers; send e-mail to customers; retrieve delivery report; remove bounced addresses from the system; mark non-bounced as ‘recently contacted’.

Step 2: Identify options for tasks

Split the team into several small groups and ask them to define quality for each task, or what would make a particular task good. Then they should write down several options on different levels of quality on post-it notes.

For example, speed of execution and accuracy of results might be a measure of quality for database query for dormant customers. Two possible options could be to make a slow and relatively inaccurate query on all customers compared with overnight transaction reports, which won’t pick up intraday changes. Another option to increase accuracy would be to have a nightly job making customers dormant and remove the dormant mark on every transaction, which would enable us to be 100% accurate and faster. The first option would work only if we send e-mails once a month, the second would work at any time.

Another example might be the volume we can send, the content personalisation and spam-regulation compliance for sending e-mail. We have an option of sending things manually, slow and low-volume. Second option is to use an automated process and send generic e-mails, with a manual unsubscribe process. A third option would be to use an automated process and send personalised e-mails, with manual unsubscribing. The fourth option would be to send personalised e-mail automatically and enable people to unsubscribe automatically. Yet another option would be integrating with a third party service that does all that.

Step 3: Combine results

Create a single hamburger on a big board. Ask representatives from each team to bring post-it notes and fill in the layers of your hamburger, briefly introducing what each note it. Identify duplicates and throw them away. Align task options from left to right based on the level of quality.

Step 4: Trim the hamburger

As a group, go through tasks and compare the lowest quality options with things next to them, based on how difficult it would roughly be to implement each option. Mark that information on the post-its. It might be worth breaking things down into relatively same-size technical tasks to do some simple comparisons. Think about throwing away lower quality items that would take more or less the same to implement as a higher quality option.

Also decide what is the maximum needed level of quality for each task. For example, intra-day bounced e-mail removal won’t really bring much more value than overnight bounced e-mail removal. Take items over the line out – that’s what will be left in the box after you eat the hamburger.

Step 5: Take the first bite

Now that you have a hamburger, decide how deep you’ll take the first bite. Discuss what is the minimum acceptable level of quality for each step. For example, manual sending might not be acceptable at all, because of the low volume. But sending e-mail once a month might be acceptable. If the lowest quality option is more-less the same size as the higher quality one, you might go deeper straight away. For example, sending generic e-mail with manual unsubscribing might be more or less the same effort as integrating with Mailchimp. On the other hand, a fast realtime update of customer activity might be much more difficult than a on-demand slow database query. For some steps, eg removing bounced addresses, doing nothing might be a valid level of quality initially.

Step 6: Take another bite… and continue

From there on, any further bite should provide more quality, regardless of what you add. Eg implementing initial bounced e-mail removal enhances the quality. So does more frequent e-mail sending. Use complexity comparisons between tasks to decide how deep you want to take further bites.

This method works very nicely because it is visual, and it gets people thinking about alternatives while still staying in their comfort zone. It also works nicely with ‘bite-size chunks’ analogies. And you can easily explain why releasing just a technical task doesn’t make sense because no sane person would eat only the lettuce. On a final note, make sure not to do this while people are hungry.

If this sounds interesting, you can practice many other collaborative techniques for delivering the right software iteratively at my upcoming two day workshop in London in mid-March.

Although we have a long way to go things are going OK at the moment and we feel it is bringing some real focus to the development process. However a lot of the early stories are largely admin type CRUD – for
example functionaility to set up user defined entities and their user defined properties within the system and to provide a mechanism for relating these entities to each other
…
Does anyone have any advice about how to write tests for this sort of stuff or any experiences in starting out with BDD they can share.

CRUD is not a user story, it’s a screen. It’s not a business function, but implementation detail. Why do the business users need a particular CRUD screen? (I know it sounds as a stupid rhetorical question, but I’m serious) What does it allow them to do?

Often you don’t need to implement an entire CRUD screen and deliver them one by one. Sometimes there is value in releasing two screens that allow you to set a subset of properties but together they bring value. You can then automate these tests through the UI and use the CRUD screens, but that will be hidden in the automation layer. Say for example that we have a risk report that lists users and their card numbers:

Scenario: Only regular customers with a specific risk category and 
              card type show up in risk reports
Given the following users
|name| type| card type| card number| risk category|
|Mike |VIP | Mastercard |53111 11111 11111 1111|X|
|Tom |VIP| Visa |41111 11111 11111 1111|Y|
John |Regular |Mastercard |51111 11111 11111 1111|X|
|Steve |Regular |Mastercard |52111 11111 11111 1111|Y|
Then the risk report for Mastercard and category X contains the following data
|John | 51111 11111 11111 1111|

This could, for example, invoke the user CRUD to save a user name, type and risk category and a completely different CRUD to save card details for that user. Any other information that would go on that CRUD (addresses, card expiry dates etc) aren’t part of this story or criteria because they are not important for this particular report.

Start with the outputs, the reports that the system produces, instead of data entry operations. this ensures that you have the data you need to produce the reports at the end, and that you don’t have superfluous data that nobody really cares about. if you don’t do that, the resulting data schemas are often overcomplicated and contain many things that simply aren’t used at the end at all.

Here is a situation I have seen a few times more than I would have wanted: the team and the clients split the whole project into several phases that will be shipped every few weeks; early deliverables mostly provide the plumbing for the later ones, and have only enough user-interfacing functions for the plumbing to be tested functionally. The clients set up and play with each new release. They dutifully give their feedback and participate in testing each time. When the whole thing comes together on the end, it turns out that the system still requires a lot of changes to solve the problem that it was intended for — the iterations and customer feedback failed to provide the directions.

The problem hides in way that the project was split: early deliverables mostly provide the plumbing for the later ones and the whole thing comes together on the end. So, until very close to the end of the project, the deliverables could not really be used in production. Unless the software is really being used, we cannot expect authoritative feedback. The customers may be able to give us their general gut-feel and comment on functional correctness, but usability and feasibility can only be measured once the software is really actually used by people in the course of their daily business. Technical integration issues as well as usability (human-integration) issues are expected and typically not problematic to solve, but we must not let them pile up. Early feedback should allow us to solve some of those problems during the whole project, instead of finding them out on the end.

One more reason why deliverables should be used in production is that the customer’s business will rarely stand still during the course of the project, making the end result effectively a moving target. If the deliveries are not really used until the end of the project, then we have no idea whether they are in touch with the present business requirements. On the other hand, if the deliverables are used every day, we will know that straight away – and most likely the clients will ask for adjustments early.

So how do we avoid this problem?

Here are three ideas to avoid the problem:

• Focus on user stories, not on technical use cases
• Plan deliveries so that they are complete in the sense that they can really be used in production
• Divide and conquer: Split a big project into several mini-projects, and then focus on delivering those in a sequence

Focus on user stories

User stories have emerged over the last several years as the preferred way of planning scope for agile projects. Focusing on user stories is a great way to split the project into parts that actually bring value to the customer, since every user story should have a clearly defined customer benefit. So planning deliverables in terms of user stories instead of technical use cases is a great first step.

A pitfall that we must avoid is to then batch stories based purely on technical dependencies. Although it makes perfect sense to have early deliverables provide the technical infrastructure for latter ones, and that type of dependency must be taken into consideration when planning deliverables, ordering purely based on that leads to the exact problem that we are trying to avoid. This is effectively incremental, not iterative development.

Have production-worthy deliveries

Instead of focusing on technical order, it is better to try plan deliveries so that every shipped version of the software can really be used in production. Although it is not always possible to adhere perfectly to this idea, it is a good general rule and we should try to keep the deliveries complete in that sense. While working on the delivery plan, just ask the clients this simple question: “pretend it’s magic and in a few months we give you these stories — would that piece of software be complete in the sense that it can actually be used in production?”. If the answer is no, go back to the drawing board.

Doing production-worthy deliveries several times during the project might require writing some throw-away code, but that is a small price to pay for early feedback (to programmers) and getting business benefits early (to customers). For example, several years ago I worked on a trading system that was supposed to replace excel-based trading and distribution management. One of the major problems that we were supposed to solve was optimising transport routes and providing transport plans to local centres. So our first delivery was a piece of software that allowed traders to input their trading plans (from the excel sheets), produced an optimised transport plan, and allowed them to export it back to excel. The first and third part of that delivery were thrown away later, but we got good feedback on a fairly complex part of the system and solved tons of edge cases that were not identified during initial planning. Traders were happy because this saved them a few hours of work every day, and they got that several months before the whole system was in place.

Break a big project into mini-projects

Reg Braithwaite wrote that his Golden Hammer for building software iteratively is to divide it into separate products. This makes excellent sense for building larger systems where the software will perform a lot of functions for a lot of people on the end. By dividing one single large product into smaller products we force ourselves to have cleaner interfaces and looser coupling between software components. This can allow us to develop them independently, and will also reduce the risk of the whole thing missing the target on the end. Each of those mini-products would be a deliverable, and it should be usable on its own.

I recently worked on a highly scalable e-commerce web site. We split the project into three separate products: an e-commerce integration platform, high-performance caching system and the actual of web site. The integration platform was delivered first, with a low-fi card processing web that was used for some 3rd party integration. It went live a year before the rest of the system, helping us flush out any problems with transaction processing. We had new requirements about this part of the system every few months, driven by the fact that people were actually using it. So when the project was close to the end, this integration platform was in touch with reality. The caching system went live next, and it was used to support a mobile content distribution application. Both the low-fi card processing site and the mobile app were not part of the initial plan, but we were able to slot them in nicely. The web site front end then built upon the foundations of those two systems that were in production for months. Thinking about the whole project in terms of separate products both helped us deliver it quicker (because we had real production feedback for parts of the system early on) and also increased the value of what we were doing (since the same components were used to support different applications and smaller projects that started after that big one). Divide and conquer approach paid off big.

Deliveries and iterations

I know that some of you, reading this, will now start to complain how iterations should be two to four weeks long and that the business problems are often larger than what can be solved in such a short period of time. I intentionally avoided using the word “iteration” in this discussion, because I do not think that every iteration needs to be delivered.

If an iteration result is not complete in the sense that it can really be used every day, then the iteration delivery will just sleep on the UAT servers after testers play with it for a while. To get quick feedback on such iterations, I propose giving clients access to the local integration environment and letting them play with the software there. You can avoid packaging and polishing and they do not have to update their local installations.

If an iteration produces software that can really be used every day, then ship it and make the clients use it. It is definitely useful to get feedback early, but having something go live, even if it causes rework or throwing away parts of software later, is much much more valuable than software that is only tested. It has more value for the customer, since they are getting a part of their problem solved earlier. It also has more value for the development of the whole project, since it brings real, authoritative feedback from everyday use.

Image credits: Emin Ozkan and Sachin Ghodke/SXC