Quick note on timeliness
I wrote this article in 2006, and I was quite fond of it then. Most of the theory explained here still stands, but since then, I’ve learned much better techniques for implementing those ideas. Check out more recent planning articles on this web site for more on current techniques.
Up until a few years ago, most projects I got involved in had a recurring pattern of getting stuck before they even started. They clearly needed more direction in the opening scenes. I am not really a fan of big, formal methods, so I was looking for something light-weight which would help, but not get in the way. Focusing on goals turned out to be a very effective approach to setting the stage for software development.
The way we used to work was quite common – clients would come with a list of wishes, we would meet with them, write down ideas, go back and try to turn produce a list of requirements and tasks; this process would then repeat until the clients were happy with the list. After a few rounds, we would start prototyping and writing specifications, then developing, but features would just get thrown in and out all the time. It was hard to decide when to start turning customers’ ideas and wishes into specifications and it was difficult to know when we have covered everything. If we started to prototype and draw specifications too early, then a lot of work would be wasted when the requirements change; if we waited for too long, the projects would just be a pile of loosely coupled ideas, too broad to take any form or have any sort of unified vision.
##Goal-driven requirements gathering
Goals are excellent as a utility to drive requirements gathering. Goals are much less likely to change than requirements, so focusing on goals provides a stable platform for project work. I cannot pinpoint exactly where I picked up the idea of focusing on goals, so it has probably crystallised influenced by various books and project experience over the last four years. This is by no means my original idea, just a set of established techniques that work well. Scott Berkun’s excellent book The Art of Project Management</a> helped me express my ideas much better and make the technique more effective. Goal-driven requirements gathering process looks like this:
- Identify project goals, prioritise and number them
- For each goal, identify features that help or fulfill that goal. Number the features, and write down related goal next to each feature.
- For each feature, identify system requirements. Number them, and write down related feature next to each requirement.
- When writing specifications or any other supporting documentation, include labels of relevant requirements.
I typically prefix goal numbers with G, features with F and requirements with R. So when I see F16 or R22 I know exactly where to look for more information. Goals should ideally be prioritised, but more importantly divided into at least two groups: ‘must have’ and ‘would be nice’. If the clients can split them into three groups, even better. If they can be divided into four groups, the project is too broad - and should probably be divided into several sub-projects or phases.
One feature may map into several goals, but it has to help at least one goal. If I cannot write a goal label next to the feature, then the feature does not really belong to the project. At this point, there are two choices - either expand the project by adding a new goal, or scrap the feature and leave it for some later project or phase. Same goes for the requirements - if there is no related feature, then the requirement does not belong in this project.
It’s not always a straight-forward process as it sounds, and it’s normal to run up and down the hierarchy of goals, features and requirements. At least there is some sort of traversal algorithm for that tree, which makes sure that everything is covered. And, there is a clear signal when the process is complete.
##Focused projects = everybody happy<
Projects become more focused, as divergent ideas get flushed out into the open - they have no parent goal. Though it might not sound hard to identify unrelated features, that task is very difficult without actually knowing the project goals. During a recent meeting on web portal restructuring, clients asked us to include support for marketing mailing lists. How do we decide if this is actually relevant to the web portal restructuring or not? Where should the line be drawn? Imagine a goal like a spot-light, shining on a few features, then on a bit more requirements and finally on parts of the system intended to answer those requirements.
Magic of written goals is incredible, since it instantly becomes obvious if the project is too broad or has not been thought over properly. Too many goals clearly signal that the project should be split into several phases or sub-projects. In my experience, having two or three top-priority goals is ideal, but I can live with five or six. This is consistent with Berkun’s rule of thumb that ‘project vision document should have somewhere between three and five high-level goals’. With ten or more top-priority goals, project should really be split into phases, and each phase should be considered a project.
If, on the other hand, the project does not have clear and coherent objectives, then I see a big red flashing DANGER sign. A sudden pain in my stomach comes shortly after, reminding me of a community portal in which I was involved about six years ago. The original estimate was that it would take three months, but the project was cancelled after two years of heavy development, and everybody was angry. Clients were just throwing requirements in and out, never happy with the way the portal works. Developers were angry because they were running around in circles. Management was angry because there was no money.
Contractual obligations and financial issues set aside, I strongly believe that clients should get a solution for their problem on the end of the project, even if that solution is not what they originally requested. It’s up to us to provide focus and guidance for the project, and make sure that their goals are met. That also means helping them to identify and express objectives if they cannot do it themselves. However, if it’s impossible to define clear and consistent objectives for the project, I suggest not starting at all. Somehow, I don’t like to embark on a ship when it’s only destination is the bottom of the ocean.
Having goals prioritised makes it easy to prioritise development, since it’s obvious which features and requirements are essential for top-priority goals. On bigger projects, preparing for actual software development may take months, and it’s not uncommon for features to get thrown in and out after the first draft of specifications or prototyping. Sometimes, goals are redefined. Having a labelled tree that spans from goals down to specifications makes it easy to cut an entire branch, and this came in very handy as a way to make sure which parts of the system are no longer required. Without a written trail, it would be hard to throw out all unnecessary (and unwanted) requirements and parts of the specifications. Also, if something turned out to be much harder to implement than we originally thought, it was easy to check if the feature was really important and could it be re-negotiated.
Preparing a list goals at the beginning of a project, and maintaining them throughout the implementation, also helps communicating objectives with team members. That is a good technique for improving understanding and coordination.
On the end, written list of goals clearly defines an exit criteria and measure of success for the project.
A better perspective for looking at the requirements
It turned out that focusing on goals helps a lot to identify true requirements and design better solutions. Clients and users think about software in different terms and on different levels of abstraction then developers. Their comprehension is, in technical terms, superficial - so their ideas can be a tip of an ice berg. Though the requests might seem simple (‘add a button to this screen that recalculates the liability’), they describe genuine problems - which we should really be interested in. Why do they need to update that kind of information manually? Is the liability not being calculated properly? Asking the clients to describe the actual problem or objective, not a sequence of steps to implement the solution, makes a big impact - we can then see the problem from a large-scale perspective, usually leading to a systematic solution.
Sending movies over Bluetooth
My team designed and built a J2EE-based system for Bluetooth content distribution. It was initially used for pushing small files and text messages to mobile phones. After a few months, the client wanted to add on-demand content pull, and distribute large animation/film files. Enabling on-demand content pull required significant system changes, and the Bluetooth network is not suitable for sending five-megabyte movie files to hundreds of mobile phones all the time. As always, the client wanted it done as soon as possible, ideally in less than two months. There was no way that such functionality could be implemented and work properly in that time, and even if it was possible, such a hack would make future support a nightmare.
We asked the clients to identify the problems they wanted to solve by pulling films to the phones. They had an opportunity to sell the system to an art fair, where the visitors would use it as a video tour. From that perspective, ‘video tour’ had absolutely nothing in common with the original solution, except the idea of client software running on mobile phones. We decided to build a small stand-alone player for the mobile devices, that would read all files from a MMC card, and would not communicate with any servers at all. That was done in time for the art fair, and did not break the architecture of the server.
Moral of the story: If a client comes with a list of tasks, ask them for problems and goals.
Problems with goals and how to solve them
The first bump in the road is probably getting clients to participate in the process - most of them come with ideas or a list of tasks, typically already implying a solution. It’s a bit hard at first to make them work with you up the pyramid, from requirements to features and goals, since this approach requires a few steps back and a different perspective. However, benefits of focusing on goals quickly become obvious, as they bring more structure into the conversation and make communication easier. From my experience, clients tend to view discussing goals as genuinely trying to understand their problems, and appreciate it, which is an additional benefit.
Another big challenge is identifying and expressing goals, which I’ve touched upon in How to develop software like commanding a tank. Scott Berkun offers a mnemonic to identify good goals: SMART, which stands for Specific, , Measurable, Action-oriented, Realistic and Timely.
Once you reach the top of the pyramid, and have a list of goals, the hard part is getting the clients to prioritise objectives and split them into groups. Clients will try to define too many goals and insist that all are critical - but writing down goals gives us some advantage in negotiations. Having a list of 20 goals in front of you makes it easy to argue that the project is getting too big, and that it should be split into several phases. Focus only on the first phase, and you are back on track.
Refusing to include ad-hoc features is not easy, and it has to be done politely. The trick is to make sure that the project is focused and does not get out of hand, but at the same time make clients feel that they are in control. Rejecting a divergent requirement should not be seen as refusing to do work, but as a suggestion to implement (and charge) it separately, as part of some other project. I usually create a ‘waiting list’ for features and requirements and just throw ideas into it. Then, after the meeting, I process all the ideas, trying to fit them into the structure. If a requirement does not map to a higher-level concept, I’ll call the client or bring that up during the next meeting - asking to either add a new feature or goal into the list, or set the requirement aside for now and include it in another project. Re-defining goals should be intentionally made a bit burdensome. After a goal gets added, I try to get re-prioritisation, and if the list is getting too big, then start asking for splitting the project into phases.
Scoring goals
For me, focusing on goals helps a lot in providing structure for the opening stages of projects, stable framework for mid-game and a light on the end of the tunnel. It is not a silver bullet, but does provide a level of safety by establishing focus for projects and helping to mitigate some development risks, without bureaucracy required by formal software engineering methods. It is a simple, but very powerful guideline that fits into any development methodology.
Goal-driven requirements gathering is most useful for ground-up development or big system modules - there is not much point in doing it for small change requests. However, even in trivial cases it’s good to understand what the clients are trying to achieve, since there might be a simpler or better solution from the technical perspective.
