In The Sources of Power, Gary Klein describes his research of variations in understanding orders among commanders and tank platoon leaders, reaching conclusions that giving answers to ‘what’ and ‘how’ does not prepare individual teams for reacting to unforeseen problems. As I was reading this, it struck me that this mismatch is also present in software development. Most of the time we get or give answers to ‘how’ and ‘what’, and then spend enormous effort on coordination when problems arise, as people start pulling in different directions.
Real-world projects are far from ideal - most of the time we have to compromise between cost, time, features and quality. Project stakeholders decide globally on trade-offs, but even junior programmers make those choices every day, on a smaller scale. When someone decides to spend a few more days testing the transit algorithm and covering every imaginable edge-case, he is following an idea that the best possible transit in some strange exceptional case is more important then earlier delivery. That might, or equally possible, might not be the case. You can hardly blame anyone for choosing the wrong option without actually having a framework on which to base an educated guess.
Dr. Klein and his researches reviewed various Commander’s intent documents, reaching the conclusion that they should be more focused on goals and contain ‘why’ and ‘watch out for’, in order to give individual platoon leaders enough information to improvise and cut corners when needed.
Applying to software development
Focusing on goals is a well known technique for setting up a discussion framework with clients or stakeholders. However, goals are rarely passed on – instead they transform into specifications and requirements. Goals surface only when they become a code-red priority (‘must be live before Expo opens on Monday’) or after the initial delivery failed to meet client’s expectations. Even then, they are again passed on wrapped into a list of tasks or requests.
If communicating goals and anti-goals helped to coordinate tank platoon leaders in low-visibility desert conditions, it should also help programmers whose biggest optical obstacle is an occasional lack of caffeine in the veins. I decided to test that theory. For the past twelve months, project goals have found their way into everything from wiki sections, to power-point presentations, requirements and specifications, and I have made sure that everybody, down to the youngest junior involved, knows why we are doing something.
The other part of Dr. Klein’s suggestion, anti-goals, was a bit more tricky. Certain problems like changing requirements or resource issues can hardly be called unexpected - we just don’t know specifics of those issues, but they will undoubtedly happen. So, we build in some slack into the plans in advance - most software development methods minimise or mitigate the risk of such issues in one way or another. Since it’s not very likely that Luftwaffe will carry out a surprise attack our database team, this ‘watch out for’ advice on a high level typically consisted of a small list of pitfalls - either identified during prototyping or design brainstorming. It was much easier to identify ‘watch out for’ on a lower level, related to individual tasks - so in every hand-off I would try to devote at least ten minutes to discuss possible technical problems.
Improved communication
It is very hard to quantify the effect of this technique, since software projects are not developed in lab conditions. Even if we could set up two exactly same teams, with same skill sets and mindsets, put them to work on identical projects, one communicating goals and other communicating only tasks, I do not think that the results would be relevant in the real world. Change, at least in my software projects, is mostly driven by external factors, so projects developed in isolation would not face the same challenges. Though I cannot give you some nice round figure followed by a percentage sign and preceded by a fancy three letter acronym like ROI, my subjective feeling is that this experiment was a great success.
The biggest difference with junior programmers is less wandering into the wilderness. After hitting a programming equivalent of the brick wall, it was typical for them to just slide from the main road and start working on utterly complex workarounds, until somebody discovers what they are doing or until they just give up and ask for help. As we have started talking about some of those brick walls before the journey, there was a lot less of searching through the woods.
With senior colleagues, there was not much difference. They would know the overall goals anyway, and would understand the pitfalls - having a list of goals readily available all the time just made it a bit easier to communicate, as it set a clear platform for adding or removing features (Should we do a new management console? Does it help any of the goals? If not, scrape it!).
The effect was most noticeable with mid-level developers. First, people started reporting problems early - which is a very, very good thing. Sometimes they found a genuine problem, sometimes they just misunderstood the original idea, but in any case there was a lot less unpleasant surprises later. I guess that everybody could compare their understanding of ideas and specifications with the framework set by overall goals, and quickly notice any conflicts. Knowing goals probably helped them to see the big picture (or at least see a bigger part of it), and make sure ‘what’ and ‘how’ are understood consistently throughout the teams.
Even more important was the fact that people, who were previously just ‘doing their job’, started to get much more involved. I guess that knowing goals was received as a sign of personal recognition or respect and an invitation to give their opinion on things. There was much more feedback then usual - including some very good ideas.
More communication is consistent with Klein’s final addition to Commander’s intent: ‘Now, talk to me’.
The Modified Commander’s intent template from Sources of Power:
- Here’s what I think we face
- Here’s what I think we should do
- Here’s why
- Here’s what we should keep our eye on
- Now, talk to me
Important challenges
Though the technique sounds very simple, it’s not without challenges. The greatest one was identifying and expressing true goals, not just repeating specifications or requirements. ‘Our aim is to develop a transaction server’ is not goal, but ‘our aim is to enable the system to serve ten times more customers’ is. However, it is hard to distinguish between those two in the beginning. If goals just repeat the specifications or requirements, then the effort of communicating them throughout the organisation is pointless. This problem was also observed by Dr. Klein in his analysis of tank commanders’ orders. My rule of thumb is that a goal must be an answer to a real-world problem or customer benefit, and not suggest the solution. I consider replacing anything that might provoke a question like ‘but why?’ with an answer to that question. On the other hand, taking that to the extreme of including ‘making more money’ or ‘beating the competition’ into project goals is useless. True goals lie somewhere between customer’s mission statement and their project vision.
The second big challenge was explained by Scott Berkun in his book The Art of Project Management (another very strong case for of focusing on goals but from another perspective - I will publish the review of that book along with the sequel to this article next week). True goals are consolidated and prioritised and a single project should not have too many goals. It’s very hard to get everyone to agree on less than 10 goals and even harder to identify 2-3 ‘must haves’ and consider the rest as ‘nice to have’. Clients will tend to define too much goals and all of them will be ‘must have’ - but identifying non critical ‘nice to have’ objectives is necessary in order to provide a framework for correct compromises. However, putting the goal list on paper actually gives you some leverage in negotiating, since it becomes obvious when the project is too broad or if some goals are unrealistic or conflicting.
The third big challenge was to actually pass on the goals throughout the organisation without overloading people with information. The project as a whole might have 10 goals, but each sub-project or thread will have it’s own set of objectives on a lower level. Several thread goals will typically map into one main project goal, and thread objectives might also express benefits for the solution provider, not just the client. ‘Enabling easy integration of third-party e-wallets’ is an internal goal of the software provider that will speed up later development, but helps the overall customer goal of ‘supporting all popular payment gateways’. People working on a single module should be concerned with overall objectives and goals of their project thread. Overloading developers with goals of other threads will just introduce confusion. Not defining thread-specific aims is equally bad, since those individual sub-projects will not have a focused goal framework.
Focusing on goals pays off
I find this technique very positive in practice, living up to the theoretical promise. Though it is by no means a magical ACME powder that will help you conquer all the software problems just by adding water, it improves understanding and helps to establish a common framework for communication. The amount of effort required to implement it is a small price to pay for making the project cruise somewhat easier.
