Nov 23 2009
The danger of releasing too early
Southland Tales, an apocalyptic sci-fi/drama/spoof/political satire written and directed by Richard Kelly, is a movie that all software team leaders and product owners should be aware of. They should carefully study this forgotten piece of recent film history and learn valuable lessons from it. They should do so not because of the great visual effects or the plot, but because of the story of the movie production, its release and utter commercial failure.
The film is an interesting piece of cinematic work for several reasons. It was an experiment that mixed several genres and put many type-cast actors and actresses in very non-typical roles. The movie has a plot with a twist that, unlike a vast majority of recent films, isn’t at all obvious until the very end. The scenes and dialogues were well written, with a few lines that could have became cult quotes if the movie was more successful. The directing reminded me a lot of David Lynch’s work, so it should have appealed at least to the crowd that follows Lynch. The special effects, especially the Megazepelin, were very good. Moby wrote the music. In short, this wasn’t a spectacular movie by any means but it had its values.
While the movie was in post-production, the director sent a preview to the Cannes film festival board. He didn’t really expect it to be accepted, but sent it more expecting to get some early feedback. In an interesting twist, the movie was chosen for the official festival selection, presumably because the reviewers understood what an experiment it was. Kelly couldn’t finish the editing in time but decided to send a beta version to the festival, without many special effects or final edits, and use the feedback to improve the final version. So far, very agile. He used iterative releases, getting frequent feedback and adjusting the product. The feedback from the festival was painful, but it was a beta version and the director said that the final movie was better because of that. At the end, this movie should have been a success. It surely wouldn’t appeal to everyone, but there are lots of people who would love it. Instead of becoming a cult sci-fi film, the movie failed completely. For 17.5 million dollars invested, it earned only about 300.000. In two words, commercial disaster. So what happened?
The festival critics butchered the unfinished version so much that the distributors never actually gave the final product a chance. It was released to only 85 screens in the US, less than three times the expected starting distribution, and pulled very quickly. As if the distributors were expecting a disaster and it happened, sort of a self-fulfilling prophecy. Let’s be honest, nobody would have won an Oscar for this movie, but it could have become a fairly popular movie at least with the Sci-Fi or Lynch mob. Especially as Kelly already had a cult Sci-Fi film Donnie Darko on his record. It is a lot better than most of the crap that comes out of big studios these days, but it failed to perform.
What does this have to do with software
The key lesson to be learned from this is, I guess, not chasing silly deadlines if you can’t deliver a reasonable product, especially if you’re trying to do something out of the ordinary. Voice dialling is often mentioned in this context – calling a number by saying the person’s name is a very useful feature that was so flaky in early implementations that users and the mobile industry just gave up on that. There’s often a mental rush to release an innovative product or feature first and then corner the market, but there are lots of evidence that dispute this approach, at least for consumer-oriented technology. Google wasn’t the first search engine. IPhone wasn’t the first touch-screen mobile phone. Archos were the first to release a portable media player in 2002, but the market now belongs to someone else.
Releasing too early can kill the product before it has a chance to live. In the early stages of an online casino project in 2004, my server team was actually doing much better than expected. We organised a demo to get early feedback on server functionality and game play logic with mock-up game layouts as graphic designers did not have proper images ready yet but the server was almost done. All our programming achievements vanished in a second – clients hated the mock-up layouts so much that they wanted to cancel the entire project. It took two weeks of very painful politics and re-negotiation to get them to agree to continue the project, including our offer to defer any payments until the entire thing is delivered and they are happy, which meant that I’d have to pay the entire team out of my own pocket for the next six months of work and risk not being paid at all at the end. It was a dangerous gamble that paid off at the end, but I’d rather not repeat the experience.
What about agile iterations and releasing early?
At first, this might seem at odds with current trends in software development and common advise to release early and iterate. But it isn’t. Where we got it wrong in 2004 and where lots of other teams I’ve seen since get it wrong is in prioritisation. We should prioritise projects to be able to release early something that is genuinely useful, not partial functionality that is useless on its own. If we’re building a car for someone, it makes no sense to ask them to prioritise between breaks or steering in the first iteration, clients will just tell us that both are required. Instead, we should understand the underlying problem and see how we can deliver a useful slice. If the clients are really after something that can get people from point A to point B, we can build something that works and is probably not the fastest thing in the world. To continue the car metaphor, this first iteration might actually be a skate-board. Then we can improve it and build a bicycle next, then a motor bike and then a car. Software is flexible if properly designed, and if we have this idea developed upfront, we can actually plan for it and design our system so that it can evolve gracefully. (A very nice visual description of this was given by Jeff Patton during his Embrace Uncertainty keynote at XP Day 2008).
 |
 |
Hi,
Interesting problem indeed. And your proposal is also very intriguing. Jeff Patton is coming here to Iceland next week, so I will try to talk to him about this.
Just wanted to add that I find the perception of the value of an early prototype of a product is something that needs to be managed. One has to be explicit with the customers/users about what is the point of the prototype and what is *not* there yet. Also in my experience it has proven to be helpful when they see extremely distasteful cyan colors that “hurt their eyes”, just to remind them that this is something that no one has worked on.
Best regards,
Petar
I’m a big believer in iterating and releasing products early. Of course, it’s not always the right decision. In the two cases you described, it seems that limiting the scope of early releases is the common issue.
From an analyzing-not-criticizing perspective:
You released to a client but forgot to account for what the average client is looking for. They don’t focus on engineering aspects, and base everything off how it *looks* (remember the Law of Triviality, aka, “What color shall we paint the bikeshed” http://en.wikipedia.org/wiki/Parkinson%27s_Law_of_Triviality ). I’m sure you’d have gotten a similar reaction if you had shown it to a non-engineer (try running your demo past a non-technical family member, and see how they react).
The difference is that if you showed it to a family member, it would have been a lower-risk situation (as opposed to showing it to the client). Or, as the previous commenter suggests, be very sure to explain ahead of time what people should be paying attention to during a demo, and what they should ignore.
Mr Kelly released a beta to the whole world at a high profile event, which means people are going to expect something really good, not half baked. It’s not good to go big when you’re still iterating.
The equivalent of this in startup/software is to build something, use it yourself. Then iterate and release to a friend. Then another. And another. Then, once you’ve got a few friends using it, start inviting bigger batches. All the while, you’ll naturally be picking off the most obvious issues, so that when it does go big, it’s already been proven. And make sure that early users know not to send screencaps to TechCrunch
Nice post, had similar experiences few times when i blamed myself for too early delivery.
The problem with agile or iterative process(although i am a big fan of it) is that some clients (usually non techs) are just not ready for that! But its amazing how many Project managers still cant understand that most of clients just DONT NEED their product to be shown week earlier but without nice design it meant to have. They are so much inspired with books and articles about “release early and iterate” that they just assume that client will be also happy to be iterative. So they forget about more fundamental rule -keep your client happy all the time!
So keep that in mind before you decide to go agile make sure that client is fine with it, understand it and manage his expectations carefully!
Southland Tales should have been a success? The movie failed for one simple reason. It sucked. There’s a reason it rates a 1.9 on Netflix and a 36% on rottentomatoes.com.