How not to build a castle on a swamp

From time to time, programmers sail into uncharted waters – the business and clients agree to give them some time to build a prototype and evaluate a new technology, or explore ways to get something done. A month or so later, after a software scaffolding is built to show how something can be done, one of the worst examples of corporate insanity happens – almost certainly. Some brilliant mind thinks of a way to save money or deliver earlier by patching up and shipping the prototype.

I have never seen that produce any good – not once. By ‘good‘, I mean a better product, delivered faster than if the prototype was thrown away. Yet, no amount of convincing or logic seems to be effective against the promise of earlier delivery. The whole idea of prototyping is to test potential solutions, which means that a lot of them will turn out to be wrong. Prototype code is typically inconsistent, as it reflects experiments and trials, and evolves with gained knowledge. But, as the prototype is not meant for delivery, the changes do not propagate to parts already written unless absolutely necessary – so most prototype code will not be re-factored and re-tested. Prototypes simply contain too much hacks and problematic code to be used as a foundation for production software – so asking me to build something on that reminds me of the Python quote ‘Other kings said I was daft to build a castle on a swamp, but I built it all the same, just to show ‘em. It sank into the swamp‘. My only consolation, so far, has been that this insanity also affects others, and is even recorded in ‘The inmates are running the Asylum’.

Sometimes, I get away by just agreeing to patch the prototype and then re-write the software properly. But this can backfire badly if the ‘brilliant’ minds find out (and they are typically in power). A recent post Don’t make the demo look done on Creating Passionate Users attacks this problem from a different angle – suggesting to handicap the prototype intentionally. However, this can also backfire – as customers may be unsatisfied with the results.

I think that I finally discovered a good solution – one that does not require any politics, long arguments or smuggling the re-write as ‘work on cleaning features’. I tried it a few months ago and got away with it, without any complaints.

Write the prototype in an obscure language

Do the prototype so that it cannot be shipped. Choose an obscure language or platform. When someone suggests that the prototype should be patched up and shipped – the answer is simple: ‘OK, but it is written in [insert obscure language here], so we need a month to re-write it in [insert production language here], and then ship it’. No arguments, no politics. They all agreed to pay for the prototyping in the first place, so there is no additional cost – just a missed ‘chance to save some money and time’.

I did mine this time in Python and on Linux, because the target platform for the final version was Windows and C#. Don’t take this wrong – it’s not that I do not consider Python/Linux as a production platform (I have delivered Python/Linux software to several other clients, with great success), but Python/Linux combination was completely unusable for production in this case. It would cost clients much more to introduce a new platform than the amount they would ‘save’ by cutting the development instead of proper prototype re-write, as they would have to train their IT staff and support. There are quite a few choices today – Python, Ruby, Groovy, JavaScript. Even if the target platform is one of them, the prototype can be written in some other.

On the beginning, everyone knows that the prototype is not going to be production-quality code, so no one should complain about some ‘obscure’ technology, since the software is not written to be shipped to the market. The prototype will actually be written much faster in a dynamic language then in Java, C# or other similar languages – which will help in convincing stakeholders to approve it.

When our latest prototype was done, there was even no attempt to patch it up – which surprised me. In any case, this was a win/win situation – writing the prototype in Python allowed us to start a web server and open an interactive console to re-wire it and perform live brain-surgery while the server is running. I cannot imagine doing that in Java or C#. In a month, we wrote the functional equivalent of at least 4-5 months of C# code. On the end, the prototype was thrown away, and everything was re-written, discarding half-developed solutions and throwing away ideas that turned out to be wrong.

Image credit: Kiel Latham / SXC

I'm Gojko Adzic, author of Impact Mapping and Specification by Example. To learn about discounts on my books, conferences and workshops, sign up for Impact or follow me on Twitter. Join me at these conferences and workshops:

Product Owner Survival Camp

Specification by Example Workshop

Conference talks and workshops

6 thoughts on “How not to build a castle on a swamp

  1. Maybe this is a place where those long forgotten (or newly created!) obscure languages can start making headway for themselves. I mean, imagine you write your prototype in Haskell, OCaml, Scheme or Smalltalk. Those languages are pretty outside the norm today. I am starting to think Python and Ruby and becoming more and more mainstream, especially as they are ported to the JVM and .NET framework, so they might not be as good of possible targets for this kind of work — although your idea of writing it on Linux for a Windows shop is actually pretty brilliant. Excellent idea.

  2. Hand-draw your prototype on paper. Then if they want to save money, you can just xerox the paper and mail it to all your users! Seriously, though, not too many management types will confuse handwriting on paper with production software.

  3. What is a prototype?

    Is it a means of testing UI concepts? It should not be usable anyway as it should not have anything but UI.

    Is it a means of testing some real functionality? In this case, if it is developed test first there is no reason to throw it away.

    An agile prototype is no more than a specific selection of stories to achieve a prototype milestone. It makes total sense to continue and deliver it — we are not wasting effort.

    Is it a means of exploring a technical issue? This is where spikes come into play. You test ONLY the technical issue without any connection to real stuff.

    In all of these cases there is no danger in the prototype being carried forward:
    In the first case – it’s just no possible – there’s no real code there.
    In the second case – go ahead, it’s production code as is
    In the third case – there is no prototype anyway.

    So, I guess I don’t quite see where the problem is. With an agile state of mind the problem just isn’t there.

  4. Hi Amir,

    thanks for the comment. I was referring to a prototype as a technical a way to test and convince the stakeholders that something is feasible (and that they should finance the project). Depending on the key risks, this can be a technical spike, but can also be a mini version of the target system, stripped of most non-essential functionality, but again showing something end-to-end.

    The particular prototype I wrote about involved several middle-ware components and a web site built on top of them. Each of those MW components could be seen as a technical spike, but they did not prove the concept alone – only a mini-system as a whole could prove the concept.

    So it was a way of testing real functionality, a specific selection of stories. However, I think that it was beneficial to do it as quick and dirty as possible, as the goal of that “mini-project” was to find a way to solve the risks (technical spike) without burning resources. We estimated that doing it properly would take 3-4 months (which turned out to be correct when we did it on the end) and the quick-and-dirty prototype was done in a month. As it was quick and dirty, building the rest of the system based on that would be very error-prone and cause a lot of problems in production.

  5. I agree with picking an obscure language as a general strategy, but if you look at some recent success case studies…

    YouTube – no rewrite

    del.icio.us – being rewritten now using a php framework, but got acquired and had massive traffic as a perl app

    Digg – probably has had a

    Twitter – despite their Ruby scaling woes, they are now in the Alexa top 700, and are still RoR-powered. Their backend tho does have a bunch of fancy mechanisms to deal with massive API requests.

    If I had to deal with management, I’d tell them “this is just a prototype” and write it in ruby/python/whatever, but in the back of my mind I’m really thinking the “prototype” can and will evolve into the final app.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>