Gojko Adzic » specification by example

Designing good Cucumber feature files

gojko — Wed, 18 May 2011 23:43:14 +0000

Alister Scott wrote a really nice paper on designing good specifications with examples/bdd scenarios/acceptance tests – focused on Cucumber, but applicable to other tools as well. He captured how to evolve horrible scripts into something quite useful. I strongly recommend reading it, especially if you’re working with UI tests.

Now go to http://watirmelon.com/2011/05/18/specification-by-example-a-love-story/ and enjoy!

A fresh perspective on the specification/script problem

gojko — Tue, 10 May 2011 01:56:59 +0000

I’ve been ranting, writing and teaching about the danger of using scripts as specifications for a while. This is one of the top reasons why teams fail with specification by example. I’m by no means the first or the only one to warn about this. Ward Cunningham and Rick Mugridge wrote against that in FIT for developing software. David Peterson wrote about that in Concordion Technique. Dan North recently wrote about that in Whose Domain is it Anyway. We all presented different heuristics to determine whether examples in a specification are at the right level of abstraction or not, mostly focusing on the difference between a series of steps that explain how something is done and a more direct explanation of what a system needs to do. But a relative novice was able to look at this with a fresh pair of eyes and come up with a much better way to explain it.

It is truly amazing how much clarity a fresh perspective, uninhibited by lots of background knowledge, can offer. During a recent workshop in Oslo, while we were discussing the difference between scripts and specifications, John Inge asked:

So if I summarise something and I use different words, then it is not a good specification?

In this question, he formulated a fantastically simple yet powerful way to decide whether we need to refine a specification with examples further or not. If I try to summarise a set of examples and use different concepts, skip over parts and explain them differently, then the examples will probably be hard to understand later. It might also point out to unnecessary details or duplication, which means that the examples are not at the right level of abstraction. This also means that the names, concepts and relationships in the mental model of the person explaining a specification aren’t directly represented in the examples. If the person explaining examples is a business user, this is a failure from the point of aligning business and software models, regardless of whether the examples illustrate a script composed of a series of steps or not.

If a summary of examples only requires the use of the concepts, names and relationships mentioned in the examples, that probably means that those examples are very direct and cannot be simplified further. But it also means that the examples are aligned with the mental model of the person explaining them, which is a good start for the application of ubiquitous language.

John wrote a blog post about this, summarising the idea:

To see if you have captured what in your examples/tests you can use these criteria:

You are using keywords from the examples to summarize them

You can’t summarize the examples in more simple terms.

Try it now, for example on the two sets of payroll examples in this post. They both represent the same thing, so you can summarise or explain the examples for both cases in the same way (the header of the second version is probably a good summary). In the first case, many of the concepts and relationships from the summary are not directly expressed in the examples and the examples contain a lot of additional cruft. In the second case, the summary and examples are completely aligned.

To me, this looks as a very simple but powerful heuristic. How does it work when you try it on your examples?

If you’re interested in good practices for ATDD/BDD/Specification by example, learn how successful teams apply that in lots of different contexts in my upcoming half-day workshop

Specification by example: Executive Summary

gojko — Wed, 05 Jan 2011 15:20:28 +0000

I’m running a half-day seminar on Specification by Example in London on 2nd of February. The seminar covers an introduction to Specification by Example and the most important results of the research I’ve conducted for my new book on how 50+ teams, from small web startups to teams working for large investment banks, got big pay-offs from BDD, agile acceptance testing, specification by example and related techniques.

The seminar should specifically fit people who are too busy to attend training otherwise, such as team leaders, managers and senior technical people. Click here for more information and to register

The principle of symmetric change

gojko — Thu, 02 Dec 2010 12:29:39 +0000

One of the most important lessons that I’ve learned as the result of the research conducted for my upcoming book Specification by Example is that many of the most common problems people have with implementing BDD or agile acceptance testing come from a misalignment of conceptual models. By changing our view at the specifications/tests we can make most of those issues go away instantly.

The most useful software models are the ones where the technical view of the world is aligned with the relevant business domain model. This ensures that one small change in the business domain (new requirements or changes to existing features) results in one small change in software. This is one of the key ideas of Domain Driven Design. The alignment of the technical and the business models ensures that software evolves nicely with the business and that we never get into a position where a small report change requires a rewrite of the entire system or six months to implement.

The same is true for the other artefacts produced for software — especially tests and documentation. When the conceptual model used in the tests is aligned with the business domain model, one small change in business will result in one small change in tests, making the tests easy to maintain. This applies to the language used in the tests, the abstractions used to describe features and the way that the tests are organised. So I propose the following principle to get good executable specifications/acceptance tests/bdd scenarios/feature files/fitnesse pages or whatever you call them:

The principle of symmetric change
One small change in a business model should require one small change to executable specifications.

To achieve this, ensure that:

each specification should be focused on a single concept of the business domain model (which might be a business rule, step in a flow, or something higher such as the overall definition of a business process)
the concepts used in key examples in a specification are closely aligned with the business domain model and reflected in underlying software domain model. this includes naming (concepts are described in the business domain language) but also the relationships between them.
the (automated) validation procedure is clearly separated from the specification of key examples
the specifications are organised into a living documentation system according to the conceptual relationships that exist in the business domain model

Thinking about aligning key examples and specifications with the business domain model instantly solves many of classic acceptance testing/bdd problems we identified during a CITCON workshop in Paris last year. For example, “Focusing on How, not What” is a sure way to write tests that are very hard to maintain but lots of teams still do it. By thinking about aligning the language and abstraction concepts in the specifications with the business domain model, we force ourselves to describe what the software should do, not how it does it. This separates the specification (what) from the validation procedure (how can we check it in software). The validation procedure can then be automated using a tool. This is a classic problem that Ward Cunningham and Rick Mugridge have identified in Fit For Developing Software nand advised readers not to automate flows. But that is an overly generalised statement – it confuses people who deal with genuine workflows, for example in payment systems where business workflows are everything. David Peterson advised people to focus on specification over scripting, which is less generalised but still not directly to the point. Flows or scripts aren’t the real problem – it is mixing the business definition of a function with the validation procedure.

Thinking about specifications and tests in this way also prevents people from falling into the trap of The Sine of Death by UI Test Automation. Tests described with low level technical detail, in the language of technical UI interactions, are everything but aligned with a business model. If anything, they are aligned with the current design and layout of user interfaces. A small change in business requirements can have a huge impact on such tests, requiring hours of updates to tests after just a few minutes of changes to the code, which then fails the symmetric change principle.

Ensuring that the specifications are aligned with the business model and in the business language, organised according to the relationships the underlying concepts have in the business model solves all the reasons behind the “Tests unusable as live documentation” family of problems, and ensuring that the previous documents are still aligned with the new model resolves most of the issues we described as “Test code not maintained with love”.

Specification by Example – First few chapters available

gojko — Wed, 03 Nov 2010 01:32:04 +0000

The first few chapters of my upcoming book Specification by Example are now available through the Manning Early Access Program. Use the promotional code specbyex50 to get a 50% discount on the list price until December 1st.

The book is a collection of 50+ case studies of teams that got big payoffs from specification by example, behaviour driven development and agile acceptance testing.

We’ll update the early access shortly with more chapters and a draft of the entire book should be available relatively soon. I’d love to hear your comments.