To enable this feature, override the default test listener in the JUnitHelper instance you use for test runs (third constructor argument), and supply a JUnitXMLTestListener instance passing the folder where you want the XML files in the constructor argument. For example:

 String htmlOutputDir=new File(System.getProperty("java.io.tmpdir"),
                                  "fitnesse").getAbsolutePath();
  String xmlOutputDir=new File(System.getProperty("java.io.tmpdir"),
                                   "fitnesse-xml").getAbsolutePath();
  String fitNesseRootDir=".";
  JUnitXMLTestListener xmlTestListener=new JUnitXMLTestListener(xmlOutputDir);
  JUnitHelper helper =
      new JUnitHelper(fitNesseRootDir,htmlOutputDir,xmlTestListener);
    helper.assertTestPasses(".....");
 

Grab the source from my github repository. Also, check out the full example.

My experience with Fit and other agile acceptance testing tools is that they cost more than they’re worth. There’s a lot of value in getting concrete examples from real customers and business experts; not so much value in using “natural language” tools like Fit and similar.

The two failure patterns that Shore describes in his post are falling back on testers to write everything and merging acceptance and integration tests. I’ve experienced both of these myself, and it seems that they are common in general. We discussed both during the top 10 ways to fail with acceptance testing openspace session at CITCON Europe last year. However, there are good ways to solve both problems.

I never really expected customers to write anything themselves, but I was relatively successful in persuading them to participate in specification workshops that led to examples which were then converted to acceptance tests later. Another idea I discovered while doing the research for my new book is discussing the key examples with customers and then going off to write detailed test pages, which then get verified by the customers. The third good idea is doing ad-hoc specification writing sessions when a developer needs information, by involving a tester and a business analyst. This is a lot less formal than a specification workshop and gives you similar benefits if you have all the knowledge in the room (or readily available) most of the time.

Not preserving acceptance tests as a separate group and mixing quick and slow tests is something that most people, at least according to my ongoing research, get burned with at some point but again teams learn from that and improve.

One of the biggest benefits from acceptance testing for me was that the teams finally get a source of information on what goes on in the system as reliable as the code itself. Without acceptance tests, code is the only thing you can really trust and any other documentation gets outdated very quickly. And such tests are much easier to read and understand than the code because they are on a higher level and in a natural language. Having a live specification helps me quite a lot when change requests come in later. It also helps with handing over and taking over code. Acceptance tests stay relevant throughout the project because they are automated, and automated tests are kept up to date in order for them to pass. Automation, and consequently a tool, are necessary to get this benefit. With informal agreements and on-site reviews that Jim Shore describes, I guess something else needs to be in place to facilitate this.

I agree with Shore that it takes a while for the problems with tools such as FIT to surface, but I’m not sure whether that is tool related or not. Most people I spoke to so far said that it took them between six months and a year to discover that acceptance testing isn’t about the tools but about communication, and that the biggest benefit is in the examples as Shore wrote. A notable exception to six months to a year rule was Lisa Crispin’s team who generally started out knowing what they need (but that’s because she had done it before). Clear examples and improved communication are the biggest benefits of the process, but using a tool brings some additional nice benefits as well. A tool gives us an impartial measure of progress. Ian Cooper said during the interview for my new book that “the tool keeps developers honest”, and I can certainly relate to that. With tests that are evaluated by an impartial tool, “done” is really “what everyone agreed on”, not “almost done with just a few things to fill in tomorrow”. I’m not sure whether an on-site review is enough to guard against this completely.

Dave and I will discuss some of the common pitfalls faced by teams in getting to grips with Fitnesse. We will show examples of good and bad acceptance tests, illustrating how different styles of fixtures lend themselves to different types of tests. We also highlight some of the features of Fitnesse that allow you to keep your tests expressive, useful and easy to maintain.

Make this interactive and bring your own test examples to be discussed, critiqued and improved upon in a group workshop.

For more information and to register, see
http://skillsmatter.com/podcast/agile-testing/david-evans-gojko-adzic-interactive-agile-acceptance-testing-clinic

The MoSCow model, at least when I tried to implement it, suffered from the Priority 1 Paradox:

The more we ask customers to narrow down priority 1 features the more they’ll be scared to leave anything out of that category.

The complaint is always the same: everything must be there – especially all the old functionality when replacing a legacy system. Prioritising so that most of the stuff ends up as ‘Must’ pretty much defeats the point of the exercise.

I am currently reading Stand Back and Deliver: Accelerating Business Agility by Niel Nickolaisen and colleagues from Accelinova. Among other interesting ideas, I found a real gem. This book advises aligning business processes (equally applicable to software features) along two axes: mission critical and market differentiation. Nickolaisens calls this the Purpose Alignment Model:

In a case study, the authors describe a discussion with a stakeholder who claimed that their IP protection was a differentiating factor. The authors then asked the VP of sales ‘When was the last time you sealed a deal by touting Lifebrands’ excellence in IP protection? When was the last time you felt you should include the IP protection process in sales presentations and collateral?‘ (pp 36) They also offer a simpler version of this rule of thumb as a Billboard filter:

Would you put this feature on a billboard?

If so, the feature is obviously differentiating and should be invested in. If not, this feature falls into the parity category at best. The introduction of the parity category which, as the authors repeatedly point out, holds mission-critical features which are not billboard material, seems as a great way to avoid the Priority 1 paradox. This is to intentionally avoid the feeling that anything not Priority 1 won’t be delivered. The system must have these things too offer a full service, but it just isn’t worth investing too much or being too creative there. Nickolaisen and co-authors suggest that the Parity category is where companies should apply industry best practices and streamline processing.

Overdoing it in this category might actually be damaging to the bottom line. In another case study, the authors describe a project to automate complicated pricing rules. After it was identified that complex pricing isn’t really billboard material, the project was completely turned on its head. Instead of implementing a complex rule engine, the pricing system was streamlined using industry best practices. It turned out that customers were actually happier with the new model, so simplifying a parity process increased the revenue. This was possible because the stakeholders agreed that while the company must have an automated pricing system, it wasn’t a core differentiating factor.

Strategic design ideas of Eric Evans also suggest some good guidelines what to do with such software: if the thing is available off the shelf (generic domains), buy it. If not, possibly get a third party to develop and maintain it. Evans calls these domains supporting, and they are prime candidates for outsourcing.

This article is part of my Building software that matters series. See other articles here

Btw, I’ve created a mailing list for this topic so rather than commenting here, post your comments to the Building Software That Matters mailing list and let’s discuss them!

Image credits:Steve Rhodes