• Start with a list of things I’d like to talk about (let’s say good practices for a acceptance testing); it often helps me to create a powerpoint slide deck for a short talk
• identify solutions I’d like to present (say specification over scripting, cross-functional teams)
• for each solution, identify a good problem example that demonstrates how the solution brings a good benefit compared to alternatives (for example, a script that’s perfectly good for a regression test, but useless as a communication tool)
• add the problem before the solution in the slide deck; make sure to remove anything that actually presumes a solution in the problem description, especially things that would give away what you want to present.
• break down the slides so that I can pause between the problem and the solution

This gives me the basic structure of a mix of exercises and presentations. I start with the presentation as I would normally and stop after presenting each problem. Then I split people into groups and let them try to solve the problem. I time-box this and go around while they are solving it, joining each group for a short while. Depending on the problem, I apply two strategies while doing this:

1. If the problem deals with a new technology that people might not know about (eg new specific CLR features), see what they are doing and gently push them in the right direction.
2. If the problem is more conceptual than technical, and has a classic bad solution that people fall into, let groups fall into the trap. Answer questions they ask but don’t try to point them into the right direction yet.

In the first case, we expect everyone to end up in more-less the same place because of the help, so presenting one good solution either from a group or the remaining slides (solutions that come after problems) is a nice end for that exercise.

In case of the other group of problems, we expect groups to arrive at different places – one group might directly fall into the classic pitfall, another might be halfway to the right solution, a third might come up with something alternative that you haven’t thought about. Let one person from each group present where they are, and then discuss solutions with everyone as a group. If your preferred solution was reached by anyone, discuss why that’s better than the rest. If not, present your solution after this and discuss that as well, comparing it to the other solutions.

Aim to run each block for 30-45 mins and have 15-20 mins of discussion after. A typicall 3 or 4 hour workshop gives us enough time to work through two or three of these blocks, depending on the remaining slides to discuss (intro/retrospective etc).

I just wanted your opinion on SLIM as opposed to standard FIT/Fitnesse. Are there things that can only be done via the FIT/Fitnesse route that cannot be done via SLIM? So for acceptance tests and integration tests can I just use SLIM?

We want to exploit the BDD abilities of Scenario tables in SLIM. Ideally I would like to use SLIM to undertake all kinds of tests. I assume it has all the same capabilities? Are there any issues?

On the face of it, SLIM seems to be the preferred way forward for new FitNesse implementations as it is being actively developed and maintained by the same folks who develop FitNesse. FIT is a bit of an orphan at the moment, Rick Mugridge and I were talking about taking over that integration and enhancing it.

In terms of features, SLIM gives you better compatibility across platforms because a lot of the test system responsibility has been taken over by FitNesse itself (parsing HTML, deciding how to interpret a table, storing and reading symbol values). Watch this video for more information about the differences in responsibilities.

On the other hand, because of the way SLIM works, fitlibrary flow-mode interaction is practically impossible. Most of my clients still use flow mode tests as that is a great way to write and maintain a very thin fixture layer for complex tests.

SLIM also allows you to use Scenario tables, as you mentioned. Scenario tables give testers a lot more power as they can script multi-step execution and compose lower level fixtures into higher level tables, without involving developers to do the same in fixtures. Depending on your environment and team, this might or might not make sense. For covering and existing system with regression tests, it probably does. For using acceptance tests as a guide for development, beware of overdoing it. I am very concerned about long-term maintenance costs of such tests. What happens in this case is that people are effectively programming with tables – doing the same in code would allow you to benefit from IDE support for refactoring, file management and all sorts of other things that make IDEs useful. You lose all that by using scenario tables in order to make testers a bit more independent. I would rather suggest training the testers some basic coding skills so that they can write fixtures.

My experience with Fit and other agile acceptance testing tools is that they cost more than they’re worth. There’s a lot of value in getting concrete examples from real customers and business experts; not so much value in using “natural language” tools like Fit and similar.

The two failure patterns that Shore describes in his post are falling back on testers to write everything and merging acceptance and integration tests. I’ve experienced both of these myself, and it seems that they are common in general. We discussed both during the top 10 ways to fail with acceptance testing openspace session at CITCON Europe last year. However, there are good ways to solve both problems.

I never really expected customers to write anything themselves, but I was relatively successful in persuading them to participate in specification workshops that led to examples which were then converted to acceptance tests later. Another idea I discovered while doing the research for my new book is discussing the key examples with customers and then going off to write detailed test pages, which then get verified by the customers. The third good idea is doing ad-hoc specification writing sessions when a developer needs information, by involving a tester and a business analyst. This is a lot less formal than a specification workshop and gives you similar benefits if you have all the knowledge in the room (or readily available) most of the time.

Not preserving acceptance tests as a separate group and mixing quick and slow tests is something that most people, at least according to my ongoing research, get burned with at some point but again teams learn from that and improve.

One of the biggest benefits from acceptance testing for me was that the teams finally get a source of information on what goes on in the system as reliable as the code itself. Without acceptance tests, code is the only thing you can really trust and any other documentation gets outdated very quickly. And such tests are much easier to read and understand than the code because they are on a higher level and in a natural language. Having a live specification helps me quite a lot when change requests come in later. It also helps with handing over and taking over code. Acceptance tests stay relevant throughout the project because they are automated, and automated tests are kept up to date in order for them to pass. Automation, and consequently a tool, are necessary to get this benefit. With informal agreements and on-site reviews that Jim Shore describes, I guess something else needs to be in place to facilitate this.

I agree with Shore that it takes a while for the problems with tools such as FIT to surface, but I’m not sure whether that is tool related or not. Most people I spoke to so far said that it took them between six months and a year to discover that acceptance testing isn’t about the tools but about communication, and that the biggest benefit is in the examples as Shore wrote. A notable exception to six months to a year rule was Lisa Crispin’s team who generally started out knowing what they need (but that’s because she had done it before). Clear examples and improved communication are the biggest benefits of the process, but using a tool brings some additional nice benefits as well. A tool gives us an impartial measure of progress. Ian Cooper said during the interview for my new book that “the tool keeps developers honest”, and I can certainly relate to that. With tests that are evaluated by an impartial tool, “done” is really “what everyone agreed on”, not “almost done with just a few things to fill in tomorrow”. I’m not sure whether an on-site review is enough to guard against this completely.

Dave and I will discuss some of the common pitfalls faced by teams in getting to grips with Fitnesse. We will show examples of good and bad acceptance tests, illustrating how different styles of fixtures lend themselves to different types of tests. We also highlight some of the features of Fitnesse that allow you to keep your tests expressive, useful and easy to maintain.

Make this interactive and bring your own test examples to be discussed, critiqued and improved upon in a group workshop.

For more information and to register, see
http://skillsmatter.com/podcast/agile-testing/david-evans-gojko-adzic-interactive-agile-acceptance-testing-clinic