An excellent agile testing experience report

The agile testing evening sessions at Skills Matter are getting better and better. I thoroughly enjoyed yesterday’s session with an experience report given by Nathan Bain and Anand Ramdeo. Anand and Nathan talked about their work at Global Radio focusing on how they implemented agile testing, and the topic turned out to be so thought-provoking that the presentation often turned into a free discussion and the event lasted twice longer than expected. In spite of that, some attendees complained that it was too short. Luckily the local pub is open till much later than Skills Matter so we continued the discussion there.

Anand focused on the process, and key take-away ideas from his presentation for me are:

• Four-column scrum wall: they added a “done-done” step after “done”, making it very explicit that a task needs to get verified after it is thought to be finished. Testers owned this phase which gave them the responsibility to see it through, not necessarily perform it themselves. For tasks that are not easily verifiable through the user interface or things such as usability or design where testers were not qualified to assert completeness, they were still expected to chase someone else to verify that the task is done. This could involve a code review, another developer or a designer confirming that something is OK etc.
• Requirements meetings: Product owner, technical owner and a tester met before a story went into development to discuss what the acceptance criteria is. This made at least three people capable of answering questions from developers and it cut down the need to document stuff. It also ensured that testers knew what they should expect and test on the end. This idea is going along the same lines as specification workshops but seems to be focused more on fleshing out key ideas rather than getting to the full specifications.
• Two-step continuous integration: Their functional test suite took a few hours to run (as it was running through the UI), and running this as part of the build even in continuous integration would make the feedback too slow. On the other hand, these tests had to be executed. Anand suggested doing continuous integration in two steps. First build and run basic (quick) tests. When that succeeds, run the entire test suite. I would typically investigate whether these two hours could be cut down or automated at a lower level to make it quicker, but this requires development resources so Anand’s solution might be very interesting to people who don’t have the option or knowledge to automate below the UI.
• Using named UI elements in Selenium scripts: to make UI tests easier to maintain and understand, they used macros to encapsulate locators for UI elements and gave them meaningful names such as “register button”. When the UI changes, they only update a macro, without the need to change tests. This also enabled them to write tests before the web UI was actually developed, and put in values for locators into macros later. This is again an idea similar to page objects but implemented completely from a tester’s perspective without the need to involve developers.
• Web testing below the browser – they used a tool of which I haven’t heard before, called Twill. Twill automates web testing by executing HTTP requests but not through the browser, making it easier and faster to automate tests than with Selenium. It does not execute javascript so it cannot be applied to complicated browser-oriented tasks, but it is very useful for form submission, link parsing etc. With Python, they built a DSL for testing web pages using Twill. It seems like it could do a very good job for a large number of web tests that don’t involve JavaScript.

Nathan started with a great cartoon introduction to acceptance testing (he promised to upload the slides to nathanbain.com) and then went on to talk about what he would have liked to have known when he was starting with agile testing. Some of the key take-away points are:

• The rules of agile testing are still being written, and lots of information is only available online on blogs and mailing lists – testers need to read and research more. Nathan also recommended Agile Testing by Lisa Crispin and Janet Gregory and my Bridging the Communication Gap as good sources of information on the current state of agile testing.
• Although testers might find themselves in uncharted territories on an agile team, the team expects of them to be the testing experts so they have to find ways to help the team rather than waiting to be told what to do. Testers should be proactive and help define their role by looking at the ways they can improve process and product.
• Promote proactive testing and defining acceptance criteria as tests in the team to raise awareness of what testing is about and get support and help from developers and business. Nathan organised workshops and presentations on acceptance testing to help other members of his team understand how he can help them better.
• Getting more involved in the process is the key to success. Nathan suggested learning about the domain, working closely with developers, going to meetings even if you’re not invited and making sure that you are on all the mailing lists. Nathan warned that people who are outside the loop get left behind, and by being more involved testers can help the team more but also show the others their value better
• Instead of writing down bug reports, discuss them – sometimes the bug turns out not to be a bug at all and the discussion helps to make discovering a real bug constructive rather than an act that conflicts with development. This discussion helps the morale on the team.
• Recruiting casual testers elsewhere – try to get project managers, product owners, developers and anyone else involved with testing. Prepare test scripts as simple check-lists that they can follow. Describe manual tests in plain English so that you can get more people involved.
• Using acceptance tests as a good start for exploration – instead of specifying everything in detail, specify the core as acceptance tests and then use them as a start for exploratory testing, sort of a reminder of where to begin and what to look for.

Altogether a very enjoyable evening. I really liked this and I think that we should have more experience reports like these. The event was filmed and I strongly recommend watching the videos (they should be available in a few days on the Skills Matter agile testing page (subscribe to my rss feed to get notified when they become available).

The next agile testing evening at Skills Matter will be on the 18th of August. David Evans and Mike Scott from SQS-UK will talk about Testify, a project that helps kick-start Test-Driven Development in .NET by installing and setting up all the basic tools that people typically need and creating a basic project structure that makes TDD easier. The event is free as usual, but up-front registration is required for capacity planning. You can find more information on this event and register at skillsmatter.com.