Gojko Adzic » selenium

Putting Selenium in the right place

gojko — Tue, 06 Oct 2009 08:15:18 +0000

Although Selenium is an essential trace element, it is toxic if taken in excess. That is what Wikipedia has to say on the chemical element Selenium, but pretty much sums up my feelings about the web testing tool of the same name as well. I like very much how easy it is to implement web tests with Selenium, but I’ve seen so many teams shoot themselves in the foot by misusing it and wasting a ton of time on writing and executing tests that simply got thrown away on the end. The Page Object pattern, popularised by Simon Stewart with WebDriver, seems to be the universally accepted best practice to manage UI tests efficiently and the preferred way to implement Selenium tests. However, at the recent CITCON Europe conference in Paris, Antony Marcano spoke against this and offered an alternative.

One of the biggest issues with UI tests described at the task level (such as click there, type this etc) is that they are very brittle and hard to understand. Too many details in technical steps make it hard to see the big picture, so tests work nicely as a regression check but once something fails it is hard to figure out what exactly went wrong. They are also bound to a particular UI design, so when the design changes, lots of tests break. Selenium works on the technical task level and tests written for Selenium frequently fall into this black whole. Once the design is changed, you might as well do everything again from scratch.

Instead of writing tests at the technical task level, the Page Object pattern suggests creating one more level of indirection and describing business functionality of pages with objects (eg search for customers, submit registration form) and then implementing individual methods of these objects with Selenium. Tests are then written with the page object methods, which make them much more understandable and easier to maintain. Once the UI changes, we can reimplement only the relevant methods of Page Objects making the effects of changes relatively contained. Tests themselves don’t need to change. Similar ideas are implemented by xUseCase recorders and Domain-Specific Testing Languages in Selenium (see Mickey Phoenix’s Agile 2008 presentation).

Marcano said that page objects focus on the wrong thing. Taking ideas from interaction design, he suggested that it is natural to think about functionality on three levels — goals, activities and tasks:

Goal: Buy a book
Activities: Add the following items to the cart,…
Tasks: click search, type in book name, click book image, click Add to cart…

When thinking about web site functionality, people don’t think about pages but about activities that span multiple pages. Focusing on goal oriented cross-page activities instead of page oriented functionality makes tests more natural, easier to understand and maintain. Instead of Page Objects, Marcano suggested implementing activities with Selenium and then composing tests from activities. One more significant benefit of this approach is that activities aren’t baked into the user interface, unlike objects representing web pages, so the same test definitions can be kept for testing below the UI with different task implementations.

An excellent agile testing experience report

gojko — Thu, 23 Jul 2009 07:24:43 +0000

The agile testing evening sessions at Skills Matter are getting better and better. I thoroughly enjoyed yesterday’s session with an experience report given by Nathan Bain and Anand Ramdeo. Anand and Nathan talked about their work at Global Radio focusing on how they implemented agile testing, and the topic turned out to be so thought-provoking that the presentation often turned into a free discussion and the event lasted twice longer than expected. In spite of that, some attendees complained that it was too short. Luckily the local pub is open till much later than Skills Matter so we continued the discussion there.

Anand focused on the process, and key take-away ideas from his presentation for me are:

Four-column scrum wall: they added a “done-done” step after “done”, making it very explicit that a task needs to get verified after it is thought to be finished. Testers owned this phase which gave them the responsibility to see it through, not necessarily perform it themselves. For tasks that are not easily verifiable through the user interface or things such as usability or design where testers were not qualified to assert completeness, they were still expected to chase someone else to verify that the task is done. This could involve a code review, another developer or a designer confirming that something is OK etc.
Requirements meetings: Product owner, technical owner and a tester met before a story went into development to discuss what the acceptance criteria is. This made at least three people capable of answering questions from developers and it cut down the need to document stuff. It also ensured that testers knew what they should expect and test on the end. This idea is going along the same lines as specification workshops but seems to be focused more on fleshing out key ideas rather than getting to the full specifications.
Two-step continuous integration: Their functional test suite took a few hours to run (as it was running through the UI), and running this as part of the build even in continuous integration would make the feedback too slow. On the other hand, these tests had to be executed. Anand suggested doing continuous integration in two steps. First build and run basic (quick) tests. When that succeeds, run the entire test suite. I would typically investigate whether these two hours could be cut down or automated at a lower level to make it quicker, but this requires development resources so Anand’s solution might be very interesting to people who don’t have the option or knowledge to automate below the UI.
Using named UI elements in Selenium scripts: to make UI tests easier to maintain and understand, they used macros to encapsulate locators for UI elements and gave them meaningful names such as “register button”. When the UI changes, they only update a macro, without the need to change tests. This also enabled them to write tests before the web UI was actually developed, and put in values for locators into macros later. This is again an idea similar to page objects but implemented completely from a tester’s perspective without the need to involve developers.
Web testing below the browser – they used a tool of which I haven’t heard before, called Twill. Twill automates web testing by executing HTTP requests but not through the browser, making it easier and faster to automate tests than with Selenium. It does not execute javascript so it cannot be applied to complicated browser-oriented tasks, but it is very useful for form submission, link parsing etc. With Python, they built a DSL for testing web pages using Twill. It seems like it could do a very good job for a large number of web tests that don’t involve JavaScript.

Nathan started with a great cartoon introduction to acceptance testing (he promised to upload the slides to nathanbain.com) and then went on to talk about what he would have liked to have known when he was starting with agile testing. Some of the key take-away points are:

The rules of agile testing are still being written, and lots of information is only available online on blogs and mailing lists – testers need to read and research more. Nathan also recommended Agile Testing by Lisa Crispin and Janet Gregory and my Bridging the Communication Gap as good sources of information on the current state of agile testing.
Although testers might find themselves in uncharted territories on an agile team, the team expects of them to be the testing experts so they have to find ways to help the team rather than waiting to be told what to do. Testers should be proactive and help define their role by looking at the ways they can improve process and product.
Promote proactive testing and defining acceptance criteria as tests in the team to raise awareness of what testing is about and get support and help from developers and business. Nathan organised workshops and presentations on acceptance testing to help other members of his team understand how he can help them better.
Getting more involved in the process is the key to success. Nathan suggested learning about the domain, working closely with developers, going to meetings even if you’re not invited and making sure that you are on all the mailing lists. Nathan warned that people who are outside the loop get left behind, and by being more involved testers can help the team more but also show the others their value better
Instead of writing down bug reports, discuss them – sometimes the bug turns out not to be a bug at all and the discussion helps to make discovering a real bug constructive rather than an act that conflicts with development. This discussion helps the morale on the team.
Recruiting casual testers elsewhere – try to get project managers, product owners, developers and anyone else involved with testing. Prepare test scripts as simple check-lists that they can follow. Describe manual tests in plain English so that you can get more people involved.
Using acceptance tests as a good start for exploration – instead of specifying everything in detail, specify the core as acceptance tests and then use them as a start for exploratory testing, sort of a reminder of where to begin and what to look for.

Altogether a very enjoyable evening. I really liked this and I think that we should have more experience reports like these. The event was filmed and I strongly recommend watching the videos (they should be available in a few days on the Skills Matter agile testing page (subscribe to my rss feed to get notified when they become available).

The next agile testing evening at Skills Matter will be on the 18th of August. David Evans and Mike Scott from SQS-UK will talk about Testify, a project that helps kick-start Test-Driven Development in .NET by installing and setting up all the basic tools that people typically need and creating a basic project structure that makes TDD easier. The event is free as usual, but up-front registration is required for capacity planning. You can find more information on this event and register at skillsmatter.com.

Introducing Alt.NET courses at Skills Matter

gojko — Mon, 22 Dec 2008 08:42:35 +0000

From February 2009, Skills Matter will start organising public Alt.NET courses (first in London and then across Europe).

Opensource .NET tools crash course

The first will be a three-day crash course on tools and practices aimed at .NET developers that want to learn about Alt.NET tools and Java developers that are migrating to .NET and looking for good equivalents to the tools that they are used to working with. The course gives an overview of the most popular opensource .net tools and introduces modern development practices that these tools promote, such as test driven development, continuous integration, dependency injection, object-relational mapping and web development using the model-view-controller pattern.

Learn how to:

Implement TDD in .NET using NUnit, MBUnit, Rhino Mocks and FitNesse
Utilise Aspect oriented programming and Dependency Injection using Castle Windsor
Efficiently build Web applications using the MVC pattern in Monorail and utilising Monorail and Script# for Ajax and test them using Selenium Remote Console.
Manage persistence easily using ORM tools such as ActiveRecord and NHibernate
Introduce continuous integration in your projects using CruiseControl.NET and CI Factory

See the full programme.

Agile Web Development with the Castle Framework

The second one is a two-day course on Agile Web Development using the Castle project, teaching the basics of the Castle Framework and helping people develop a solid understanding of its benefits. Over the course of the two days, attendees will create a simple but complete web application using agile Web development practices such as Inversion of Control, Dependency Injection, Aspect Oriented Programming, Object/Relational Mapping and applying the Model-View-Controller pattern.

Learn how to

Apply agile web development practices like MVC and dependency injection
Use ActiveRecord to manage the object-relational mapping and the database layer
Use the Monorail MVC engine to create web applications that are easy to maintain and test
Explain the basics of Monorail views, layouts, rescues
Use the NVelocity view engine to build web UIs for Monorail
Apply Windsor Microkernel to configure and wire application components
Unit test the data access layer with Castle
Unit test web controllers
Describe how Castle components come together to help us develop web applications easier
Explain why this approach is much more effective than ASP.NET
Apply best practices, common pitfalls, and tips and tricks for Castle Web development

See the full programme.

Video from the Selenium talk

gojko — Wed, 03 Sep 2008 19:22:15 +0000

Here is the video from the Testing Web Applications with Selenium and Selenium Remote Control talk that Milan Bogdanovic, Ivan Sanchez and I organised last week at Skills Matter. The first part of the talk introduces Selenium and some related tools that allow us to use Selenium easier. In the second part, Milan demonstrates Selenium IDE and talks about Selenese language. In the third part, Ivan talks about Remote Control, how to make tests easier to manage and introduced the Page Object pattern. Download links and slides from the talk.

Links and slides from “Testing Web Applications with Selenium”

gojko — Thu, 28 Aug 2008 22:34:53 +0000

Here are the links and slides from my talk today at Skills Matter on testing web applications with Selenium.

Download Powerpoint slides from here.

Selenium-core: selenium-core.openqa.org/
Selenium-Remote Control: selenium-rc.openqa.org/
Selenium-IDE: selenium-ide.openqa.org/
Selenium-Grid: selenium-grid.openqa.org/
Domain-specific testing languages presentation from Agile 2008 by Mickey Phoenix from Solutions IQ: www.solutionsiq.com/agile2008/agile-2008-domain.php
StoryTestIQ: storytestiq.solutionsiq.com
Cubic Test: www.cubictest.com
WebTest Fixtures: fitnesse.info/webtest

Ivan Sanchez will put his slides online at www.isanchez.net.