Automating web tests with FitNesse and Selenium

By
gojko
– May 20, 2007Posted in: articles, fitnesse

UPDATE: My thinking on this has changed significantly in the years following this post, but according to Google it’s still quite popular. If you’re interested in combining FitNesse and Selenium, make sure to read this post as well: How to implement UI Testing without shooting yourself in the foot. It explains how to avoid some of the most common problems.

Web user interfaces have traditionally been hard to integrate into an automated test process. Selenium+FitNesse combination, with just a bit of coding, solves this task incredibly well.

Selenium is a free browser automation and testing library, written by folks at ThoughtWorks. It can simulate text input, mouse events and execute various tests on page content. It’s written in JavaScript, and is compatible with all major browsers and platforms.

FitNesse is an open-source test and collaboration server, based on the Framework for integrated tests (FIT), and supports testing Java, .Net, Python and even some other code. I think that it is a good choice for the second side of the web UI testing coin, because it enables tests to be written almost like in English language. As the UI is very close to clients’ eyes, tests can and should be written so that clients can verify them (and if you are really lucky, even help with writing and maintaining the tests).


Installing Selenium

The core of Selenium is written in JavaScript, and is compatible with all major browsers. The Selenium Remote Console provides the glue between a test browser and .Net, Java or Python code, and enables us to write and run Selenium tests from almost all popular test frameworks.

Just download Selenium Remote Console from http://www.openqa.org/selenium-rc. Unpack the files somewhere on your disk, and then start the server from the server folder of the package, by executing java -jar selenium-server.jar. The server starts on port 4444 by default, if that port is taken you can change it by adding -port number to the command line.

Selenium works by embedding the test site, along with some control scripts, in a frameset web page. The control scripts are then used to automate the test page in the main frame, and execute various tests. That method of work will clash with browser security rules if both frames do not come from the same domain. Remote console can open Internet Explorer and Firefox with a security tweak to allow cross-domain scripting, but if you don’t want that, or want to use a different browser, then put Selenium JavaScript files into your web site. Download them from http://www.openqa.org/selenium-core/, and put the files from core folder in the archive into the selenium-server folder of your web site (so that RemoteRunner.html is available on /selenium-server/RemoteRunner.html).

A quick test

Let’s first do a quick test to make sure that Selenium RC is running correctly. We’ll start a browser window, go to Google, enter a search phrase and click on the Search button. I’ll do the examples in C#, but you can also use Java – interfaces are very similar. Create a new .Net project, add thoughtworks.selenium.core.dll as a reference (it’s in the dotnet folder of Remote Control installation), and then create this class:

using System;
using Selenium;
namespace SeleniumTest
{
    class Console
    {
        static void Main(string[] args)
        {
            ISelenium sel = new DefaultSelenium(
              "localhost", 4444, "*iehta", "http://www.google.com");
            sel.Start();
            sel.Open("http://www.google.com/");
            sel.Type("q", "FitNesse");
            sel.Click("btnG");
            sel.WaitForPageToLoad("3000");
        }
    }
}

Make sure that Selenium RC is running, close all open Internet explorer windows, and then execute the program. Selenium RC will open a new IE window (it may be minimised on start, but you will see a new button in the task bar), and then go to Google and execute a search.



click for full-size image

Selenium window has three frames. Top-left enables the user to display the log and debug the execution by examining the DOM tree. Top-right frame displays the last four executed Selenium commands, which comes useful when troubleshooting tests. The page under test is displayed in the central frame.

The first line initialises a selenium console instance, connecting it to a remote console on localhost port 4444. 

ISelenium sel = new DefaultSelenium(
"localhost", 4444, "*iehta", "http://www.google.com");

The third argument is a browser string, and *iehta is the code for security-tweaked Internet Explorer which allows cross-domain scripting. Use *chrome for firefox with cross-domain scripting, or *iexplore, *firefox and *opera for Internet Explorer, Firefox and Opera without cross-domain security tweaks. You can also specify a full path to the executable instead of these special keywords.

The fourth argument is the URL to the selenium test site, and is really important only if you do not use cross-domain scripting, but want Selenium to run the scripts from the local domain. In this case, it will try to get the remote runner from http://www.google.com/selenium-server/RemoteRunner.html, but since Google is not hosting a Selenium installation, it will run the local scripts instead. In any case, this argument needs to be specified, so just set it to some live site even if you use cross-domain scripting (like in this case).

As you can guess from the names, method Type will simulate entering data from keyboard into a text field, and Click will simulate a mouse click. We accessed the text field and the button by their names, q and btnG. Selenium can also find elements by their ID (prefix it with “identifier=”), XPath expression (prefix it with “xpath=”), DOM path or a CSS property. See http://www.openqa.org/selenium-core/reference.html for more information on locators.

Connecting from FitNesse

FitNesse is really great for automating acceptance tests, and if you have not had a chance to play with it yet, I suggest reading through the online user guide for testing Java code, or my tutorial Getting fit with .Net for working in the .Net environment.

Instead of writing a new fixture type for every page or web site, we use a generic WebTest fixture. It extends DoFixture, so that other fixtures can be easily embedded into it (for checking the back-end data after a web script, or preparing the stage for the web test). Web test fixture allows the scripts to be described almost like English prose, like this:

User opens URL http://localhost:7711/login.aspx
User types testuser into username field
User types testpassword into password field
User clicks on Log In
Page reloads in less than 3 seconds
Page contains text You have logged in

To achieve the best effect, the script must relate to what users see on the screen. However, that is easier said than done. Text fields, check boxes and buttons are all instances of the “input” element, and although text fields are mostly distinguished by name, buttons names are typically not important, and users only see the “value” attribute of a button. WebTest fixture looks for various combinations of attributes: for example, when searching for a button, it first looks for an input element with type equal to submit or button, and the name attribute matching the query. If no such element is found, it looks for similar elements, with value attribute matching the query. On the end, it looks for a button with matching ID. All these combinations are listed in the buttonLocators array in the code:

 public static readonly string[] buttonLocators = new String[] {
	"xpath=//input[@type='submit' and @name='{0}']",
	"xpath=//input[@type='button' and @name='{0}']",
	"xpath=//input[@type='submit' and @value='{0}']",
	"xpath=//input[@type='button' and @value='{0}']",
	"xpath=//input[@type='submit' and @id='{0}']",
	"xpath=//input[@type='button' and @id='{0}']"};

Although looking for an element by ID or name would be quicker, we intentionally use Xpath and add extra information, to catch errors caused by wrong element types. GetLocator method checks a list of XPath expressions and returns the first matching element from the active document, so UserClicksOn method is very simple:

public void UserClicksOn(String buttonCaption){
            instance.Click(GetLocator(buttonCaption, buttonLocators));
}

We use the same technique to put text into text fields, areas and password fields. An array of Xpath expressions is used to check for various combinations of valid element types and attribute values, and the first matching element is then passed to Selenium Type method. 

public void UserTypesIntoField(String what, String where){
      instance.Type(GetLocator(where.Replace(" ", ""), textFieldLocators), what);
}

Because text fields will mostly be referenced by a name or ID, but users will see the text before the field on the page (and probably try to reference it with that), we also strip the blanks from the name. So the test script line:

User types 10109 into security number field

will correctly map into an element named “securitynumber”.

With web tests, we typically want to check if the result was correct and if the site was responsive enough, and Selenium method WaitForPageToLoad can be used to check if a page loads in any given amount of time. The API is a bit weird, as it expects a string containing the number of milliseconds. We wrapped that into this WebTest fixture call:

public void PageReloadsInLessThanSeconds(String sec){
            instance.WaitForPageToLoad(sec + "000");
}

On the end, we need a method to verify that page contents are correct. Selenium method IsTextPresent can help with that:

public bool PageContainsText(String s){
            return instance.IsTextPresent(s);
}

We’ll also need to open a browser and set up the Selenium environment from the test fixture. To keep the test script format in english-like prose, we’ll use the following syntax:

Start Browser *iehta with selenium console on localhost at port 4444 and scripts at http://localhost:7711

In WebTest fixture, that is covered by the following method:

public void StartBrowserWithSeleniumConsoleOnAtPortAndScriptsAt
   (String browser, String rcServer, int rcPort, String seleniumURL){
            instance = new DefaultSelenium(
                 rcServer, rcPort, browser, seleniumURL);
            instance.Start();
}

On the end of each test, we need to shut down the browser, so that remote console does not run out of resources. For that, we’ll use the following method:

public void ShutdownBrowser(){
            instance.Stop();
}

With all that, the web test can be easily described with this FitSesse script:


!|webfixture.WebTest|

!|Start Browser|*iehta|With Selenium Console On| localhost| At Port |4444|And Scripts At|http://localhost:7711|

|User Opens URL|http://localhost:7711/login.aspx|
|User types|testuser|into|username|field|
|User types|testpassword|into|password|field|

|User clicks on|Log In|
|Page reloads in less than|3|seconds|
|Page contains text|You have logged in|

|Shutdown browser|


click for full-size image

Ajax testing

With web 2.0 pages, waiting for the page to reload before we continue testing is not the best choice. Selenium also supports
waiting for a JavaScript condition. Remember that Selenium script is being executed in a different frame, so referencing DOM elements and JS functions in your test site page will not work straight away. Prefix the references with
selenium.browserbot.getCurrentWindow() to get from the Selenium frame to your test page. Here’s an example: 

instance.WaitForCondition(
  "(selenium.browserbot.getCurrentWindow().get_username()!=null)",
  timeout);

Remote execution

Selenium Remote Console will try to open a new browser session with different security settings, but under the profile of the current user (in fact, the user which started the remote console). This may cause problems when you already have an open browser on the same machine. The workaround is to use a different browser for testing. For example, I use Firefox for normal browsing, so RC can start IE and play with it. But, there is a much better solution.

As the name suggests, Remote Console can run on a dedicated test server, and be accessed remotely. Any browsers that RC opens will run on that machine. That enables developers to work with browsers on their machines and run the tests, without clashing with test execution. It also enables us to re-use the same FitNesse scripts to check how various browsers are behaving, even on different platforms. Just separate the initialisation and browser shut-down into SetUp and TearDown scripts, and then create a couple of different test suites with different SetUps which will use various browsers and remote console installations. Then, use symbolic links to import scripts from one test suite into all others. You will maintain scripts in one place, and they will test multiple browsers and platforms.

Extending WebTest

WebTest currently supports two more interesting methods: PageURLIs(String s) and UserSelectsFrom(String what, String where). The first allows you to verify the page URL (and test redirections). The second automates selecting an option from a drop-down menu (HTML select element), in the form of User Selects NY from Countries.

Feel free to modify and adjust WebTest to your needs. For start, see dotnet/doc/index.html in your remote control installation for more information on ISelenium interface. You can a add search by ID to the top of all locator arrays, so that tests run faster, but that will make them more error-prone. You will probably want to add a few different expressions to the textbox, button or select lists. If you add something which may be useful to others, please share it by writing a comment on the WebTest fixture page.

Photo credits:Sanja Gjenero/SXC

Tags: , , , , , ,