Introduction to BDD is an intensive one day workshop which introduces behaviour-driven development to developers, business analysts and testers. The optional programming module is offered in Java or .NET, with Cucumber to automate BDD scenarios.

Hands-on BDD with Cucumber is a three day workshop which immerses the participants into a project driven by Specification by Example and Behaviour-Driven Development. The workshop is adjusted to fit your business domain and particular needs, so that the participants get real-world experience and instant benefits. It is combines the Introduction to BDD, a day of working on a realistic domain example taken from your recent project or a future phase of a project, and a day of programming exercises for test automation developers. During the workshop, we use Cucumber to manage BDD Scenarios. The programming modules can be either in Java or .NET.

These two workshops are offered at a 20% discount for all bookings made before 31st January 2010. For more information on these and my other workshops, check out the training page.

Both workshops are offered as on-site training only (there are no public workshops scheduled at this time). Contact my company, Neuri Ltd for more information and to book a workshop.

Google had a team of Test Mercenaries, who joined different teams for a short period of time to help them with testing. In most cases, they could see what was wrong after a few days and started helping the teams, but the effort wasn’t a success. When they left, teams would not improve significantly. Striebeck said that testing “wasn’t effective to teach”. Knowing what makes a good test often relied on personal opinion and gut-feel. Doing what they often do in similar situations, Striebeck said that they decided to collect data. The things that they were interested in were figuring out the characteristics of good tests and testable code and how to know in the first place that a test is effective. They decided to use a return-on-investment criteria: low investment (easy to write, easy to maintain), high return (alert to problems when it fails). According to Striebeck, Google spends $100M per year on test automation, and wanted an answer whether they are actually getting a good return on that investment. They estimated that a bug found during TDD costs $5 to fix, which surges to $50 for tests during a full build and $500 during an integration test. It goes to $5000 during a system test. Fixing bugs earlier would save them an estimated $160M per year.

To collect data, they set up a code-metrics storage to put all test execution analytics in a single place. Striebeck pointed out that Google has a single code repository, which is completely open to all of the 10000 developers. Although all systems are released independently (with release release cycles randing from a week to a month), everything is built from HEAD without any binary releases, and the repository receives several thousand changes per day with spikes of 20+ changes per minute. This resulted in 40+ millions of tests executed per day from a continuous integration service plus IDE and command line runs, they collected test results, coverage, buld time, binary size, static analysis and complexity analysis. Instead of anyone deciding whether a test is good or not, the system observed what people do with tests to rank them. They looked into what a developer does after a test fails. If the code was changed or added, the test was marked as good. If people changes the test code when it fails, it was marked as a bad test (especially if everyone is changing it). This means that the test was brittle and has a high maintenance cost. They also measured which tests are ignored in releases and which tests often failed inthe continuous build and weren’t executed during development.

The first step was to provide developers reactive feedback on tests. For example, the system suggested deleting tests that teams spent loads of time maintaining. They then collected metrics on whether the people actually acted on suggestions or not. The system also provided metrics to tech leads and managers to show how teams are doing with tests.

The second step, which is in progress at the moment, is to find patterns and indicators. As they now have identified lots of good and bad tests, the system is looking for common characteristics among them. Once these patterns are collected, algorithms will be designed to identify good and bad tests, and manually calibrated by experts.

The third step will be to provide constructive feedback to developers, telling developers how to improve tests, what tests to write an dhow to make the code more testable.

The fourth step in this effort will be to provide prognostic feedback, analysing code evolution patterns and warn developers that their change might result in a particular problem later on.

I will be covering XpDay 2009 on this blog in detail. Subscribe to my RSS feed to get notified when I post new articles.

What’s new in this version?

Since the book was originally released, both FitNesse and the .NET FIT test runner were improved significantly. All the examples in this book are now updated to be compatible with the latest releases of FitNesse (20091121) and FitSharp (1.4). I re-wrote parts that are no longer applicable to the new FitSharp test runner, especially around Cell Operators. In a classic example of self-inflicted scope creep, I also wrote a new chapter on using domain objects directly.

I also changed the tool used for assembling the book. Instead of Apache FOP, I used XEP which will hopefully make the layout a bit better. Fonts (especially the code font) were also changed to make the book easier to read.

What about the paperback

I will make the paperback available soon. At the moment, the second edition is only available online.

Given the authors’ backgrounds, it’s not surprising that this book has a lot to say about using mock object libraries. Mock objects are arguably the most misunderstood and misused concept in software development today, so this book should be a valuable resource for most software development teams. In the part dealing with mock objects you will find strategies for using them successfully for software design, guidelines what to mock and what not to mock and lots of examples how all that looks in code.

The book isn’t written in the usual imperative way (“you should use this because of…”) but reads much more as an experience report (“we use this because of”). This might be unusual at first but I really like it, as it puts the things into a much more different perspective. Many of the topics addressed by this book are quite controversial and the authors have wisely chosen the voice to avoid any notion of preaching. I found myself disagreeing with parts, especially around bundling acceptance and end-to-end testing together. However, as the material doesn’t preach but tell what the authors are thinking about, this did not bother me at all.

All in all, an excellent book. Grab a copy from Amazon now.

Here are some related links:

• notes from Steve Freeman’s presentation on mock objects at CITCON Europe 09
• book web site
• Steve Freeman’s blog
• Nat Pryce’s blog

First of all, get mockito from http://mockito.org/. Almost everything really interesting can be imported with the org.mockito.Mockito class (or a static import of its methods, which I’ll use in this post). So let’s get right into it.

To create a stub (or a mock), use mock(class). Then use when(mock).thenReturn(value) to specify the stub value for a method. If you specify more than one value, they will be returned in sequence until the last one is used, after which point the last specified value gets returned. (So to have a method return the same value always, just specify it once). For example:

import static org.mockito.Mockito.*;
import static org.junit.Assert.*;
import java.util.Iterator;
import org.junit.Test;
....
	@Test
	public void iterator_will_return_hello_world(){
		//arrange
		Iterator i=mock(Iterator.class);
		when(i.next()).thenReturn("Hello").thenReturn("World");
		//act
		String result=i.next()+" "+i.next();
		//assert
		assertEquals("Hello World", result);
	}

This example creates a mock iterator and makes it return “Hello” the first time method next() is called. Calls after that return “World”. Then we can run normal assertions.

Stubs can also return different values depending on arguments passed into the method. For example:

	@Test
	public void with_arguments(){
		Comparable c=mock(Comparable.class);
		when(c.compareTo("Test")).thenReturn(1);
		assertEquals(1,c.compareTo("Test"));
	}

This creates a stub Comparable object and returns 1 if it is compared to a particular String value (“Test” in this case). If the method has arguments but you really don’t care what gets passed or cannot predict it, use anyInt() (and alternative values for other types). For example:

	@Test
	public void with_unspecified_arguments(){
		Comparable c=mock(Comparable.class);
		when(c.compareTo(anyInt())).thenReturn(-1);
		assertEquals(-1,c.compareTo(5));
	}

This stub comparable returns -1 regardless of the actual method argument. With void methods, this gets a bit tricky as you can’t use them in the when() call. The alternative syntax is doReturn(result).when(mock_object).void_method_call(); Instead of returning, you can also use .thenThrow() or doThrow() for void methods. For example:

	@Test(expected=IOException.class)
	public void OutputStreamWriter_rethrows_an_exception_from_OutputStream()
		throws IOException{
		OutputStream mock=mock(OutputStream.class);
		OutputStreamWriter osw=new OutputStreamWriter(mock);
		doThrow(new IOException()).when(mock).close();
		osw.close();
	}

This example throws an IOException when the mock OutputStream close method is called. We verify easily that the OutputStreamWriter rethrows the exception of the wrapped output stream. To verify actual calls to underlying objects (typical mock object usage), we can use verify(mock_object).method_call; For example:

	@Test
	public void OutputStreamWriter_Closes_OutputStream_on_Close()
		 throws IOException{
		OutputStream mock=mock(OutputStream.class);
		OutputStreamWriter osw=new OutputStreamWriter(mock);
		osw.close();
		verify(mock).close();
	}

This example will verify that OutputStreamWriter propagates the close method call to the wrapped output stream. You can use arguments on methods and matchers such as anyInt() similar to the previous example. Note that you can’t mix literals and matchers, so if you have multiple arguments they all have to be either literals or matchers. use eq(value) matcher to convert a literal into a matcher that compares on value. Mockito comes with lots of matchers already built in, but sometimes you need a bit more flexibility. For example, OutputStreamWriter will buffer output and then send it to the wrapped object when flushed, but we don’t know how big the buffer is upfront. So we can’t use equality matching. However, we can supply our own matcher:

	@Test
	public void OutputStreamWriter_Buffers_And_Forwards_To_OutputStream()
		throws IOException{
		OutputStream mock=mock(OutputStream.class);
		OutputStreamWriter osw=new OutputStreamWriter(mock);
		osw.write('a');
		osw.flush();
		// can't do this as we don't know how long the array is going to be
		// verify(mock).write(new byte[]{'a'},0,1);

		BaseMatcher arrayStartingWithA=new BaseMatcher(){
			@Override
			public void describeTo(Description description) {
				// nothing
			}
			// check that first character is A
			@Override
			public boolean matches(Object item) {
				byte[] actual=(byte[]) item;
				return actual[0]=='a';
			}
		};
		// check that first character of the array is A, and that the other two arguments are 0 and 1
		verify(mock).write(argThat(arrayStartingWithA), eq(0),eq(1));
	}

That’s it – all you need to know to get started. Now go forth and refactor all that easymock ugliness from your projects.