Ford suggested going outside the box and looking at writing unit tests in other JVM languages even if the target language is Java. “Always test a weaker language with a stronger one”, said Ford, suggesting that people should use Groovy to write Java unit tests. According to him, Groovy makes writing unit tests easier because it ignores privates and exceptions. Another option is JRuby. “Ruby guys have the coolest testing tools in the world”, said Ford. Java developers can use those tools with the help of JTestR, which makes it easy to write Java unit tests in JRuby and use standard Ruby testing tools for Java projects.

For teams working with J2EE technologies, he suggested using Mockrunner, which “stubs out J2EE by magic”, according to Ford. Mockrunner has stubs that provide out of the box functionality for JMS, JDBC, Servlet and many other J2EE APIs. It should allow unit testing of code that depends on J2EE APIs outside a container.

For teams that work with the Spring stack and related APIs, he suggested using Unitils, a “swiss army chainsaw of useful stuff”. It supports stubbing out Hibernate, Spring, JPA and similar APIs to speed up unit testing without having to really load the spring context or work against a database

The main author of this e-book is my colleague David de Florinier, and I’ve also contributed a bit.

The first version covers the basics of Cucumber, setting up for Ruby, Java and .NET projects, implementing basic features in all three languages, effective test design and managing complex features efficiently. We plan to develop this guide incrementally, and the next version will include web browser automation topics.

To download the PDF or read the e-Book online, point your browser to cuke4ninja.com. For news and updates, follow cuke4ninja.

First of all, get mockito from http://mockito.org/. Almost everything really interesting can be imported with the org.mockito.Mockito class (or a static import of its methods, which I’ll use in this post). So let’s get right into it.

To create a stub (or a mock), use mock(class). Then use when(mock).thenReturn(value) to specify the stub value for a method. If you specify more than one value, they will be returned in sequence until the last one is used, after which point the last specified value gets returned. (So to have a method return the same value always, just specify it once). For example:

import static org.mockito.Mockito.*;
import static org.junit.Assert.*;
import java.util.Iterator;
import org.junit.Test;
....
	@Test
	public void iterator_will_return_hello_world(){
		//arrange
		Iterator i=mock(Iterator.class);
		when(i.next()).thenReturn("Hello").thenReturn("World");
		//act
		String result=i.next()+" "+i.next();
		//assert
		assertEquals("Hello World", result);
	}

This example creates a mock iterator and makes it return “Hello” the first time method next() is called. Calls after that return “World”. Then we can run normal assertions.

Stubs can also return different values depending on arguments passed into the method. For example:

	@Test
	public void with_arguments(){
		Comparable c=mock(Comparable.class);
		when(c.compareTo("Test")).thenReturn(1);
		assertEquals(1,c.compareTo("Test"));
	}

This creates a stub Comparable object and returns 1 if it is compared to a particular String value (“Test” in this case). If the method has arguments but you really don’t care what gets passed or cannot predict it, use anyInt() (and alternative values for other types). For example:

	@Test
	public void with_unspecified_arguments(){
		Comparable c=mock(Comparable.class);
		when(c.compareTo(anyInt())).thenReturn(-1);
		assertEquals(-1,c.compareTo(5));
	}

This stub comparable returns -1 regardless of the actual method argument. With void methods, this gets a bit tricky as you can’t use them in the when() call. The alternative syntax is doReturn(result).when(mock_object).void_method_call(); Instead of returning, you can also use .thenThrow() or doThrow() for void methods. For example:

	@Test(expected=IOException.class)
	public void OutputStreamWriter_rethrows_an_exception_from_OutputStream()
		throws IOException{
		OutputStream mock=mock(OutputStream.class);
		OutputStreamWriter osw=new OutputStreamWriter(mock);
		doThrow(new IOException()).when(mock).close();
		osw.close();
	}

This example throws an IOException when the mock OutputStream close method is called. We verify easily that the OutputStreamWriter rethrows the exception of the wrapped output stream. To verify actual calls to underlying objects (typical mock object usage), we can use verify(mock_object).method_call; For example:

	@Test
	public void OutputStreamWriter_Closes_OutputStream_on_Close()
		 throws IOException{
		OutputStream mock=mock(OutputStream.class);
		OutputStreamWriter osw=new OutputStreamWriter(mock);
		osw.close();
		verify(mock).close();
	}

This example will verify that OutputStreamWriter propagates the close method call to the wrapped output stream. You can use arguments on methods and matchers such as anyInt() similar to the previous example. Note that you can’t mix literals and matchers, so if you have multiple arguments they all have to be either literals or matchers. use eq(value) matcher to convert a literal into a matcher that compares on value. Mockito comes with lots of matchers already built in, but sometimes you need a bit more flexibility. For example, OutputStreamWriter will buffer output and then send it to the wrapped object when flushed, but we don’t know how big the buffer is upfront. So we can’t use equality matching. However, we can supply our own matcher:

	@Test
	public void OutputStreamWriter_Buffers_And_Forwards_To_OutputStream()
		throws IOException{
		OutputStream mock=mock(OutputStream.class);
		OutputStreamWriter osw=new OutputStreamWriter(mock);
		osw.write('a');
		osw.flush();
		// can't do this as we don't know how long the array is going to be
		// verify(mock).write(new byte[]{'a'},0,1);

		BaseMatcher arrayStartingWithA=new BaseMatcher(){
			@Override
			public void describeTo(Description description) {
				// nothing
			}
			// check that first character is A
			@Override
			public boolean matches(Object item) {
				byte[] actual=(byte[]) item;
				return actual[0]=='a';
			}
		};
		// check that first character of the array is A, and that the other two arguments are 0 and 1
		verify(mock).write(argThat(arrayStartingWithA), eq(0),eq(1));
	}

That’s it – all you need to know to get started. Now go forth and refactor all that easymock ugliness from your projects.

• run fitnesse fit and slim tests without starting the server as part of your build
• run fitnesse tests from JUnit within your IDE
• debug and troubleshoot fitnesse fixtures as if you were working with unit tests
• easily include fitnesse acceptance tests into your build (through JUnit)
• easily run tests in transactions and roll back after each test (with Spring)

For example, here is how you can run an entire FitNesse test suite from JUnit

public class JUnitExampleFitTest {
	JUnitHelper helper;
	@Before
	public void initHelper() throws Exception{
		helper=new JUnitHelper(new TestRunner(
				new FitNesseRepository("target/test-classes"),
				new FitTestEngine(),
				"/tmp/test-output"));
	}
	@Test
	public void runSuite() throws Exception{
		helper.assertSuitePasses("JavaExamples");
	}
	@Test
	public void runSingleTest() throws Exception{
		helper.assertTestPasses(
                   "JavaExamples.CommonExamples.SetUpFixture");
	}
}

At the moment, Trinidad only supports Java FIT/Slim, but .NET is in plan soon. For download links, usage instructions and examples, see http://fitnesse.info/trinidad.

Concordion is really simple. Its instrumentation only allows programmers to set global test variables, execute fixture methods and compare actual results with expected values. Programmers can use special HTML element attributes to mark words or phrases that are used as test inputs or compared to test results. Web browsers will just ignore unknown element attributes, so Concordion test instrumentation is effectively invisible to people that are not interested in test automation. For example, here is a HTML document that I’ve used as a simple test:

<html xmlns:concordion="http://www.concordion.org/2007/concordion">
<body>
<h1>Free delivery</h1>
<ul>

 <li concordion:execute="#offer = checkFreeDelivery(#type, #books)">
 Free delivery <span concordion:assertEquals="#offer">is</span>
 offered to a <b concordion:set="#type">VIP</b>
 customer with <b concordion:set="#books">10</b> books in the cart.
 </li>

 <li concordion:execute="#offer = checkFreeDelivery(#type, #books)">
  It <b concordion:assertEquals="#offer">is not</b> offered to
  a <b concordion:set="#type">regular</b> customer
  with <b concordion:set="#books">10</b> books</li>

 <li concordion:execute="#offer = checkFreeDelivery(#type, #books)">
  It <b concordion:assertEquals="#offer">is not</b> offered to
  a <b concordion:set="#type">VIP</b> customer with
  only <b concordion:set="#books">9</b> books.</li>

</ul>
</body>
</html>

The concordion:execute command on the LI element specifies that the offer variable is set by executing the checkFreeDelivery method and passing type and books variables set in the element text. The concordion:set command is used to set a variable based on the inner text in the element. The concordion:assertEquals command inspects the inner text in the element and compares that with the result of a method or current variable value. For repetitive specifications and calculation rules, Concordion also supports attributes for tables similar to the FIT ColumnFixture.

At the moment, Concordion only supports Java fixtures — It actually works as a JUnit extension. This provides direct integration with JUnit test runners, making it much easier to execute Concordion tests from popular development environments and integrate into continuous build systems. It also shortens the learning curve required to start using Concordion. The fixture is simply a JUnit test class. It should be called the same as the HTML file (in this case, the file was ConcordionTest.html, so the class is ConcordionTest.java), and declare the methods that are used by concordion:execute.

package conctest;

import org.concordion.integration.junit4.ConcordionRunner;
import org.junit.runner.RunWith;

@RunWith(ConcordionRunner.class)

public class ConcordionTest {
	public boolean freeDelivery(String type, int books){
			if (books<9) return false;
			if (!"VIP".equals(type)) return false;
			return true;
	}
	public String checkFreeDelivery(String type, int books){
		return freeDelivery(type, books)?"is":"is not";
	}
}

My first impression of this is that the fixture model of Concordion does not depend on inheritance so it is a lot simpler to learn than FIT and somewhat more flexible to write Concordion than FIT fixtures. On the other hand, Concordion lacks the extensibility and powerful model of type adapters and cell handlers that enable FIT to bind domain objects and business services directly to the specification.

When the JUnit test is executed, Concordion runs through the files and executes commands, verifying expected outputs with actual results in assertEquals. JUnit test run will tell you whether all tests passed or were there failures, and Concordion also saves the results in the system temporary folder in HTML form, making it easier to see what actually went wrong. The example above intentionally has an error — here is the screenshot of the result:

Concordion does not have a test management tool and relies completely on the programmer’s IDE to manipulate and execute tests. I have mixed feelings about this. Although file management within an IDE makes it much easier to put tests in the same version control system as the domain code, I miss the ability to re-use parts of the test specification such as common set-ups or test components with macro variables that are available in FitNesse. It also puts tests completely under the control of programmers.

Concordion takes some ideas that have evolved as best practices to use FIT/FitNesse and makes them very explicit, actively discouraging other ways of working. For example, acceptance tests have to be stored in the same version control system as the code, there are no pre-built test building blocks that would encourage scripting. On the other hand, while FIT/FitNesse community is moving firmly towards using domain objects directly in acceptance tests and reducing the amount of glue code, to eliminate translation between layers and promote domain-driven design, Concordion model is stuck in requiring you to create an explicit test method for every verification in the fixture.

The thing that really worries me is the fact that HTML files are stored next to Java classes and that developers need to add non-standard HTML attributes to the test page. This smells to me of a hand-over of tests from business people to developers at some point, which is a practice that I don’t approve at all. In my opinion, tests should be shared between the whole team and not handed-over down the pipeline. I guess that this can be avoided with some discipline and research into tools that will surely not overwrite non-standard HTML attributes. But I would still like to see a proper test management tool for business people to use.

I’ve been hearing a lot about Concordion recently, mostly being mentioned by people as an interesting tool to look into. Safari has no books about it, all I could find on Google and Technorati blog searches are reports of people coming across the tool and doing some basic things with it, such as this post. I would be really interested in hearing stories from people that have real-world experiences with this tool to share. What happens six months later — are tests easier to manage than with FitNesse, has it turned out to be good enough tool to talk to business people? Is my fear of hand-over real, or is it not a problem at all?