Gojko Adzic

Declarative transactions in Spring and Hibernate make programming enterprise applications much easier, but they also make integrated database tests a bit tricky. I’ve finally found good solution for this problem — it is a bit dirty, but very effective.

The problem

The issue with declarative transactions is that integration tests need to make calls into the service layer, which is below the transaction boundary, so when the execution goes back above the transaction boundary data changes become persistent. Then unique constraints kick in, making tests non-repeatable. If several developers run tests at the same time, they might obstruct each other, so a test failure might not really mean that there is a bug in the code. All that is just slowing down testing and making test maintenance harder.

Ideally, the database transaction should isolate us from other tests running at the same time, and just rolling back that transaction should make tests automatically repeatable. However, due to way that FIT loads and executes fixtures, they will by default be outside the transaction boundary. Pushing them below that boundary turned out to be quite a challenge.

I’ve seen a few teams that tried to attack this problem by doing some sort of database reset or clean-up to bring the database into a known state. That slows tests down unnecessarily and still does not solve the problem of concurrent test runs. In my opinion, it makes it even worse — since your data might disappear without any explanation.

Individual fixtures or their methods can be made transactional with some additional code, but this becomes painful to write and maintain, and removes some flexibility from FIT and FitNesse. That approach requires specific fixture types to be written just for transactional management and directly cancels all the benefits of FitLibrary SUT domain object wrapping. Extra code has to be added to fixtures, and they have to be injected into the session context, which makes the effort error prone and terribly ugly. And it also does not work if the page is not in flow mode, since individual fixtures will be instantiated by FIT in separate method calls from the static context, crossing the transaction boundary. Even in flow mode, you still have to take care not to go out of the transaction boundary — the fixture constructor will be, for example, outside the transaction scope. I had a gut-feel that there was a much better solution, but I simply could not see it.

A better way

Yesterday, it finally hit me — a totally non-intrusive solution that automatically works for all fixtures. There is no need to replicate transaction management code all over the test suites. No special fixture code is required in fixtures or services, and it does not matter whether the page is in flow mode or works with standalone fixtures. No special SetUp or TearDown pages.

The trick is based on the fact that FitNesse does not execute FIT in-process, but as an external program in order to allow test runners for languages like C# and Python to be plugged in. Test runner is defined by a simple parameter, which can be overriden for a single test page, individual test suite or for the whole server globally. We can use this feature to supply a different Java test runner, which will be injected into the Spring context straight from the start. We can mark the entry point as @Transactional, so no matter how many fixtures are instantiated and how ever they use the service layer, the calls will never cross the transaction boundary and require the transaction to commit. On the end, the test runner just has to roll back the active transaction and everything that the test changed will simply disappear.

You can download the code from here — but you will need to tweak it just a bit depending on your configuration (will be explained below). Now, for the interesting part — how the thing actually works:

Rolling back transactions

The first part of the puzzle is to create a Spring bean that will run under the control of the transaction manager, execute the appropriate command, and then roll back the transaction. Java SDK has a standard interface for the command pattern, called Runnable, so we will use it:

public class RollbackNow extends RuntimeException
{ }
public class RollbackBean{
  @Transactional
  public void process(Runnable r){
    r.run();
    throw new RollbackNow();
  }
}

The command processor, that will use this to inject Fixtures under the transaction boundary, will just have to catch the RollbackNow exception and ignore it. Because a runtime exception crosses the transaction boundary, Spring will automatically roll back the transaction. This is a bit ugly, because exceptions are used for flow control, but it is the cleanest way I found to do it — if you know a better one, please let me know. I have initially tried to fetch the current transaction from the transaction manager and roll it back in the code, but then I got “Unexpected Rollback” exceptions after the processing. Marking the transaction as rollbackOnly also did not help. In any case, this approach works ok. This ugliness is encapsulated and hidden from the users completely.

So this new Rollback bean needs to be registered in the spring testing context, for example under the name “rollback”.

Injecting the test runner

The second part of the puzzle is to change the test runner to inject tests into the rollback bean. FitServer, the class that runs the tests by default, is not designed to allow such injection easily. Putting the whole test runner in a Spring context does not do the trick because a single FitServer execution might process several tests (eg test suite), and each test should be executed in an isolated environment. When you look at the FitServer source code, the lines between 70 and 83 where the transaction boundary should be.

So the idea is to modify the FitServer code a bit, cut the part that does individual tests into a separate procedure, and inject that procedure into a transactional context. For the injection, we can use the standard Runnable interface and pass that on to a a simple Spring bean that runs under the control of Spring transaction management. Ideally, I’d do this by extending FitServer and reusing most of the functionality, but because of private variables used in the inner part, that is not possible. That class is open-source, so I just copied the code, renamed it and modified it a bit.

A new class implements the Runnable interface and takes over processing of a single document:

 public class DocumentRunner implements Runnable {
    private int size;
    public DocumentRunner(int size) {
      this.size = size;
    }
    public void run() {
      try {
        print("processing document of size: " + size + "\n");
        String document = FitProtocol.readDocument(socketReader, size);
        Parse tables = new Parse(document);
        newFixture().doTables(tables);
        print("\tresults: " + fixture.counts() + "\n");
        counts.tally(fixture.counts);
      } catch (Exception e) {
        exception(e);
      }
    }
  }

The only thing left to do is to change the process method in the FIT Server, so that it uses the RollbackBean to run individual documents under a transaction, and ignores the RollbackNow exception. Change the start of the method to load your context file and bean.

 public void process() {
    ApplicationContext ctx = new FileSystemXmlApplicationContext(
        "lib/test.xml");
    RollbackBean rollbackProcessingBean = (RollbackBean) ctx
        .getBean("rollback");
    fixture.listener = fixtureListener;
    try {
      int size = 1;
      while ((size = FitProtocol.readSize(socketReader)) != 0) {
        try {
          rollbackProcessingBean.process(new DocumentRunner(size));
        } catch (RollbackNow rn) {
          print("rolling back now" + "\n");
        }
      }
      print("completion signal recieved" + "\n");
    } catch (Exception e) {
      exception(e);
    }
  }

With this, everything works almost out-of-the-box. No changes are required in fixture code, nor in test pages, except the runner definition. In my example, the new runner class is test.RollbackServer, so my definition looks like this:

!define TEST_RUNNER {test.RollbackServer}

In fact, since the default FIT test runner is being used as a base for the new runner, there is no reason not to override the runner on the global level (setting the TEST_RUNNER environment property in run.sh/bat or in /root page) and have all tests on the FitNesse server automatically roll back.

Conclusions

I still have mixed feelings about this solution — I love it because it is very effective and does not require any changes in the fixture code or services, and it does not take any flexibility out of FIT and FitNesse. On the other hand, I’m really not happy about using exceptions to fool the transaction manager or the fact that I had to copy and modify the test runner instead of extending it. According to SVN logs, FitServer does get changed every couple of months, so theoretically the way this is currently implemented may require merging the changes with new versions in the future.

The fact that it magically pushes all tests under the transaction boundary and instantly makes data-driven tests repeatable should save us enough time to justify that merging every once in a while.

Just in case that you missed the note about code download — get the code from here.

Image credits: Steve Woods

This book takes you on a journey through the wonderful world of FitNesse, a great web-based collaboration tool for software acceptance testing. FitNesse enables software developers and business people to build a shared understanding of the domain and helps produce software that is genuinely fit for purpose. “This book fills a big gap that’s kept a lot of people from using Fit successfully.” – Mike Stockdale, author of FitNesse.NET More information | Sample chapter | Buy the book | Buy the PDF | Source code | Register your book

Get notified when I post something new - subscribe via RSS or Twitter!

Trackback URI | Comments RSS