Thoughtworks Studios were very kind to organise a Twist 2.0 demo for me this morning. I was very impressed with the new productivity features in the tool. Twist is the functional testing component of their lifecycle management tools, started with the idea to empower teams to manage complex test sets and promote collaboration in agile testing with non technical users.
As with many other agile acceptance testing tools out there, one of the core ideas of Twist is to separate the intention of the test from the implementation (automation). Tests are managed in a simple WYSIWYG editor (actually an application built on Eclipse RCP) and stored on the file system in a wiki-like format. The authors of this tool have done a good job of masking Eclipse into something that won’t scare the business users away. This enables users to keep tests and code in the same version control system easily (as you can use any SCM plugin for Eclipse) and also provides a good platform for intelligent test management functionality. For example, Twist hooks into Eclipse refactoring functions to support “rephrasing” (change the language in the test and it automatically propagates the change to other test scenarios and the code below it), code assistance (press control+space while editing a test to see what available options you have already automated) and generating stub automation code for unimplemented steps. This goes a long way to address one of the most common problems with acceptance testing tools today – easy test maintenance. Integrating with Eclipse gives developers the power to easily execute and debug tests from an IDE as well.
Other interesting features, added in version 2.0, include parameterised scenarios and abstract concepts. Parameterised scenarios are similar to scenario outlines in Cucumber or scenario tables in SLIM, with sets of inputs and expected outputs defined in a table and a separate workflow to define how these are used in the test. Twist visual editor has a very impressive function to extract parameterised scenarios from existing workflows and help users create the argument tables. Abstract concept functionality allows a user to extract a part of a workflow and define an abstract concept from that – essentially creating a parameterised workflow step to be used in other workflows. This is similar to how Robot framework uses lower level keywords to define higher level keywords.
Tests scenarios can have one or more contexts and tags assigned to them. Contexts might be defined as “logged in”, “basic currencies entered” and so on, effectively controlling setup and teardown activities. Tags allow free-form grouping and search. Both contexts and tags are managed on the test scenario screen, making it very easy to see what is going on and to change it.
Finally, Twist supports automated, manual and hybrid test execution modes to support things that aren’t that easy to automate or that require a human eye to decide the outcome. Manual or mixed tests are seamlessly integrated into the test management system as well, and produce the same kind of management reports as automated tests do.
Under the hood, Twist runs Java classes to automate test execution and connects workflow steps to Java methods using reflection, very similar to how Cucumber works apart from the fact that quote marks are used to separate arguments instead of regular expressions. For parameterisation, it supports basic Java types but you’ll have to do conversion into your own classes manually. Twist uses JUnit assertions to compare expected and actual results, which means that the test report shows you a failure but not what the actual result was inline (this is shown in a separate window).
At the end of the demo, I was very impressed with the refactoring and test management capabilities of this tool. Seamless roundtrip from test to code with refactoring support is a killer feature of this tool, and functionality such as rephrasing and abstracting concepts will no doubt save a lot of time in test maintenance long term. Whether this will justify going for a commercial tool, I’m not sure. Selling tools to developers and testers is becoming increasingly hard and many shops are moving away from QTP and things like in favour of opensource tools because of seat licensing.
For more information about the product, see http://www.thoughtworks-studios.com/agile-test-automation/.
|
|
Hi Gojko,
we have been using Twist in anger since beta. TW Studios have been very open and for once we feel like we have a real relationship with a vendor. We have around 200 scenarios now, but tha figure changes as we refactor to improve test execution times.
I think you captured the demo very well in your post. But there are some gotchas!
Stuart.
I’m evaluating twist vs cucumber cs fitness at the moment, twist seems like a really effective tool. Although I’m an avid supporter of free and open software, in this instance it seems worth paying the license fee for commerical support – particularly as I expect this to be primarily used by business users and testers, not developers.