Lambert said that creativity is crucial for software development problem-solving and a key factor for exploration, adapting and iterating, which are some of the core values shared by successful agile software delivery teams. Some structure is required to facilitate creativity, but excessive structure kills it, said Lambert. “We need enough structure”, said he, presenting several case studies of very creative companies. According to Lambert, the features of teams with “enough structure” are:

• people and their talents are a sole measure of competency
• they do not use generic best practices but apply relative judgement
• they spread skills and merge roles
• they create cross-functional teams
• they empower teams

This leads to short feedback loops, responding to change fast and craftsmanship. People in such teams are treated as humans, not as resources, according to Lambert. With excessive structure, people are treated as resources and the environment creates long feedback loops and slow responsiveness to change. He gave several examples of excessive structure:

• formal education and certificates as measures of competency
• enforcing generic best practices for everything
• creating silos of skills
• putting barriers between teams
• enforcing layers of approval

Lambert then said that Agile Manifesto is an example of “enough structure”, and that it supports creativity in teams. Opposing the trend to adopt agile development because “of its coolness”, Lambert said:

Moving to agile won’t make you creative.
It doesn’t mean that you can only be creative if you are agile.
But if you remove excessive structure, and have just enough of it, creative open minded people will naturally gravitate to you.

He warned against making agile “more palatable to companies who would not adopt them in the first place”, saying that the calls for quantification and certification of agile skills and teams create a risk of putting in too much structure. There is no such thing as “A scrum team is able to produce this… and if your team doesn’t, you can do this …”, said Lambert. Introducing metrics and check lists for what makes a team agile might make it easier to larger companies to adopt something, but that has nothing to do with original ideas of agile development. “It’s just a complete myth”, said Lambert, adding that “agile has been working fine”.

I’m not so sure about talent being a sole measure of competency, because lots of talent can lead to no effect without effort and utilisation, but I do agree with most of the other points. Certification and checklists do make it easier for large organisations to adopt the agile cargo cult, but I do not see how this can give anyone any benefits apart from the organisations that sell certificates. (Which was nicely picked up by Tobias Mayer recently in his blog post where he tries to wash his hands from certification like a modern day Pontius Pilate).

See the presentation slides on Rob’s blog.

Also see other articles from Agile Testing Days.

See more ports from Agile Testing days 2010

Record/playback implementations are tightly coupled with user interface layouts which makes them brittle and hard to maintain. That is the reason why teams gave up on user interface tests. According to Bache, the problems with data-driven approach (how he called API-level testing with tools such as FitNesse, Concordion or Cucumber) are that it doesn’t give non-programmers confidence in the whole system working and it deals with abstractions, and abstract tests require writers who can think in abstractions.

To get the confidence in the system and also easy maintenance, Bache advised combining domain-language descriptions of use cases with recorded user interface actions. The pyUseCase recorder tool he built allows teams to record actions and assign domain names to such recorded actions, which can later be combined into scripts. This provides a level of abstraction to make the test system more reusable and easier to maintain, but also makes recorded test cases easier to understand. There are similar tools for Java and .NET. (Bache said that Java Use Case recorder is no longer maintained, though).

Bache also advised teams to re-think the traditional approach of assertions as a way to validate actions in scripts. “Assertions mean variables, and variables mean programming. This breaks nice domain language use cases and starts mixing them with programming concepts” said Bache. This reduces readability. Instead of that, he advised verification by logging – inspecting textual log outputs of the system to validate that a function was performed correctly. “Tests are much more about behaviour change management than assertions”, said Bache, “It’s more about here is what my test does today, and I can manage a change of that tomorrow”. He uses TextTest to compare textual outputs of the system.

This approach allows teams to record domain activities, combine them into readable scripts and, save detect behaviour changes by comparing log outputs. Teams can focus on behaviour changes and domain language use cases. “Often the devil is in the detail and you want to manage changes in detail when your system behaviour changes”, said Bache. Instead of defining assertions, teams can inspect changes and decide whether the changes were expected or unexpected.

As additional advantages of this approach, Bache said it allows teams to create new tests required little or no programming. Resulting tests are robust and readable and cheap to maintain when user interface layout changes.

From my experience, aligning tests with the business domain model (which Bache called “thinking in abstractions”) works well for user interface tests as well and it allows teams to avoid the sine of death of UI test automation. More importantly, it allows teams to drive their business model using tests. But this is not at odds with the Use Case recording model. I see how Use Case Recorders can be very useful when recorded user interface testing is the only thing you can do, such as in implementing functional test automation on legacy system, especially when the state of the system depends on a large number of factors.

The traditional view for defects is that teams want to track and keep a record of them to perform analysis such as defect rates and root cause analysis. Many of the traditional advantages of defect tracking systems do not apply to agile development, said Crispin. Metrics, traceability, knowledge base of past defects and prioritisation are handled better with other tools in agile teams, such as unit tests, continuous integration and task cards. This makes the use of defect tracking tools on agile projects a bit controversial.

In the lean and agile view on software development, defect queues are queues of rework, and waste (Poppendieck). The lean approach is to write a test to confirm a defect, fix it and the test will prevent it from re-occurring. Teams using these processes treat bugs as technical debt.

As some potential uses for traditional defect tracking systems in agile teams, Crispin mentioned coordinating distributed/large teams and giving customers access and visibility over system issues. “If you’re using post-its or cards on the wall, people in other location can’t see that”, said she.

Offering an alternative perspective to bugs, she quoted Antony Marcano saying that defect reports are missed features and misbehaviour of the system. Bugs can point to missing features, so a defect tracking system can help identify them, but defect tracking systems are not necessarily the best tools for the job.

Instead of logging bugs, she advised writing tests to confirm the bugs. This is a good opportunity for the team to practice writing good tests, which they can apply later for test-first development. (This is the idea behind Defect Driving Development approach to teaching TDD). “Tests that reproduce bugs can be used as a good communication tool between developers and testers”, said Crispin.

Crispin said that choosing the right way to track or solve defects should be a team problem, that each team should understand their real needs and choose a solution by team consensus. Crispin advised teams to experiment. Some good ideas to experiment with are:

• Try starting out without a defect tracking system
• Try to turn bugs into stories
• Set rules such as “no more than 10 bugs at once”
• Try to fix all bugs immediately
• Make bug cards visible
• Prevent bugs in the first place

Crispin’s team uses a defect tracking system for production bugs. For problems caught in development they just communicate the issues and not track them. “A problem caught in development is not a bug” said she, as the team is still working on it. When they find such a bug it is usually fixed straight away. If the team decides not to fix it immediately, it is put on a task card.

Not all bugs need to be tracked, though. Crispin gave an example of when her team spent an iteration fixing low priority bugs, and the business users were surprised by that – they preferred to have new functionality built in and did not care that much about low priority bugs. Tracking bugs that nobody cares about is waste in the process. On the other hand, she said that sometimes they push back and explain to customers that some lower priority bugs should be fixed because the team will be able to move faster later.

Related to metrics, Crispin said that her team values trends and goals more than actual metrics. She gave an example as of a goal: “no more than 6 bugs in production over the next six months”. They use the goals to measure how they are doing and decide whether to change the process.