The case against inheritance

FIT and its architecture started back in 2002 and heavily relies on inheritance as a way to extend the framework and integrate business domain code into the framework. That makes it hard to maintain the framework, which Mike pointed out, as lots of people depend on inner workings of the Fixture class. (I myself have complained a few times about his refactorings of the .NET test runner that broke my existing code). It also limits the options we have to integrate domain code with the framework. I often write a lot of boilerplate code to wrap domain objects into fixtures, since the integration layer has to extend from the Fixture class. Some options like target objects, system under test and domain adapters make it easier to connect domain objects directly to fixtures, but they require us to write lots of boilerplate code and we still have to use fixtures. It is hard to incorporate some generic test management functionality, such as starting and rolling back transactions or invoking the debugger, without changing the test runner or again extending fixtures.

Rich domain models and fixtures

Both FIT.NET and Java FitLibrary have started to move towards using a rich domain model (as in DDD) more effectively than with traditional mapping of each step into a fixture or a Dofixture method. Domain fixtures and domain adapters in FIT.NET and Java FitLibrary “explain” how to utilise rich domain objects. What we’ve noticed on the mini-session is that these domain adapters effectively do two things:

1. explain how to create and access domain objects (for setup and verification tasks)
2. explain how to find domain objects and execute methods on business services (action tasks between setup and verification)

In addition to that, it is interesting that FIT.NET and Java FitLibrary now have a mechanism to define meta-data about a whole test suite (suite configuration file in .NET and the suite fixture in FitLibrary).

Moving forward in that direction, we felt that we could lift the requirement to implement fixtures for a large majority of cases if the underlying business code is developed using a rich domain model and other DDD principles.

Integration with repositories

Most projects based on a rich domain model will now effectively use a repository to store and find objects, so the first benefit could be achieved by implementing support for a few popular repository method naming conventions instead of encapsulating it into a custom domain adapter/fixture. The suite meta-data could be used to select the appropriate naming convention and define the appropriate repository objects for domain objects. This could be used to prepare domain objects for the test or verify changes in domain objects later. For example, the traditional setup fixture becomes obsolete and could be replaced simply by a repository call. For example, this is a common set-up table:

If there is a repository defined for the Customer class, this table in the setup part of the test would be executed by instantiating a Customer object for every row, populating FirstName, LastName and Phone and calling CustomerRepository.Save(Customer c) method.

A very important new idea is to store the result of this operation directly into a symbol; we discussed the option to mark a column that contains the appropriate symbol key with a * or similar, but concluded that it would be better to just use the first column as the key name. So the previous table would create three Customer objects and store them automatically into symbols “Mike Smith”, “Mike Scott” and “Tom Cruise”. So this would effectively replace even Column Fixtures that are used to extract the result and store it into a symbol. The rationale behind storing objects automatically into symbols is that the objects prepared in the setup are most likely required later in the test (why else would they be created?).

If there is no repository, objects would just be created by instantiating the domain class and storing it into the symbol. If there is a repository, this would result in an additional call to the repository and the result of the repository call would be stored into the symbol. The reason behind that is to allow the repository to further amend the object if required by business rules.

If the first cell in the first row is not a class name, then it could map to a service method name or a repository method name. The corresponding method would get called by mapping the parameters directly and the result (if not void) would again be stored in the symbol. Repositories and services would be configured in the suite meta-data, so there would be no requirement to implement additional code.

In the verification part of the test, the same table (if the first cell maps to a domain class name) would act as a column fixture that retrieves the symbol value based on the first column, and then compares the other fields to actual domain object values retrieved from the symbol. (Possibly by going to the repository again to find an object by id if there is a repository, to support test-specific stuff). If the first cell in the first row maps to a method name, the method would be executed and the results would be compared to the table. A third option would work on repository finders and would also test for list length (similar to a list or set fixture). For example

would call CustomerRepository.FindAll() and compare the results with the table looking for missing or surplus customers as well.

would call CustomerRepository.FindActive() and so on.

Test-specific functionality

The acceptance sometimes do not list all the properties, to make tests more focused. If there is no fixture in between, the creation of objects in the repository might fail because of that. A solution for this case is to implement a decorator over the normal repository that adds the test-specific functionality (eg populates 10 remaining mandatory fields), and then using the test repository instead of the default repository in the suite meta-data configuration. This would also be the way to implement any other test-specific functionality. There is no way to avoid implementing this test-specific code, but this model would not require test-specifics to inherit from Fixture or require writing any other boilerplate code.

Talking to business services

One of the most important practices that DDD introduced is the ubiquitous language, a common jargon shared across all phases and participants of a project. One of the best practices for acceptance testing arising from that is using the same phrases for the same concepts in examples, acceptance tests and code. With this in mind, we really should not need any translation layers between the fixture tables and the domain code (both in objects and services). They should use the same jargon so we should be just able to glue them together by implementing a few naming conventions.

DoFixture and SequenceFixture support the concept of system under test, mapping calls directly to domain services. However, using SequenceFixture makes the test too technical and not really suitable for discussion with business people. Although this is relatively understandable:

It is much more natural to write this example as

DoFixture allows us to write scripts like this, but then requires the system under test to implement silly method name such as UserSendsInvitationToUserForRoom. No self-respecting programmer would create such a name in a business service.

Guessing the operation

Working on a few examples from the FIT book, we discussed how this link could be implemented better, for example creating something similar to DoFixture but that would map to more sensible method names, which would be written in normal domain objects and services. This would promote the idea of ubiquitous language even further, because it would suggest the correct service method or domain object method name. The final idea was to try out a few keyword combinations and find what the method name is, similar to service fixture but not requiring it to be the first keyword.

This could theoretically map to three methods: ChatService.Enter(User u, Room r), Room.Enter(User u) or User.Enter(Room r). If we have a ChatService in the suite meta-data configuration, then the algorithm could try to map keywords to either the service methods or class methods and look for the appropriate call. A possible pitfall is that this could lead to ambiguity. To promote the use of natural language, we would apply some basic transformations (such as stripping the final s from Enters), and ignoring some words (such as “a” or “on”). So we could use “Book a flight”, not forcing people to write “book flight”. Then:

would map to FlightService.BookFlight(Location from, Location to). The list of ignored words would probably also be configured on the suite level with some defaults.

The sentences could be even shorter, since the setup part would already have stored Ana, LOTR, LAX and JFK in symbols and we could extract the correct type.

We also discussed automatic object tree traversal, for example “Mike’s credit card number” would map to (Symbol(‘mike’)).CreditCard.Number.

Checking in natural language

Instead of

we would use the “is” keyword:

this would work very similar to “check” at the moment, meaning it will execute everything on the left how ever it is marked up, and compare the result to the cell on the right, but it reads much more naturally.

Tables or no tables

An interesting discussion followed after this on whether we need tables at all, or should we just try to recognise keywords. My personal opinion is that tables are good because they clearly identify what is a test script and what is just a description on the page. In any case, a bit of CSS tweaking can make the cells invisible.

Extension points

We also discussed the importance of providing a number of extension points or hooks for acceptance test execution, that would enable us to augment or modify the way tests are executed (eg run the test in a transaction and roll back, filter/debug on individual cells). The idea was to use something similar to filters (aspect-oriented) to attach to test runners and again encapsulate only test-specific code into that. The suite meta-data configuration file should allow us to specify filters that are applied to the whole test, to individual tables and individual cells (possibly with regex content matching to narrow it down).

Conclusion

With direct domain mapping to repositories and domain objects, setup and verification parts of most of the tests in a rich domain model project can easily be automated without writing fixtures if there is no test specific functionality. Smart keyword mapping will allow us to do the same for the middle parts of the test, where we mostly talk to domain services. Automatically storing objects from the setup part in symbols will make it easy to use those objects in the rest of the test. Using all that, we could effectively connect acceptance tests directly to rich domain models, without fixtures. Repositories and services would be configured in the test suite configuration file, so there would be no additional code required apart from genuinely test-specific code. Any genuinely test specific code could be encapsulated in test-specific repository decorators, and test-specific services, without the requirement to extend any class from the Fixture framework. Generic test-control functionality could be injected and reused across tests using configurable extension points, which should probably just implement a particular filter interface.

Challenges

Ambiguity seems to be the biggest challenge at the moment, and we need to work out strategies for avoiding ambiguity. Another challenge is to identify important naming conventions for repositories so that people can use this approach without changing the way they work at the moment.

An open question is how to divide the test clearly into three parts (setup, action, verification) while keeping them optional (eg support setup-verification or setup-action or action-verification). Rick’s domain fixture at the moment relies on a horisontal line (<HR/>) but this does not work with the standard FitServer. Possibly have some keywords (such as Given, When, Then from BDD) that stand as separate tables on the page.

Going forward

We experimented with relatively simplistic examples, and although this looks promising we need to try it out on some more complicated code. I promised to try to re-write some of my production tests using this model to identify potential pitfalls. If you did not give up reading this by now, then you are probably genuinely interested in the subject, so your feedback and ideas would also be greatly appreciated.

Was it too big?

Trying to keep the conference spirit but still grow, the organisers decided to try out a multi-stage format with lots of parallel sessions split into some general themes. I was a bit overwhelmed and somewhat disappointed that I could not attend all the sessions that looked interesting, but I guess that is a nice problem to have. During his session, Henrik Kniberg asked the audience to vote on the “this conference is too big” issue, with most people raising red cards to signal that it was a real problem for them. In general, I think that the stage model worked relatively well, but I would probably prefer having only one session per stage next time. I decided early on not to attend any more workshops. Two workshops I have attended on the first day ended by participants only practicing their writing skills, without any recap or analysis on the end, leaving me with no feedback on what I have done good or what I should do better. Even worse, I might have missed something more interesting on a different stage. Presenters organising workshops should really leave more time to provide feedback for learning. One more thing that seemed to concern a lot of people is powering their laptops between sessions. It would be great if conference tables had power extension cords next year.

Here are the sessions that really caught my attention:

• Marcus Evans: the FrAgile organisation
• Robert C. Martin: Quintessence
• Marry Poppendieck: The elephant in the room
• Aslak Hellesøy: Executable User Stories with RSpec and BDD
• Henrik Kniberg: 10 ways to screw up despite Scrum and XP
• James Surowiecki: The Wisdom of Crowds

We are now officially mainstream

If anything, this conference has made me realise that agile is now definitely mainstream. As Ron Jeffries put it during his session on Natural laws of programming, “Agile is dead. Sixty million people are using it and it is no longer a revolution”. Henrik Kniberg said that one of the biggest mistakes with Scrum and XP was expecting that there is an installation CD for Agile. Ironically, just outside the doors of the conference room where he talked, there were dozens of companies bidding to sell that very CD to oblivious punters.

There was a surprising number of sessions on acceptance testing in various forms and under different names, which signals a shift of focus from programming practices to things that need to happen before the coding starts. Acceptance testing is gaining a lot of momentum, although I think that the current focus on tools is wrong and that people should be focusing more on the communication benefits and flushing out inconsistencies with realistic examples. It seems that the next big thing for agile practices will be emerging a coherent best practice for specifications and requirements out of all those seemingly different approaches. A thing that Ron Jeffries said about classic requirements during his session on Natural laws of programming really stuck with me — “the most important thing on a requirements document is the phone number of the person who wrote it”.

Snake oil and bureaucracy

Seeing so many agile project management products on offer at this conference scares me to death. Most booths in the hall were selling some sort of magical software that allows us to communicate through clicking and issue tracking and generates very nice graphs for project managers to misunderstand and misuse. I have still to be convinced in usefulness of any such tool, agile or otherwise. To me, this seems to completely miss the point and discourage personal communication.

Another thing that really bothered me was the emergence of preaching for a particular specific way to write things down which I noticed on few sessions. I attended quite a few sessions on acceptance testing and user stories and I’ve seen people advocate a particular format of writing user stories and bashing other formats, as if a format change would really make an important difference for the project. This reminds me of the famous curly brace discussions and similar coding format issues, that were always so pointless but wasted so much time. Ken Arnold’s legendary article asking for style wars to stop somehow always comes to my mind at these situations.

My personal perspective

On another note, I am very happy that I attended the conference. I met so many great people and put the faces on lots of e-mail addresses. There were quite a few interesting sessions and I heard a few new ideas and learned about enough interesting books to keep me reading until the next year. The conference was a great chance to present my new book in progress and the ideas for that book to a few people that I really respect and admire and the feedback was very encouraging, so I will now focus on completing it as soon as possible.

I was very glad to present with Marisa Seal in front of nearly a full room of people interested in database TDD. Judging from the comments and questions after the session, DbFit might be getting a few new users out of this.

One of the most interesting things for me was not actually a scheduled session, but a mini-discussion on FIT without inheritance that Mike Stockdale organised. This lead us to rethink the whole model of FIT a bit and digressed into FIT without fixtures, looking for ways to map acceptance tests directly to business domain services, repositories and objects. The idea is still half-baked but looks relatively promising. Expect a lot more writing on this idea soon on my blog.

According to Evans, when he joined the BBC six years ago, the organisation suffered from lots of communication problems with people sitting in the same room not even knowing each other by name. There was no clear project management method at the time and people were vertically split according to projects.

Scrum was introduced in the teams very softly, without any big political announcements or fanfare. It was not hailed as the future, and Scrum was even not mentioned as such. The approach met with very little resistance. If there had been a big announcement, Evans assumed that there would be a much bigger reaction. The subtle way of introducing Scrum allowed teams to discover the functions of Scrum from inside. Rather than applying practices by the book, teams at BBC embraced principles and that influenced their internal culture.

In the next phase, the whole organisation moved to a matrix structure and people started meeting other people doing the same job in other teams. That helped a lot with sharing ideas and then people realised that other teams were doing agile practices as well. Agile advocates formed a regular “scrumtalk” meeting to share best practices, but there was still very little collaboration between teams.

Then they started working on BBC iPlayer, hailed as the project of the same importance as moving from B&W to colour TV. The project was huge and required teams to start working together, but using agile methods for that project was not questioned at all. Evans commented “that meant it was our culture”. People just looked for ways to scale scrum and extend it to a single project involving multiple teams.

Another restructuring hit the company during the project, going back to the pre-matrix organisation with independent teams, independent resourcing and training. The scrumtalk meeting became a “hothouse for anxiety” and “an environment of paranoia”, losing its energy. However, when the participants decided to do a retrospective and analyse whether they are still doing agile, involving an external consultant to help them get a more realistic perspective, they realised that they were actually still doing OK.

Marcus Evans pointed out the following few things as his key learnings from the project:

• The way that they implemented agile was very much in their favour. Subtle, no PR, no big positive speeches reduced the resistance and allowed the culture to grow from inside
• Reorganising, fanfares and political announcements caused fears, but there was rarely a significant effect on people actually practicing agile practices. Although they had serious concerns about the way they work from inside, looking at the same thing from outside actually was quite comforting
• The matrix structure really helped
• The antidote to concerns was to be inspective, reflective and hold retrospectives and get a more realistic view.

read other news from the Agile 2008 conference

Pointing out how the Agile conference was created by merging two earlier conferences a few years ago, one primarily related to XP and one primarily related to Scrum, Martin suggested that a merged conference predicted the fact that those two methods have by now effectively merged into a single set of agile development practices that people actually use. He then pointed out how this still does not solve some core problems. When he asked the audience who has at least once been significantly impeded by bad code since going agile, most people raised their hands.

Martin criticised both Scrum and XP for becoming “too meta”. Scrum was criticised for unrealistically expecting everything to be ideal and for promoting certifications as a way to confirm someone’s professionalism. On the other hand, XP was criticised for being too strict. As an interesting demonstration, he asked everyone in the audience to raise their hands and then started listing original XP practices, asking people that do not use a particular practice to put their hands down. My estimate is that there were about one and a half thousand attendees, but only a few hands remained raised after the fifth or sixth practice being listed. When he asked who in the audience has unit test coverage larger than 90%, most people simply laughed.

Commenting on the current state of affairs in the software industry, Martin said that “The fastest way to get done with dinner is not to do the dishes. Making dinner starts taking some serious time and after a while you start to outsource dinner… “. He then pointed out that we should focus more on delivering stable and high-quality code rather than going fast, and not give in under pressure of schedules and deadlines to compromise quality. Calling for programmers to be more professional, he asked “If you believe yourselves to be professional, how can you release code that you are not sure it works?”.

Martin then talked about Ignaz Semmelweis, a hospital administrator in 19th century Vienna who discovered that the mortality rate in a maternity clinic could be drastically cut by making doctors wash their hands when moving between the autopsy ward and the maternity ward. Semmelweis met with stiff resistance, including the claim that doctors had no time to wash their hands, according to the talk. Wikipedia entry on Semmelweis states that he is now considered the “savior of mothers”. (Interestingly, Wikipedia also has some grimmer facts about his fate: “His contemporaries, including his wife, believed he was losing his mind and he was in 1865 committed to an asylum (mental institution). Semmelweis died there only 14 days later, possibly after being severely beaten by guards.”)

Drawing parallels between that story and software development, Martin asked programmers to “wash their hands” and write clean code, not giving in to time pressures or claiming that they have no time for that or that they will not be allowed to do that. Asking for people to do a good job and be better professionals, he suggested a change to the agile manifesto, adding the sentence “we value craftsmanship over crap”.

see other news from the Agile 2008 conference

Quoting Jeffrey Pfeiffer’s testimony to US Congress in 2007, Mary Poppendieck said that there is a lot of evidence that intrinsic motivation is much more effective than external factors such as financial bonuses, pointing out that external motivators can actually undermine intrinsic factors. Opensource projects, for example, work because people care about them and starting to pay some of the contributors will negatively affect the motivation of other contributors on the project.

Citing a research by Arie de Geus, author of The living company, Mary said that highly successful companies foster a culture which does not use financial compensation as a primary rewards motivator. The same research shows that those companies pay people above market rates to to reduce turnover and attract the best people, coupled with rewards that share organizational success, but money is not the primary reason why people work there. As an extreme case, Mary mentioned Tandem Computers, who according to her gave no salary information on job interviews. Their CEO was of the opinion that people who come for money will leave for money and did not want those people onboard.

Team payment does not work

Mary said that 80% compensation systems reward individual, not team performance and that
only 10-20% of appraisal systems include team performance, causing a lot of competition inside teams that discourage team behaviour. In software development, it is very hard to establish the effects of individual contributions and good teamwork is key to the project. Most individual compensation schemes, according to the presentation, absorb vast amounts of management time and resources and leave nobody happy, but team compensation strategies are not easy to implement. Mary presented results from HP’s experiments during the beginning of the nineties, when HP allowed 13 local organisations to experiment with team-incentive plans. All programs were discontinued by the 4th year, due to constant changes to the plans which were needed to distribute available money among the teams and a wide dissatisfaction with the plans by employees. On the end, all projects concluded with no relationship or negative relationship between actual incentives and performance. San Diego team pay for performance program started great, with everybody feeling very positive about the plan in the first six months and most teams outperforming their goals. However, that meant that a lot more money was given out than HP planned for, so the standards were adjusted in the next six months which met stiff resistance. High performance teams started excluding new members, managers felt that teams concentrated on money not jobs and the plan was discontinued because it caused way too much hassle.

Mary also gave a case study of a high performing team that was given a large bonus to distribute internally. Everyone agreed that individuals should get a part matching their contributions, bot no one could agree on individual contribution levels. After a heated discussion, team decided on an equal split which left everybody unhappy, and the same group of people never worked again well as a team.

Instead of financial incentives, companies should focus more on encouraging and training teams to provide internal intrinsic positive feedback and non-financial positive reinforcements, according to Mary.

Fair compensations are effective

Citing a research from Elliot Jacques, she said that the most effective financial compensation schemes are those that are felt fair in the organisation. Differences in salaries can exist between different levels that are perceived fair. In software, criteria that is generally accepted fair are the timespan of discretion (how long can you work without your boss looking over your shoulder) and job complexity. That kind of compensation schemes is, according to Mary, validated by many studies over several decades. An especially important idea was that the money should not be used a differentiator at the same level, and that if money moves rapidly in the market, everyone in the company should be moved up regularly, not generating a gap between new hires and older employees.

On the end, Mary gave some advice on effective incentives and compensation schemes:

1. abandon the zero-sum game: eliminate competition for a bonus pool
2. tell people directly what’s important and why. it is cheaper and easier than financial incentive systems signaling what is important
3. base pay differentials on job complexity, avoid individual performance bonuses because they extinguish collaboration
4. base collective rewards on broad measures — the larger the group that is measured, the more reliably performance can be accessed. Use profit sharing schemes instead of bonuses to tie people to the organisation goals.
5. keep in mind the norm of reciprocity — if people feel that they are being treated generously, they will reciprocate it with increased discretionary effort.

read other news from the Agile 2008 conference