I got this e-mail recently from a blog reader, and decided to share the answer as it might be interesting to others as well:
I listened to your podcast with Scott Hanselman and the discussion touched on the mix of unit tests and BDD acceptance tests, but didn’t really go as far as to offer guidance on where each is really best suited.
Since the BDD tests are integration tests it would seem that for ultimate confidence in the BDD tests as a regression suite they should cover all of the corner cases that the unit tests cover where possible (strange input cases that lead to null references, divide by zero, zero, one, edge case testing, etc) and be exposed to the BA’s to ensure that the code is handling these as expected. This seems to require essentially a near duplication of the cases handled in unit tests to achieve this level. The tests in the unit tests are still useful as they give guidance to developers on which layer the expectation is failing at, especially if the tests are layered with mocks stuffed with the correct behaviors at higher levels.
All of the examples of BDD I’ve seen follow mostly the happy path with only an occasional nod to the unhappy path and don’t seem to go into as many corner cases and essentially code coverage as unit tests too, but this feels incomplete.
I saw this article referenced in the show note comments, but it also felt incomplete
I’m curious if you have any advice or good examples of projects covered by both BDD and unit tests that achieve a good mix.
Regarding the division of responsibilities, I like Brian Marick’s quandrants as a model, they explain that nicely.
I disagree with the statement that BDD tests are integration tests. BDD tests are a functional specification of a system (in the quadrant model, they are business facing and supporting the team), unit tests guide technical design (in the quadrant model, team facing and supporting). They address two different issues. Unit tests tell you that the system works as the programmers wanted it to in the technical aspects, BDD scenarios tell you that the right functionality is there from a business perspective. They interact with the system on completely different levels and have different key examples. Null pointers don’t exist in most businesses. Integration tests check that the system components are connected correctly and that they talk to each other (including third parties), not that the business functionality is there.
I wrote about this problem of mixing up BDD and integration tests recently.
If the BDD scenarios you’ve seen cover only happy paths, you haven’t seen good examples then. They should contain key examples – positive and negative. See what Cuke4Ninja has to say on the key examples. (in the cucumber language, a feature file is a bdd test).
For examples of projects that got BDD right, see specificationbyexample.com.
|
|