In our previous programming jobs, neither Gordon nor I had to do much testing. We tended to focus on architecture and development issues. Just prior to forming Infovark, we were acting as consultants for an ECM vendor that has since been snapped up by HP. So while we were aware of things like unit testing and test-driven development, we’d never had to do much of it ourselves.

We knew it was important, though. We’d seen too many things go wrong in software projects that didn’t make testing a core part of their standard operating procedures. So we incorporated testing into our product development and we’ve been learning about this testing stuff as we go. (It always happens that way, doesn’t it?)

We began with NUnit, the gold standard for unit testing on the .NET Platform, and quickly discovered some things that everyone else already knew:

1. Good unit tests are essential for refactoring. It’s the only way you can make big OO changes with confidence that you haven’t broken anything that used to work.
2. Good unit tests are are essential for troubleshooting. It makes it much easier to pinpoint the source of the errors when things go wrong.
3. Good unit tests are essential for making your product fail gracefully. Every program will fail at some point. It’s how a program reacts to a failure condition that counts.
4. You can’t write good test suites without good specifications. You’ll forget to test key assumptions and dependencies. You won’t confirm error conditions. You’ll couple your tests to your current object model, rather than to program requirements.
5. If you’re doing it right, you’ll write far more code for testing than for the product itself. It’s much harder to verify correctness than it is to write correct code.
6. Unit testing is different from integration testing. Integration testing is different from system testing. All three are important.

In addition to these common nuggets of TDD wisdom, we also discovered something else: It’s dreadfully easy to shoot yourself in the foot with NUnit tests. It’s easy to write tests that aren’t atomic, for example, or tests that expect errors to occur in one line of code that pass because another line of code throws a similar error.

To address some of these quibbles, a splinter group has launched xUnit on CodePlex. It aims to simplify testing syntax, incorporate more .NET 3.5 constructs, and prevent some of the more common NUnit anti-patterns that testing neophytes (like us) regularly develop.

We’ve messed around with XUnit a bit, and so far we like what we see. It’s not enough yet to make us abandon NUnit though, especially with NUnit 2.5 on the horizon. Besides, the NUnit framework is integrated with a variety of other development tools we’ve found indispensable. (I.e., everything made by JetBrains.)

But that’s just our opinion. What do you think? How do these things stack up? And what about the testing framework built in to Visual Studio Team System? We’d love to hear your experiences with these tools.

Continue Reading