Here's an excellent article about why you should be doing Test-Driven Development.
No, really, it's excellent; go there and read it, then come back here.
A little harsh, isn't it? But very true. It's excellent.
However, something in it made me a little uncomfortable while reading, and it wasn't too hard to figure out what.
There's a lot of people out there under the misconception that unit tests and TDD are a QA method, and that if they do it right their software will have no defects (or “bugs”). That's a dangerous misconception. It's bad for your software, because it won't work; and it's bad for TDD, because when it blows up in your face, there's a pretty good chance you'll go out there telling other people that TDD doesn't work. It does work; and it probably did work for you. It just didn't do what you were mistakenly expecting it to do.
Now, if you will, go back to the article and search for any instance where Uncle Bob tells you TDD will make your software defect-free. He never claims that. The closest he says is “your software will work better”, which is true; TDD reduces bugs a lot, but most TDD champions (at least the ones who know what they're talking about) consider that a nice side-effect at best. (So if he doesn't make the wrong claim, why am I uncomfortable with the article? Because I can easily see proponents of the “TDD as QA” misconception misusing Uncle Bob's article as proof that they're right.)
TDD is not a QA tool. TDD is a development process, I'll even say a programming process. Its main benefits are, in order of (IMO) importance and relevance:
- Clearer and cleaner design. I'm talking about technical, architectural design, not visual. By forcing yourself to write down what you expect the software to do in a formal language (code), you come out with a clearer idea of what you're going to do; and by designing your internal APIs so that they can be easily called by unit tests, you end up with more modular and maintainable structures.
- Cleaner code. I've seen people whose unit tests are confusing but production code is crystal-clear. That's obviously not ideal, but it's much better than confusing production code. By focusing most of the effort in writing the test (therefore understanding what you're doing) and then writing the simplest code that makes the test pass, you make it harder to write convoluted code. (Harder, not impossible.)
- More confidence. Once you've written the test and you're confident that the test expresses the problem, you'll understand exactly what the solution is, and later after the code is written and deployed, you'll trust your old code a lot more.
- More reuse. To be honest, this isn't even about writing the test first, but in fact there's a step that often comes before writing the test: looking at the appropriate test file, reading the other tests, and checking if what you want is already there. (Because, you know, you need to find the right file in the tree and the right place in the file to add your test.) If there's something that does almost exactly what you want, and that you had never seen before, you'll write your new test and modify the existing functionality. If there's something that does exactly what you want, you save time and don't increase the code complexity.
- Faster. This is almost always difficult to claim, but it really does stand to reason. Think about the other benefits above; they alone make your coding a lot faster already, enough to offset the time you spend reading and writing tests. You'll end up writing less code, because you know exactly what you need and you won't write fluff. You'll end up rewriting your code less as you iterate, because writing the test made the solution clear to you. Writing code is much like the scientific method; you come up with a working hypothesis, check if it works, adapt as necessary. It might feel like we spend most of our time (in the non-TDD world) writing code, but in reality we spend most of our time figuring out stuff, followed by checking or rewriting code. Clearer code reduces time spent on the former, and writing your verification first as code reduces the latter.
As a nice side-effect, TDD also reduces defects. It does that by (a) making the design and structure cleaner and clearer; (b) making the code cleaner, therefore easier to work with later; (c) encouraging the programmer to think about the problem being solved and write “the right code”. See a pattern? And yes, (d) preventing regressions on the unit level by keeping the unit tests around to run later. But let's be honest: how many regressions are at the unit level? If your answer wasn't “very few”, there might be something else wrong with your process.
Now here's a few reasons why TDD will not take you to the magical no-bug land:
- Each unit test was written by the same person who wrote the corresponding code. Therefore, any misunderstanding of the problem, incorrect assumptions, or weakness in skill (come on, we all have those, that's why we work in teams and continuously learn from each other) will be reflected in the test as well as the code.
- It won't catch bugs on the feature/functional level; combine them with acceptance/functional/customer tests.
- It won't catch “subjective” bugs, also known as poor design. Even if the acceptance tests exist and say the software should do X and do it in such and such way, it takes a critical human to look at the running software and realise it's stupid to do X in practise. That sometimes takes the form of a technical or business attribute, but quite often it's visual or even aural. How often have you heard an UI/UX designer tell you something like, “I know we consistently paint this kind of widget red everywhere else, but now that I'm looking at it in this instance, it looks stupid”? Or “This is actually not related to the stuff around it, it's related to that stuff on the other side of the screen, so it should be over there”? How do you write an automated test for that? Or for the sound coming out crisp? Some classes of bugs need to be found by a human first, and then you can use that information to write an automated test.
- Negative tests. Strange corner cases (“what if the customer is in a zip code with extra sales taxes and extra shipping cost and uses a coupon?”) also fall in the category of, someone needs to think of it first before an automated test can be written. Some developers are really good at this, most aren't. Testers are trained to think this way and will come up with this kind of thing. More importantly, trying to come up with all the negative tests in the testing phase of TDD is really tedious, and can easily make the process so slow as to justify the arguments of the detractors. In the semi-official TDD graph (found in many places, but for example at Ward's Wiki) you'll see that testing is not only done first, but it's done between every two steps; the finding of corner cases should happen between coding and integration, and between integration and deployment.
- In a perfect world, you'll write your software from scratch, using TDD from day one, and should you decide to use any third-party libraries, you'll chose only ones that are fully tested (or if you're an open source or Free Software kind of person, only Free/open source libraries with full-coverage test suites). In real life, you'll be working with legacy code that doesn't have full unit or acceptance test coverage, so manual/exploratory testing will be essential to prevent regressions in pre-TDD features. Hopefully, as you find those regressions, you'll write new tests; but it will take you years to get enough coverage to be fully confident, if ever.
Conclusion: TDD is great for developers and you should use it everywhere. But it's not a QA strategy.