Test first or not test first, that is NOT the question
TDD (Test Driven Development) is becoming more and more popular because it helps
- Enable friendly APIs since they are constantly challenged by unit test cases. As developers write test cases, they are actually using their own API and get a feel of its friendliness
- Encourage developers to stick with the requirement and implement no more than what is necessary (i.e. “gold plating”)
- Avoid developers from falling into the trap of testing what the code does instead of what it is supposed to do
- Allow the changes to be made to the code without silently breaking any other dependants since ttest cases will detect the broken code immediately
- Serve as a comprehensive working documentation for the API’s users
The only problem with TDD is that many people rigidly believe that writing test first is the only way to do effective TDD. I disagree. The point of TDD is not about writing test first or not, the point is to have a good set of unit test cases which are update-to-date, has high coverage and quality (the same quality expected for production code) and at any time, you can be confident about that set of test cases. Some people feel it easy to achieve these objectives by writing test first, but others may feel comfortable writing the code first.
Some of you will say “you’re missing the point, test-first is also supposed to be a design technique,” however, I think that’s been over-hyped too much. The truth is that a good design can only be achieved with a solid understanding about design principles and application trade-offs as well as the continuous evolution of the design to cope with new or modified features, not by writing test first. I have tried both in my projects and that’s my conclusion so far. Disagree? Tell me examples of something you think only achievable with test-first and I’ll tell you how implementation-first can also do that.
Venkat Subramaniam has a very nice analogy between the decision whether to write code first or write test first with the decision to walk the left foot first or the right foot first. The point is very obvious: it doesn’t matter which foot you walk first, as long as you walk in “sync” instead of dragging one foot miles before moving the other, you’ll reach your destination safely.
I tend to write the implementation first before writing any test for it, mostly for 2 reasons: I feel it is more intuitive to verify things already existing (I sometimes think it a waste of time to run the test just to see the red bar for the code does not exist) and the IDEs I use can only generate test from code but not the other way around.
For TDD newbies, I would suggest them to practice test-first since it does enforce them to be more disciplined in writing test. After all, it is sometimes very appealing to not to write test after you have the code which seems to run well by a Ctrl-F5.
[...] there should be no chair. In fact, there are huge differences between developer testing (unit testing) and QC testing (integration testing, performance testing, security testing, usability testing [...]
I am interested in the paragraph: That’s my favourite comparative way!
“the decision whether to write code first or write test first with the decision to walk the left foot first or the right foot first. The point is very obvious: it doesn’t matter which foot you walk first, as long as you walk in ’sync’ instead of dragging one foot miles before moving the other, you’ll reach your destination safely.”
However, I am not very satisfied when reaching to the end of the article. Let’s see Buu said:
“I tend to write the implementation first before writing any test for it, mostly for 2 reasons: I feel it is more intuitive to verify thing already existing (I sometimes think it a waste of time to run the test just to see the red bar for the code does not exist) and the IDEs I use can only generate test from code but not the other way around.”
These sentences persuaded readers and led them to the direction that there would be no erroneousness to write code first for senior devs (?). In additional, the last paragraph Buu suggested novice to write test first. It was appearance to me that writing test first should apply for junior ones only(?). Furthermore, if we had had your IDE, things would easy, we would have not been here to discuss such a boring topic.
This may be an unintentional shortcoming in the article. The only problem with TDD is that I bet there is more than one. Let say writing and maintaining test can be time-consuming. I don’t bring out here all disadvantages of TDD because this is article does not concentrate on that.
Phong, I did say not test-first should only be done by junior devs and code-first should be done by senior folks :-). In fact, I have made my point very clearly: it does not matter whether you test first or code first, it is personal taste and is not the goal. The goal of TDD is to have a up-to-date, high-quality, high-coverage suite of test cases which make you at all time confident in your ability in changing one part without affecting the others (because a good test suite should detect any problem immediately).
However, I do indeed suggest TDD newbies to practice test-first for several reasons, including:
- Unless you are really disciplined, it is very tempted to NOT to write the test after you have the working code (at least, you think it works)
- It does encourate them to focus on the things that are really necessary (instead of spending time “gold-plating”, a mistake that some junior developers have)
- It give them a chance to understand why some people advocates writing test cases is designing software, as well as a chance to determine whether they do like test-first or code-first
Finally, while I do agree with you that TDD may have more than one problems associated with it (no method is perfect), I think many of the problems are due to people do not apply it correctly. For instance, the writing/maintenance is only a problem if the effort you spent on them cannot justify the problems which may arise by hidden broken modules (when you make change to one area of code) or the time-consuming effort of stepping through each line of code with a debugger to identify a defect without being really sure where it is located.
Buu, you did say it clearly in first parts of article. And you didn’t say different suggestions to juniors and seniors. However, your reasoning in latter parts somewhat distracted readers and led them to that thoughs as I stated in the previous comment.
I knew you didn’t think “there is only one problem…” in literal sense. And I do understand that the sentence is just used to emphasize that “…many of the problems are due to people do not apply it correctly”
There is a post at Artima website in which the idea of whether to test-first or code-first is aligned with my thinking
+1, Test first or not test first, that is NOT the question –> That’s true in practical. I often write ‘code first’ and ‘test right after that’ too. In some cases, it saves quite a lot amount time.
@Lam: yes, as long as we are disciplined in writing high-quality test cases with high level of coverage, we should be fine. It may be a waste of time if you have to do something you are not comfortable with (e.g. must write test first, or must pair programming, or must sit in an open space etc.) Some people are just fanatic about a particular practice and believe it’s the only right way to do something. Sigh…
right Buu, furthermore, it is not fully safe to say that: “senior writes test before code and junior is hard to follow that discipline, so they will be encouraged to have code before testcases”. We just need to be flexible but creating the rule which requires >90% of coding has had coverage already. If one project could achieve this constraint, then I am sure that its test cases would be up-to-date. In this case, who need to care about code first or test first? Now our concern is just about the capability of writing good test cases which will leverage the quality of the coding and the whole project, of course.
Who said that? I did say that people should pick whatever they seem right, but TDD newbies should practice test-first so that they can understand deeply about it (before deciding as to whether they like test-first or implementation-first) and because it helps them to be disciplined about writing test.
Coverage, while being an important metric, by itself does not determine the test quality since the code “covered” by the test does not necessary work correctly. We can also consider complementary tools like Jester.
Totally agree
Thanks to the introduction to Jester. It is quite a great idea and it might see as a supplement approach for the tradditional code coverage tool. I just wonder if you have ever seen a case study/success story of running the tool in reality?
Generally speaking, the more tools you use, be it a mutation testing tool (like Jester) or a static code analysis tool (like FindBugs), the more chances there are for you to identify defects in the code and in the test cases. In practice, the benefits must be considered against the overhead of adding another tool into your development life cycle (e.g. not every reported issue is useful while you need to spend time review it).
I have used Jester, FindBugs, PMD etc. in my projects on a irregular-basis (milestone/external release), instead of having them integrated as part of the development life cycle (like JUnit and JCoverage), and so far I think it’s been helpful.
I think it’s pretty straight-forward as to how Jester, as a mutation testing tool, can help assure the quality of your test suite, so I’m not really sure why you need a case study. (The simplest example is that Jester changes a conditional expression in your code from true to false and see if your test suite report a defect for that modification.)
Buu, I meant that the Jester is a great concept. But I still want to evaluate its maturity in order to apply to our SDLC. You know, in open source perspective, there are a large number of tools outdate. Assumed that their purpose are quite great, but they are just not usable due to lack of development progress. One important idea come from Agile theory is: just use enough and right tools to do your job.
I agree, Lam. As a matter of fact, I would also be very cautious before applying any new tool, be it open-source of commercial, and usually perform several evaluation to make sure it can add value to the project. Back to the topic, I suggest you download Jester and kick off a run over an existing application, and then see whether the things it identifies are useful or not - while I find it helpful, your projects are different from mine and there’s no tool that is beneficial for all projects.