Ok, now we know what TDD is all about and how to do TDD. Really? Well, no. In fact we only know how TDD works, but TDD is about writing tests (to be more precise – unit tests). What should our tests look like? Here it comes…
What is the Unit?
Why do we call these tests unit tests? The answer is simple – because we test only a unit. Very good answer, but what is this unit? Method, class, module? Unit is simply the smallest part of your system that you can test. If you think about testing something, think about testing only a part of it. If you can imagine this, it means that was not a unit. Now, think about its part. Can you think of testing only part of it … (I think you’ve got my point and if not – read about recursion)
What is the (Unit) Test?
Ok, we have the smallest thing that we can think of. It’s probably a method, maybe a class, doesn’t matter. Think about a simple, but representative scenario of using it. Let’s play with “log in” service. Let the simple scenario be: correct username and wrong password (I’m not saying that this should be the first scenario, but it should be some). The second thing – expected behavior. What should it do under given circumstances? Let’s assume that it should return the message “Wrong password”. And this is our test. Oh, ok, but how do we get the user who exists in database? We have to connect to the database, file or something else. This is exactly my point: separations of concerns, isolation. We have to test our “system” in isolation. We don’t want to test our database; we want to test only the “log in” service. So, we have to provide fake data (but about fakes, stubs, mocks and isolation I will write another time) to test only our service’s behavior.
First pillar: Trustworthiness
Unit tests should be our parachute, bungee rope, safety net. Therefore, we have to trust them, otherwise they are useless. If your tests pass, are you confident that it means something? Maybe it is always green no matter what or maybe it doesn’t test what you think it is testing. A common problem appears if you change or only refactor something, run your tests and all are passing but you still aren’t sure if you didn’t break anything. But if you follow red, green, refactor cycle, you can be sure. Firstly, you saw your tests red (all of them) at the beginning. Secondly, you know they are only passing because you actually wrote a specific code. So, you are confident that if all tests are green your module is fine. You don’t have to check it in another way.
Tip
If you have to modify somebody else’s code (let’s assume that there are tests :)). First, try something like this. Remove one random line (make it compile, if you have to) or change if/for/while condition and run tests. If they are still passing – it’s bad for you. This parachute is leaky, better don’t use it (or do, but don’t be surprised if you fall down).
Second pillar: Maintainability
It is obvious that tests require care in the same way as production code. If you change production code (refactor, fix bug or add new functionality) some of your tests will break (it means that neither will they compile nor fail). It is a great issue. The more tests the more attention they need. We can reduce the time we spend on nurturing by writing test only against public method. Public methods are contracts. If we specify some contract we will rather not change it as often as private methods. If you feel that you have to test private method it is a sign of a bad design (it’s called “code smell”). Probably functionality of this method belongs elsewhere. Next thing we can do is to remove duplication in tests, move parts that can vary (e.g. construction, initialization) into one place. You can use factory method or common “set up” for your test. If some tests in your test class need different “set up” than others, split them into two test classes (yeah, two test classes can test one production class or even one method, why not?). Tests need to be refactored as often as production code.
Third pillar: Readability
Last but not least, readability. Since unit tests are executable specifications, you should be able to read them as specifications. First (and I think the most important) aspect is good naming convention. I can’t say that the convention I will present is the best, but I use it and it works for me. It is the same convention as Roy Osherove recommends. Here is the pattern:
[csharp]WhatAreWeTesting_InWhatConditions_WhatAreExpectedResult[/csharp]
E.g. LogIn_ExistingUsernameWithIncorrectPassword_ShouldReturnMessageWrongPassword.
First of all, why is the name of a method so important? Because, in every (as far as I know) test runner this is first (and most visible) information about failing test. If you follow my guidance, you won’t need anything more. You will know what’s wrong right away. Neither asserts messages, nor are comments necessary. Method name is all you need. Don’t worry about a long method name. IntelliSense will help you, but in this case you won’t write it anywhere, so there is no problem at all.
Next aspect is a single assertion in the test. It is strongly connected with the test name. If you have multiple assertions in the test, how do you name it? It’s the first clue that something is wrong with your test, when you cannot name it in a simple way. And if you have a single assertion for the test, what is the purpose of assert message? It’s obvious from its name what the test is asserting. There can be exceptions to this, e.g. when we want to do assertion on an object which doesn’t have overridden Equals() method (but maybe it should have?).
I strongly recommend using AAA (a.k.a. triple A) pattern for the test layout. It stands for Arrange, Act, Assert. It means that the test should be divided into three parts. First part is Arrange. Here are all the possible preparations for the test execution to take place: initialization, stubbing, etc. Second one is Act. This is where method, which we are testing, is invoked. The last one is Assert. This is the place for assertion. These parts could be separated with comments like “//arrange” etc., but I figured it out that these aren’t necessary. I separate these parts using empty lines (each part has no empty lines inside), so every test has only two empty lines.
If you trust your tests you use it more often, so you will write more of them. When you have more tests you trust them more (do you see the cycle?). If tests were easy to maintain you wouldn’t be scared of writing more of them and you wouldn’t waste your time on changing them. If tests were readable as a spec, you’d find them useful as an introduction to somebody else’s code (or reminding your own code). It’s a great power of a good test suite.
If you are interested in topic I recommend watching Roy Osherove video from NDC2009
If you have worked out your own conventions or you want to share your opinion feel free to drop a line below
“If you feel that you have to test private method it is a sign of a bad design (it’s called “code smell”).”
I strongly disagree with this assertion. Many of the key assumptions your software will make will be created by methods that are not part of the public contract. There is tremendous value in unit testing these key assumptions as discrete unit tests in addition to your public contract. These methods are generally where things happen and/or are small helper algorithms that if a bug gets introduced can lead to very hard to trace unexpected results.
Prime example of this would be a service that does validation and your contract is Validate(myObject) inside of Validate you have a few helper methods IsMyObjectInAllowableTimes() IsMyObjectNotThisBadCase() where one of your helper methods has a logic fault that results in it returning true in cases it shouldn’t.
If you didn’t directly unit test your assumptive helper methods the only way you would reasonably find the fault is the helper method is #1 luck, or #2 100% code coverage. In complex services reaching 100% code coverage through your public contracts will be a very large amount of unit tests you will need to write.
” change if/for/while condition and run tests. If they are still
passing – it’s bad for you. This parachute is leaky, better don’t use it
(or do, but don’t be surprised if you fall down).”This directly sums up my analysis. The tests pass due to lack of 100% coverage. Generally unit tests get created to cover intended paths, or written to produce a failing test when a bug is reported. Other code paths, and unintended code paths are frequently missed in testing unless coverage analysis is done, or you test your internals to make sure you can trust methods you assume to be valid.
I strongly disagree with your disagreement. If you have so much critical logic in private methods that it isn’t feasible to exercise them thoroughly and cleanly through the public contract, it’s probably a sign that your class has too many responsibilities and is in need of some refactoring love.
I second Matt, the key is “In complex services”.
It is perfectly fine to have a complex service that provides a lot of value to the caller, But that doesn’t mean your code is a Huge single class with it.
There are better ways and that is what unit tests reveal if you pay attention to what they are saying.
These types of answers are textbook answers and ignore reality.
“But that doesn’t mean your code is a Huge single class with it.”
If you’ve never seen 3 if statements, or even 1 single if statement bring down a production website due to a logic fault…
I’ll byte.
What you call reality is really legacy code. If the class has too much logic, it needs to be broken down. There is plenty of crap in books, and the whole TDD will make the design for you is bs (it just gives you hints on the problems, that’s it). You need to learn appropiate code level design skills, really understanding SOLID goes a long way.
What I find all the time in “complex” systems, is not that it can’t be broken, is that the previous team didn’t know how to take it gradually. Trying to stabilize an error plagued system by just patching the mess, gets you huge teams barely progressing on what they’re doing.
Another big issue is completely ignoring bounded contexts, which has a lot to do with an architecture concept that wasn’t really introduced with DDD. That alone is responsible for the ugliest of the messes, and plenty of those code bases where it would seem nothing can be done about it.
Have I seen silly single liner mistakes that could break havoc? of course. Are those usually caught by unit tests over the public interface of well designed classes? you bet. Is it extremly faster/easier to detect and solve on clean code than in messy code? definitely. What else? functional smoke tests, review what you’re about to commit.
ps. I agree on using code coverage, but you can’t completely rely on that, as 100% coverage still get you bugs.
I never it wasn’t possible to exercise them through the public contract, I said the only way to know for sure if you don’t specifically test the private methods is to use coverage analysis and reach 100% code coverage.
That’s certainly not the only way. That’s a problem that TDD addresses directly. TDD combined with proper use of patterns and design principles such as SOLID leads to code that is simple, clean, and easy to test. As pointed out, that doesn’t mean that collectively a system of classes cannot accomplish something complex, but individually, the classes can (generally) be kept simple and easy to test.
Can’t agree more. TDD leads to loosely coupled classes with a single responsibility which is easy to test. Whenever you deal with complex class that has complex private methods it is a sign that maybe you should pull out functionality from these methods to other class and test it in isolation.
While I agree, you really have to qualify it as Matt did.
TDD alone only gives you hints; your code and speed only benefit when you pay close attention to the hints And combine it with proper use of patterns and design principles.
Thanks for your comment. I understand your point of view, testing private methods seems to be good practice, but it isn’t. Maybe there are some cases where testing private method isn’t wrong but generally it is. Helper methods are an effect of refactoring which had been done while doing “Red, Green, Refactor cycle”(https://blog.goyello.com/2011/09/13/red-green-refactor-cycle/), so the logic of these methods is tested by testing through a public method. Using your example of validation, if Validate() consist of some helpers, you can write tests in which each and every possible failure reason is included. Second aspect is maintainability (which I was writing about). If you test private method and later you decide to move this logic to another (helper) class (e.g. you want to use this very piece of logic elsewhere) you don’t have to change any of your tests!
About 100% code coverage. The goal of TDD is 100% code coverage, but as we all know this is pure fantasy. However, if we want to believe our tests it should be near 80-90%. These technics (changing if/for/while conditions, returning different value) can be used when checking legacy code, as well as when doing test review. You’re right that there is a place for luck in this technique, if you want deep analysis you can use tools for that.
I would add 2 pillars :
–> When a complex class is coded, make sure to cover it 100% and make sure it will remain covered 100% in future refactoring
–> Use code contracts (through MS contract or Debug.Assert()) and make sure that unit tests execution don’t violate contracts (when a contract is violated at unit testing time, make sure the failure is obvious)
Code coverage near 100% is a part of trustworthiness. As you wrote, when dealing with complex class this is even more important. Good point.
About second, It’s great idea. I love it. Thanks for writing this!
Cheers,
Paul.
>Code coverage near 100% is a part of trustworthiness
From my experience code coverage NEAR 100% is not enough, because the few 5% lines of code hard to cover in a class, typically contains bugs 🙂 The reason behind the fact that a line of code is hard to cover, is the same reason that usually provokes bug: complex, hard o understand and low quality code.
When we speak about class with complex logic you can achieve 100% but generally there are e.g. model classes which you simply don’t have to test. Now, about those 5% lines of code hard to test. It’s a sign that maybe it’s to tightly coupled. Maybe it’s worth considering pulling this out and testing it in isolation? Every case is specific and it’s hard to talk generally about them.
I understand the importance of unit test (or good unit test). But why should I be writing tests before the development (as in TDD) ? At the end of the day, what’s important is the unit test ! right ?
Yes it is important to have one (or more) test (s) that validate the code that you write. But if you type it after the writing of your code you’ll be somewhat have a biased perspective with the code you wrote. So if before writing the code, you think of correct behavior of you function or module, you will detect problems more effectively.
This is true. More informations about advantages of TDD you can find in my other blog posts: “What does TDD mean?” and “Red, green, refactor cycle”
Congratulations for the fantastic article
Thanks Paulo. I appreciate.