James Dean, James Dean, you bought it sight unseen.
You were too fast to live, too young to die, bye bye. — The Eagles
The benefits of unit-testing are enormous. I don’t think anyone can deny that, but to make unit-testing work for you actually have to write tests.
As I see it I have three (no four) main problems with unit-testing which ultimately boil down to the time and hassle it takes to write certain types of tests:
- The generally approved technique is to start with the test and start by making a test that fails for a specific functionality. You can’t compile the failed test until you’ve written some code to call. Then you find yourself writing something that will take a few arguments. For simple methods that take some arguments and return some values means coding some stub, switching to the unit-test view, write a failing test, switch back and write the functionality. This method probably took 3 or 4 times longer to write than if I hadn’t tried to unit-test it. I know that I will get that payback later but my life contains plenty of context switches already and having to add these extra ones is a pain. I know (but can’t find a link) that there’s a special tool for Ruby that automates this process. You write a test with a well known name, it creates all the stubs and makes it throw exceptions. I need this feature!
- Sometimes you will end up with abstract classes in your design. These are awkward to test for a number of reasons, one of which is that you have to create a unit-testable implementation to test it since you can’t instantiate it because it’s abstract. This means creating especially derived inner classes for your abstract class.
- Unit testing data sources. Some unit-testing (like Rails for example) include the data sources in the test. But I’ve heard the view that this is a form of integration-testing since you are testing more than just the class when you get a bunch of data from a database. This raises the ugly question of how to test data source access classes without a data source. It’s a similar problem to the abstract class problem in as much as you need a special instance of a class for your unit-test purposes. To serve this purpose you can use a mocking framework like NMock or jMock or even Mockpp! Great though these tools are it’s very easy to create a monster mock object that has all sorts of complex and hard-to-maintain behaviour.
- Tests need to be maintained. Like the code you’re testing the tests themselves have to be maintained when refactoring work takes place because they tend to start failing a lot. This maintenance usually comes in two forms: really nasty problem that I’m really glad I found and annoying little deficiency in the test that means it needs tweaking to work. This illustrates that tests themselves can have good and bad design and we can save downstream time (with yet more upfront cost) if we try to make the tests less brittle.
It’s the second and third points that prove to be a tremendous time-sink for me. You want to unit-test the behaviour so you have to create a custom class or mock to do it. Before you know it the mock / custom class requires as much effort as the original code. So my development is half as effective.
I know that all this is an upfront cost that will get repaid many times over later. I even know that the writing of the custom classes will actually help to ensure that there is a clean contract between base classes and their derived classes. And I’ve already mentioned that good test design will save me time later. But all that upfront time adds up and I need to deliver something. Going back and doing it later is just not an option either. Once the code is written the chances that you’ll go back and add that very important unit test are slim. There’s new stuff to deliver.
I don’t think that James Dean would have written software and I’m positive if he had he wouldn’t of written unit-tests. But if I’m to write code that ‘lives’ I need to. It’s just forcing myself to find that time …