It's nice to hear someone with experience from before the "Unit Test is compulsory" explosion. Programmers should always test their work, but testing comes in much more of a diverse range than mere Unit Tests.
There are plenty of cases where Unit Testing is 'embarrasingly'[1] appropriate. These pin-up applications blinds Testing advocates to the fact that Unit Tests are often inferior to other methods or simply not possible.
An example where completely automated tests are impossible is PDF generation. One cannnot 'Unit Test' this. One has to build a framework to take test data, create less than 100 pdfs, and then a human has to eyeball it. Humans cannot eyeball more than 100 images and perceive subtle errors. Less than 100 output images means this cannot exercise every codepath of even simple applications.
Often I was working on a part of the Render Pipeline which was not currently exercised by the existing test. Do I create a whole new test suite to generate test images for each branch condition? If it is important, yes, I created a new end-to-end test. But if it was not, then I addapted some existing test input and used my best judgement and my knoweldge of the internal state. This test did not last beyond my short-term memory, and my own set of eyeballs. If a problem occured later, I would recreate the test from memory.
This is still TDD, but it is so much less straight-jacket than requiring Automated-testing. The tests are 'thrown away' effectively. But the tested code remains. I would also say that the coder knowing which portions to test is superior in many cases.
thanks for backing me up. yeah i often feel un-PC when I say anything bad about unit tests. (Like saying, gee, maybe there are differences between races, or between cultures, or between genders -- Cats, what you say?!?!)
and agreed, there are situations where like you said it's embarrassingly appropriate to have tests. To me the classic case is where you are publishing a code library with thousands of real users across the internet, with real apps built against it already, themselves already in production, etc. It's probably downright stupid of the maintainer to not have a suite of automated tests they can execute, and must pass, before every release, to ensure no regressions. So the maintainers can catch them, and resolve them, before it makes apps break downstream.
But the whole 'you must write tests always, before any application code' thing strikes me as insane and masochistic. :)
Might just be a matter of context and without the context we get the zealotry. Biggest mistake everyone makes when saying "XYZ is teh lames" or "teh wins" is not describing their context.
Our OS team works differently from our apps team for example, because if something in their stuff breaks it's a big deal, they also really worry about backwards compatability and whenever I touch their code I have to create an IPrinter34 to not break things for old clients (who still want their IPrinter12).
In our apps team though we keep things cleaner and kick out the IPrinter23 and keep things as IPrinter so people can read the code more clearly. Backwards compatability isn't so much an issue for us (apart from obviously considering updates) as we release 1 single unit that replaces all our files. If somebody has an old OS then its not supposed to work anyway so the app going titsup is the correct outcome.
Therefore I don't have an opinion on this subject either way, in some scenarios you do IPrinter34 and in others IPrinter.
When people talk with strong opinions on code they should probably start by announcing their own applications of it.
Comments
Amen,
It's nice to hear someone with experience from before the "Unit Test is compulsory" explosion. Programmers should always test their work, but testing comes in much more of a diverse range than mere Unit Tests.
There are plenty of cases where Unit Testing is 'embarrasingly'[1] appropriate. These pin-up applications blinds Testing advocates to the fact that Unit Tests are often inferior to other methods or simply not possible.
An example where completely automated tests are impossible is PDF generation. One cannnot 'Unit Test' this. One has to build a framework to take test data, create less than 100 pdfs, and then a human has to eyeball it. Humans cannot eyeball more than 100 images and perceive subtle errors. Less than 100 output images means this cannot exercise every codepath of even simple applications.
Often I was working on a part of the Render Pipeline which was not currently exercised by the existing test. Do I create a whole new test suite to generate test images for each branch condition? If it is important, yes, I created a new end-to-end test. But if it was not, then I addapted some existing test input and used my best judgement and my knoweldge of the internal state. This test did not last beyond my short-term memory, and my own set of eyeballs. If a problem occured later, I would recreate the test from memory.
This is still TDD, but it is so much less straight-jacket than requiring Automated-testing. The tests are 'thrown away' effectively. But the tested code remains. I would also say that the coder knowing which portions to test is superior in many cases.
--------------
[1] Similar to :: http://en.wikipedia.org/wiki/Embarrassingly_parallel . Network stacks, account balances, Frameworks are all embarrasingly unit-testable
thanks for backing me up. yeah i often feel un-PC when I say anything bad about unit tests. (Like saying, gee, maybe there are differences between races, or between cultures, or between genders -- Cats, what you say?!?!)
and agreed, there are situations where like you said it's embarrassingly appropriate to have tests. To me the classic case is where you are publishing a code library with thousands of real users across the internet, with real apps built against it already, themselves already in production, etc. It's probably downright stupid of the maintainer to not have a suite of automated tests they can execute, and must pass, before every release, to ensure no regressions. So the maintainers can catch them, and resolve them, before it makes apps break downstream.
But the whole 'you must write tests always, before any application code' thing strikes me as insane and masochistic. :)
Might just be a matter of context and without the context we get the zealotry. Biggest mistake everyone makes when saying "XYZ is teh lames" or "teh wins" is not describing their context.
Our OS team works differently from our apps team for example, because if something in their stuff breaks it's a big deal, they also really worry about backwards compatability and whenever I touch their code I have to create an IPrinter34 to not break things for old clients (who still want their IPrinter12).
In our apps team though we keep things cleaner and kick out the IPrinter23 and keep things as IPrinter so people can read the code more clearly. Backwards compatability isn't so much an issue for us (apart from obviously considering updates) as we release 1 single unit that replaces all our files. If somebody has an old OS then its not supposed to work anyway so the app going titsup is the correct outcome.
Therefore I don't have an opinion on this subject either way, in some scenarios you do IPrinter34 and in others IPrinter.
When people talk with strong opinions on code they should probably start by announcing their own applications of it.