I disagree with your statement that a codebase without tests cannot safely be refactored.
I've been refactoring code for 20+ years, the overwhelming majority of time without any automated tests, and I'd say offhand 99% of the time it causes no bugs, and in the occasional case where it does cause a bug (because I am imperfect and sometimes make mistakes), I almost always soon find it during the same coding session and fix it.
The key is to understand the code well enough to know what effects what and how. Hold that model in your mind and you're golden. Lots of time saved not writing tests, updating them, fixing them when they break, etc.
Note that this is not an argument against tests in general, just an argument for there being cases where you don't miss them and they would be a net loss if you had them due to all the extra make-work required. I think there's a lot of kool-aid drinking going on among people who themselves probably lacked the ability to do "naked" refactors well. To those folks I say, "Great, have fun storming the castle!" but don't assume that other folks who haven't drunk your kool-aid are constantly banging their head on the wall breaking the code or living in fear of mysterious hypothetical bugs due to a lack of tests. A really excellent 'old fashioned' sort of test is to just run the fricking code -- did it work? did it do what it was supposed to do? data look good? k, move on to the next one of the thousands of other problems you have to solve and tasks you have to do in life. And use version control, so if you retroactively do discover a problem, you can review the diffs, or rollback, or do a tactical patch against the branch, etc.
I do agree with your statement, "By all means, ship. Do what you gotta do." And I agree that that attitude may cause you to at least temporarily incur technical debt, and you generally want to pay that down as soon as feasible. (backing out ugly hacks to replace with more elegant or easier to read implementations, etc.)
It's nice to hear someone with experience from before the "Unit Test is compulsory" explosion. Programmers should always test their work, but testing comes in much more of a diverse range than mere Unit Tests.
There are plenty of cases where Unit Testing is 'embarrasingly'[1] appropriate. These pin-up applications blinds Testing advocates to the fact that Unit Tests are often inferior to other methods or simply not possible.
An example where completely automated tests are impossible is PDF generation. One cannnot 'Unit Test' this. One has to build a framework to take test data, create less than 100 pdfs, and then a human has to eyeball it. Humans cannot eyeball more than 100 images and perceive subtle errors. Less than 100 output images means this cannot exercise every codepath of even simple applications.
Often I was working on a part of the Render Pipeline which was not currently exercised by the existing test. Do I create a whole new test suite to generate test images for each branch condition? If it is important, yes, I created a new end-to-end test. But if it was not, then I addapted some existing test input and used my best judgement and my knoweldge of the internal state. This test did not last beyond my short-term memory, and my own set of eyeballs. If a problem occured later, I would recreate the test from memory.
This is still TDD, but it is so much less straight-jacket than requiring Automated-testing. The tests are 'thrown away' effectively. But the tested code remains. I would also say that the coder knowing which portions to test is superior in many cases.
thanks for backing me up. yeah i often feel un-PC when I say anything bad about unit tests. (Like saying, gee, maybe there are differences between races, or between cultures, or between genders -- Cats, what you say?!?!)
and agreed, there are situations where like you said it's embarrassingly appropriate to have tests. To me the classic case is where you are publishing a code library with thousands of real users across the internet, with real apps built against it already, themselves already in production, etc. It's probably downright stupid of the maintainer to not have a suite of automated tests they can execute, and must pass, before every release, to ensure no regressions. So the maintainers can catch them, and resolve them, before it makes apps break downstream.
But the whole 'you must write tests always, before any application code' thing strikes me as insane and masochistic. :)
Might just be a matter of context and without the context we get the zealotry. Biggest mistake everyone makes when saying "XYZ is teh lames" or "teh wins" is not describing their context.
Our OS team works differently from our apps team for example, because if something in their stuff breaks it's a big deal, they also really worry about backwards compatability and whenever I touch their code I have to create an IPrinter34 to not break things for old clients (who still want their IPrinter12).
In our apps team though we keep things cleaner and kick out the IPrinter23 and keep things as IPrinter so people can read the code more clearly. Backwards compatability isn't so much an issue for us (apart from obviously considering updates) as we release 1 single unit that replaces all our files. If somebody has an old OS then its not supposed to work anyway so the app going titsup is the correct outcome.
Therefore I don't have an opinion on this subject either way, in some scenarios you do IPrinter34 and in others IPrinter.
When people talk with strong opinions on code they should probably start by announcing their own applications of it.
Comments
I disagree with your statement that a codebase without tests cannot safely be refactored.
I've been refactoring code for 20+ years, the overwhelming majority of time without any automated tests, and I'd say offhand 99% of the time it causes no bugs, and in the occasional case where it does cause a bug (because I am imperfect and sometimes make mistakes), I almost always soon find it during the same coding session and fix it.
The key is to understand the code well enough to know what effects what and how. Hold that model in your mind and you're golden. Lots of time saved not writing tests, updating them, fixing them when they break, etc.
Note that this is not an argument against tests in general, just an argument for there being cases where you don't miss them and they would be a net loss if you had them due to all the extra make-work required. I think there's a lot of kool-aid drinking going on among people who themselves probably lacked the ability to do "naked" refactors well. To those folks I say, "Great, have fun storming the castle!" but don't assume that other folks who haven't drunk your kool-aid are constantly banging their head on the wall breaking the code or living in fear of mysterious hypothetical bugs due to a lack of tests. A really excellent 'old fashioned' sort of test is to just run the fricking code -- did it work? did it do what it was supposed to do? data look good? k, move on to the next one of the thousands of other problems you have to solve and tasks you have to do in life. And use version control, so if you retroactively do discover a problem, you can review the diffs, or rollback, or do a tactical patch against the branch, etc.
I do agree with your statement, "By all means, ship. Do what you gotta do." And I agree that that attitude may cause you to at least temporarily incur technical debt, and you generally want to pay that down as soon as feasible. (backing out ugly hacks to replace with more elegant or easier to read implementations, etc.)
Amen,
It's nice to hear someone with experience from before the "Unit Test is compulsory" explosion. Programmers should always test their work, but testing comes in much more of a diverse range than mere Unit Tests.
There are plenty of cases where Unit Testing is 'embarrasingly'[1] appropriate. These pin-up applications blinds Testing advocates to the fact that Unit Tests are often inferior to other methods or simply not possible.
An example where completely automated tests are impossible is PDF generation. One cannnot 'Unit Test' this. One has to build a framework to take test data, create less than 100 pdfs, and then a human has to eyeball it. Humans cannot eyeball more than 100 images and perceive subtle errors. Less than 100 output images means this cannot exercise every codepath of even simple applications.
Often I was working on a part of the Render Pipeline which was not currently exercised by the existing test. Do I create a whole new test suite to generate test images for each branch condition? If it is important, yes, I created a new end-to-end test. But if it was not, then I addapted some existing test input and used my best judgement and my knoweldge of the internal state. This test did not last beyond my short-term memory, and my own set of eyeballs. If a problem occured later, I would recreate the test from memory.
This is still TDD, but it is so much less straight-jacket than requiring Automated-testing. The tests are 'thrown away' effectively. But the tested code remains. I would also say that the coder knowing which portions to test is superior in many cases.
--------------
[1] Similar to :: http://en.wikipedia.org/wiki/Embarrassingly_parallel . Network stacks, account balances, Frameworks are all embarrasingly unit-testable
thanks for backing me up. yeah i often feel un-PC when I say anything bad about unit tests. (Like saying, gee, maybe there are differences between races, or between cultures, or between genders -- Cats, what you say?!?!)
and agreed, there are situations where like you said it's embarrassingly appropriate to have tests. To me the classic case is where you are publishing a code library with thousands of real users across the internet, with real apps built against it already, themselves already in production, etc. It's probably downright stupid of the maintainer to not have a suite of automated tests they can execute, and must pass, before every release, to ensure no regressions. So the maintainers can catch them, and resolve them, before it makes apps break downstream.
But the whole 'you must write tests always, before any application code' thing strikes me as insane and masochistic. :)
Might just be a matter of context and without the context we get the zealotry. Biggest mistake everyone makes when saying "XYZ is teh lames" or "teh wins" is not describing their context.
Our OS team works differently from our apps team for example, because if something in their stuff breaks it's a big deal, they also really worry about backwards compatability and whenever I touch their code I have to create an IPrinter34 to not break things for old clients (who still want their IPrinter12).
In our apps team though we keep things cleaner and kick out the IPrinter23 and keep things as IPrinter so people can read the code more clearly. Backwards compatability isn't so much an issue for us (apart from obviously considering updates) as we release 1 single unit that replaces all our files. If somebody has an old OS then its not supposed to work anyway so the app going titsup is the correct outcome.
Therefore I don't have an opinion on this subject either way, in some scenarios you do IPrinter34 and in others IPrinter.
When people talk with strong opinions on code they should probably start by announcing their own applications of it.