I'm not sure what you mean by "TDD is the Cause of the bugs" - the TDD projects had 40-90% fewer bugs. Even if their remaining bugs would have been caught by test coverage of things that occur in live use, that's still a good result, right?
Well, sort of. TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.
So yes, you record fewer bugs, fix those narrow cases quickly and pretend you're done.
Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.
I vastly prefer shakedown tests of the sort where you launch a 'bot army' on cloud servers to thrash the app. Especially if there is Any network/client/server component.
TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.
I think that if you read the article linked, you'd find that the study didn't work that way. Any study that measures different things between experiment and control groups is not going to be sound.
Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.
I've worked on TDD projects, and those are definitely are counted as bugs. Severity 1.
... and then you get a regression test, so that bug never happens again. And the regression test is easy to write, since all of your code is already testable, you have all the testing infrastructure in place, etc.
Never happens again? You already have the 3-alarm fire, its too late. That's my point exactly. You catch only what you can think of and write tests to cover.
I object to the TDD fundamental precept that you code to the tests. That narrows the 'testing' until its a self-fulfilling prophecy that you will have 'fewer bugs', since you defined the bugs as 'what we're looking for'.
Reminds me of the old joke "A: I'm looking for my wallet. B: where did you lose it? A: over there. B: then why look here? A: the light's better"
Comments
I'm not sure what you mean by "TDD is the Cause of the bugs" - the TDD projects had 40-90% fewer bugs. Even if their remaining bugs would have been caught by test coverage of things that occur in live use, that's still a good result, right?
Well, sort of. TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.
So yes, you record fewer bugs, fix those narrow cases quickly and pretend you're done.
Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.
I vastly prefer shakedown tests of the sort where you launch a 'bot army' on cloud servers to thrash the app. Especially if there is Any network/client/server component.
TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.
I think that if you read the article linked, you'd find that the study didn't work that way. Any study that measures different things between experiment and control groups is not going to be sound.
Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.
I've worked on TDD projects, and those are definitely are counted as bugs. Severity 1.
... and then you get a regression test, so that bug never happens again. And the regression test is easy to write, since all of your code is already testable, you have all the testing infrastructure in place, etc.
Never happens again? You already have the 3-alarm fire, its too late. That's my point exactly. You catch only what you can think of and write tests to cover.
I'm not sure how TDD makes this worse. Unless you think that it absolves you of any end-to-end QA testing. In which case, welcome to the fire.
I object to the TDD fundamental precept that you code to the tests. That narrows the 'testing' until its a self-fulfilling prophecy that you will have 'fewer bugs', since you defined the bugs as 'what we're looking for'.
Reminds me of the old joke "A: I'm looking for my wallet. B: where did you lose it? A: over there. B: then why look here? A: the light's better"
You're repeating your false idea that bugs in TDD come only from the unit tests. Eh, see comments above.