You can't show a program to bug free. You can't prove a negative.
"the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors."
Translation: in each of the last three versions, only one error was found, in the last 11 versions, only 17 errors were found.
"the shuttle group now finds 85% of its errors before formal testing begins, and 99.9% before the program is delivered"
In order to know you've found x% of the errors in a program, don't you have to have already found 100% of them? If I find 9 errors in a program, how can I know that I've found 90% of the errors unless I also know that there are 10 errors in it?
A lot of what the author says sounds impressive, but I'd argue a lot of it is either wrong or not even wrong (http://en.wikipedia.org/wiki/Not_even_wrong). Even granting the author some artistic license, he's going pretty overboard in a lot of his claims.
I'm sure the software produced by this group is less error-prone than your typical desktop or web app, but it doesn't sound like they're writing innovative software either. Most of what they do sounds like either mathematical calculations or driving hardware devices. Both areas which seem to me to be pretty well explored at this stage.
I would disagree with this. I'm not an expert in this field at all, but from what I've read, the software they write is tested to the point where every single possible input is tested for its assumed output on every subsystem. They write software that is mathematically proven to not fail.
My only point of reference for how they work is, admittedly, this article. I'm far from knowledgeable about formal methods but, in my understanding, if they're writing provably correct programs the code should, by definition, contain no 'errors'. (That is, it should always "do the thing right". There's no guarantee they've designed it to "do the right thing". Design errors may be exactly what the article is talking about.)
I'm not sure exactly what you mean when you say "every single possible input is tested". I would claim that this is not only impractical but impossible. The range of possible inputs to a system is infinite. This may sound pedantic, but in my experience as a software tester all sorts of failures often pop up when you look even slightly outside the range of values that seems sensible/likely/exhaustive for a given input.
Comments
"This software is bug-free"
You can't show a program to bug free. You can't prove a negative.
"the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors."
Translation: in each of the last three versions, only one error was found, in the last 11 versions, only 17 errors were found.
"the shuttle group now finds 85% of its errors before formal testing begins, and 99.9% before the program is delivered"
In order to know you've found x% of the errors in a program, don't you have to have already found 100% of them? If I find 9 errors in a program, how can I know that I've found 90% of the errors unless I also know that there are 10 errors in it?
A lot of what the author says sounds impressive, but I'd argue a lot of it is either wrong or not even wrong (http://en.wikipedia.org/wiki/Not_even_wrong). Even granting the author some artistic license, he's going pretty overboard in a lot of his claims.
I'm sure the software produced by this group is less error-prone than your typical desktop or web app, but it doesn't sound like they're writing innovative software either. Most of what they do sounds like either mathematical calculations or driving hardware devices. Both areas which seem to me to be pretty well explored at this stage.
I would disagree with this. I'm not an expert in this field at all, but from what I've read, the software they write is tested to the point where every single possible input is tested for its assumed output on every subsystem. They write software that is mathematically proven to not fail.
My only point of reference for how they work is, admittedly, this article. I'm far from knowledgeable about formal methods but, in my understanding, if they're writing provably correct programs the code should, by definition, contain no 'errors'. (That is, it should always "do the thing right". There's no guarantee they've designed it to "do the right thing". Design errors may be exactly what the article is talking about.)
I'm not sure exactly what you mean when you say "every single possible input is tested". I would claim that this is not only impractical but impossible. The range of possible inputs to a system is infinite. This may sound pedantic, but in my experience as a software tester all sorts of failures often pop up when you look even slightly outside the range of values that seems sensible/likely/exhaustive for a given input.