Dijkstra is one of my favorite writers on programming, but I can't resist pointing out that he was on the wrong side of "how to program if you cannot." We have a huge, thriving, valuable, and quite reliable information infrastructure built out of incorrect programs. Dijkstra believed in correct programs and believed that software engineering would not succeed without them. He believed it was counterproductive to teach people to program and employ them as programmers if they could not write correct programs. He believed that the future of computing would be defined by the struggle to write correct programs, and that even if we managed to get by with incorrect programs, the cost would be much greater than if we had achieved the same results via correct programs.
Wow! Can you imagine where we would be if we only had code written by people who had completed rigorous training in mathematics and formal methods? Can you imagine a world where the middle ground between mathematics and the analog world was completely bare? No Excel spreadsheets made by accountants, no customized HTML or CSS, no wiki markup even, because you can't trust amateurs to do programming! It's hard to wrap my head around the idea that The Print Shop for the Apple II should never have existed, because the small minority of people suited for Dijkstra's rigorous training in mathematical programming would surely have been engaged on far more important projects.
Hmm, the essay is dated 1988. I wonder if he still believed that a small class of elite programmers armed with formal methods would have created all of the wonderful programs I used at school around that time -- Print Shop, Where in the World is Carmen Sandiego?, Math Blaster, and on and on -- did he believe his mathematician/programmers would have created all those programs and even more? Or was he simply out of touch? Ah, but how wonderfully and brilliantly out of touch.
He is also wrong about software aging and needing maintenance, as well, but there is an important lesson to learn from what he said. As usual, he is right about the mistake in language even though he is wrong about the phenomenon it refers to: programs are routinely transplanted from one context to another, because it is expensive or inconvenient to maintain their original context. Aging and maintenance are a poor analogy. Using poor language to address something is, in practice, less wrong than simply ignoring it, but we would be better off if we heeded Dijkstra's advice not to lean on bad analogies for novel situations.
Dijkstra was right though. We have credit card thefts, SCADA systems run amuck (Iran's nuclear centrifuges for example) and serious risk to life and limb as a result.
The thing is, we can't measure how much better off society would have been had we baked those rigorous methods into the base of our tools. Or the benefit in increased velocity if we didn't have to come back and fix stupid defects later.
Further, with TDD, BDD, unit tests, etc we're slowly moving towards the world Dijkstra argued for in the first place. Dijkstra could see the Forrest while industry was bedazzled by the trees.
If you want proof, you will always agree with Dijkstra ;-) Unfortunately, I can't summon an alternative universe where Dijkstra got his way. However, I can say that Dijkstra is rolling over in his grave if he heard you compare testing to his approach. To him, empirical approaches were appropriate for the physical world, where items vary, materials deteriorate, and operating conditions are unpredictable. In the digital world, where systems are vastly more complex but made out of ideal, eternal elements, the proper tool is not empiricism, but mathematics. That is Dijkstra's message: mathematics is the right tool for programming, and we are making a costly error by applying empirical engineering methods instead.
Testing is just another messy empirical approach to a mathematical system. The master would be appalled.
I would think that unit tests have nothing to do with Dijsktra's notion of verification. I even seem to recall he had a horror of software testing. If you can prove something, why test it?
I recently read the free Kindle chapter of David Gries' book The Science of Programming, recommended at a previous discussion of this essay:
http://news.ycombinator.com/item?id=1667081
My impression of his thesis so far: unit tests are great, but the challenge is figuring out exactly what unit tests you need to write. Formal methods give you a way of doing that.
It has been written, I think by Dijkstra himself, that he has a (mid 20th century) European perspective on computing: that computers are prohibitively expensive and so we must go as far as we can with math before hitting he hardware. This is contrasted to the American view that hardware is free. Obviously the American view has won out over time.
I've never heard of this difference in perspective between Europa and the US. Do you have anything to support that?
But anyway, this was written in 1988. As someone who was already a programmer at the time, I can assure computers weren't viewed as "prohibitively expensive" in Europe.
Maybe this difference existed ten years earlier, but not in '88.
I thought it had more to do with European CS departments often being considered part of math while American CS departments were often considered part of EE. That would account for some difference in the habit of buying hardware in the beginning (say 60s and 70s): EE already had to buy a lot of that while the math department did not, so it was a new thing for a department that generally did not have high capital expenses.
But I don't think the ability to buy hardware is as relevant as the different mindset between mathematicians, who focus on more abstract problems, and engineers, who focus on more concrete problems, especially today when hardware is cheap. Being an American-trained computer science and spending some time working in Europe, it definitely felt like more emphasis was placed on formalism vs. engineering.
I think it may also be related to a historical experimental/mathematical difference. Edison (US) admired Lord Kelvin (UK), who believed in constructing physical models of theories. He thought you had to be able to see it. Whereas Laplace and other French (EU) mathematicians favoured a symbolic mathematical approach.
Lord Kelvin had a minor win, in that he refuted one of Laplace's mathematical constructs with a model. Edison also had some wins. But of course, Einstein had the biggest win of all, using a symbolic approach (he began with clear visualizations which he later found mathematics for. There were other factors, but once he switched to a purely mathematical approach, he had no more breakthroughs...)
I personally wonder if this US/EU distinction is somehow be related to having all the land already being owned vs. land available.
PS: I'm taking this from biographies of Edison and Einstein, so please correct me if I'm wrong or omitted something relevant.
Implicit in your comment is the claim that the current state of software engineering is good. Just because a lot of helpful/entertaining/profitable software has been written doesn't necessarily mean the industry is in a good place. Especially outside the startup world, I suspect a lot of money and time is wasted because software projects run over budget or over time.
Comments
Dijkstra is one of my favorite writers on programming, but I can't resist pointing out that he was on the wrong side of "how to program if you cannot." We have a huge, thriving, valuable, and quite reliable information infrastructure built out of incorrect programs. Dijkstra believed in correct programs and believed that software engineering would not succeed without them. He believed it was counterproductive to teach people to program and employ them as programmers if they could not write correct programs. He believed that the future of computing would be defined by the struggle to write correct programs, and that even if we managed to get by with incorrect programs, the cost would be much greater than if we had achieved the same results via correct programs.
Wow! Can you imagine where we would be if we only had code written by people who had completed rigorous training in mathematics and formal methods? Can you imagine a world where the middle ground between mathematics and the analog world was completely bare? No Excel spreadsheets made by accountants, no customized HTML or CSS, no wiki markup even, because you can't trust amateurs to do programming! It's hard to wrap my head around the idea that The Print Shop for the Apple II should never have existed, because the small minority of people suited for Dijkstra's rigorous training in mathematical programming would surely have been engaged on far more important projects.
Hmm, the essay is dated 1988. I wonder if he still believed that a small class of elite programmers armed with formal methods would have created all of the wonderful programs I used at school around that time -- Print Shop, Where in the World is Carmen Sandiego?, Math Blaster, and on and on -- did he believe his mathematician/programmers would have created all those programs and even more? Or was he simply out of touch? Ah, but how wonderfully and brilliantly out of touch.
He is also wrong about software aging and needing maintenance, as well, but there is an important lesson to learn from what he said. As usual, he is right about the mistake in language even though he is wrong about the phenomenon it refers to: programs are routinely transplanted from one context to another, because it is expensive or inconvenient to maintain their original context. Aging and maintenance are a poor analogy. Using poor language to address something is, in practice, less wrong than simply ignoring it, but we would be better off if we heeded Dijkstra's advice not to lean on bad analogies for novel situations.
Dijkstra was right though. We have credit card thefts, SCADA systems run amuck (Iran's nuclear centrifuges for example) and serious risk to life and limb as a result.
The thing is, we can't measure how much better off society would have been had we baked those rigorous methods into the base of our tools. Or the benefit in increased velocity if we didn't have to come back and fix stupid defects later.
Further, with TDD, BDD, unit tests, etc we're slowly moving towards the world Dijkstra argued for in the first place. Dijkstra could see the Forrest while industry was bedazzled by the trees.
In short, you claim we have it better. Prove it.
If you want proof, you will always agree with Dijkstra ;-) Unfortunately, I can't summon an alternative universe where Dijkstra got his way. However, I can say that Dijkstra is rolling over in his grave if he heard you compare testing to his approach. To him, empirical approaches were appropriate for the physical world, where items vary, materials deteriorate, and operating conditions are unpredictable. In the digital world, where systems are vastly more complex but made out of ideal, eternal elements, the proper tool is not empiricism, but mathematics. That is Dijkstra's message: mathematics is the right tool for programming, and we are making a costly error by applying empirical engineering methods instead.
Testing is just another messy empirical approach to a mathematical system. The master would be appalled.
I would think that unit tests have nothing to do with Dijsktra's notion of verification. I even seem to recall he had a horror of software testing. If you can prove something, why test it?
I recently read the free Kindle chapter of David Gries' book The Science of Programming, recommended at a previous discussion of this essay: http://news.ycombinator.com/item?id=1667081
My impression of his thesis so far: unit tests are great, but the challenge is figuring out exactly what unit tests you need to write. Formal methods give you a way of doing that.
His opinion on tests: they can only show the presence of bugs, not their absence.
That statement means: "they can only show the presence of 0 or more bugs, not the absence of any bugs". That's not just his opinion: that's a fact.
It has been written, I think by Dijkstra himself, that he has a (mid 20th century) European perspective on computing: that computers are prohibitively expensive and so we must go as far as we can with math before hitting he hardware. This is contrasted to the American view that hardware is free. Obviously the American view has won out over time.
I've never heard of this difference in perspective between Europa and the US. Do you have anything to support that?
But anyway, this was written in 1988. As someone who was already a programmer at the time, I can assure computers weren't viewed as "prohibitively expensive" in Europe.
Maybe this difference existed ten years earlier, but not in '88.
I thought it had more to do with European CS departments often being considered part of math while American CS departments were often considered part of EE. That would account for some difference in the habit of buying hardware in the beginning (say 60s and 70s): EE already had to buy a lot of that while the math department did not, so it was a new thing for a department that generally did not have high capital expenses.
But I don't think the ability to buy hardware is as relevant as the different mindset between mathematicians, who focus on more abstract problems, and engineers, who focus on more concrete problems, especially today when hardware is cheap. Being an American-trained computer science and spending some time working in Europe, it definitely felt like more emphasis was placed on formalism vs. engineering.
I think it may also be related to a historical experimental/mathematical difference. Edison (US) admired Lord Kelvin (UK), who believed in constructing physical models of theories. He thought you had to be able to see it. Whereas Laplace and other French (EU) mathematicians favoured a symbolic mathematical approach.
Lord Kelvin had a minor win, in that he refuted one of Laplace's mathematical constructs with a model. Edison also had some wins. But of course, Einstein had the biggest win of all, using a symbolic approach (he began with clear visualizations which he later found mathematics for. There were other factors, but once he switched to a purely mathematical approach, he had no more breakthroughs...)
I personally wonder if this US/EU distinction is somehow be related to having all the land already being owned vs. land available.
PS: I'm taking this from biographies of Edison and Einstein, so please correct me if I'm wrong or omitted something relevant.
Sounds like the familiar contrast between continental rationalism and Anglo-American empiricism.
Implicit in your comment is the claim that the current state of software engineering is good. Just because a lot of helpful/entertaining/profitable software has been written doesn't necessarily mean the industry is in a good place. Especially outside the startup world, I suspect a lot of money and time is wasted because software projects run over budget or over time.