No. We don't. As Alan Kay said, programmers don't have a solid concept like an arch to build on. In real engineering, when you build a bridge, a single rusty rivet won't cause it to fall down. In software, a single pair of misplaced braces give you critical "goto fail" bugs. (And don't fool yourself into thinking that static analysis catches all these errors.)
It has become fashionable to say that automated tests somehow turn programming into a rigorous discipline. This is bullshit. Tests are a decent band-aid which kinda help sometimes if you spend half your programming time writing them. And will still miss plenty of horrible bugs. In truth, programming remains a poorly-understood endeavor. Checklists as auto-written automatic tests might actually be a step in the right direction, especially if compilers start to analyze the kinds of mistakes individual programmers tend to make.
In real engineering, when you build a bridge, a single rusty rivet won't cause it to fall down. In software, a single pair of misplaced braces give you critical "goto fail" bugs.
In real engineering, you design things to be built out of a whole bunch of tiny, redundant components, each of which might or might not fail catastrophically with a given probability, but where the failure of any given component won't bring down anything else. You also source your components such that the failure of one component won't increase the likelihood of a global failure of all instances of that component in your design--you don't get all your rivets from the same manufacturing batch, etc.
So, if we wanted to do software engineering the way we did other kinds of engineering, we'd have to have our computer systems resemble the physical, componentized systems that engineering principles actually apply to. I think this would involve, at least, two things:
1. Building only tiny components that could fail/crash without bringing down the entire project. Most usually, this is accomplished by separating the components into independent networked services.
2. Building several independent implementations of each component, and multiplexing requests to them using worker-pools (with client retry) and quorum-agreement protocols.
I was trying to think of what in software is the equivilent as a 'rivet'. Something small, with a near singular purpose, one who's design has changed relatively little in decades. The only thing I could argue we have in software engineering is perhaps machine operands. For example, just about every processor will have a register and can a operand that will increment that register by one.
The result of this is that most software projects are as complex as the most advanced 'real engineered' projects. As software engineers we design whole "space shuttles" in a month. The 'buggyness' of software engineering compared to 'real engineering' is the manifestation of the complexity of what software engineering does. The tools and processes of 'real engineer' are completely ill-adapted to the rapid development of the massive levels of complexity that is software.
However, there are certain projects where a developer examines every instruction, at the detail that the 'real engineering' principles can help, however, the tools at the forefront in software engineering enable the addition of billions of washers to billions of rivets in days if not hours of design.
No. We don't. As Alan Kay said, programmers don't have a solid concept like an arch to build on. In real engineering, when you build a bridge, a single rusty rivet won't cause it to fall down. In software, a single pair of misplaced braces give you critical "goto fail" bugs. (And don't fool yourself into thinking that static analysis catches all these errors.)
Yes, we do. Misplaced braces can cause catastrophic failure, but that's precisely because engineering principles were not followed; the system was left with a critical vulnerability which meant that a typo could cause catastrophic failure.
We can, however, build systems that don't suffer from these problems. Look at the Shuttle, for instance; multiple redundant machines making the same calculations, and additional machines running totally independently developed software. AFAIK the system has never failed.
So I dispute that there's a problem here in terms of our ability to develop critical software with engineering-level quality. The problem is that software in an order of magnitude more complex to reason about than e.g. a bridge, and as a result developing software to the same level of quality would in most cases be a waste of time.
software [is] an order of magnitude more complex to reason about than e.g. a bridge
I think you're trying to flatter our profession. Software is not more complex; it's just that we don't know what we're doing well enough to have useable, reliable components. Like arches. Or even just rivets.
To stretch the bridge analogy further (probably almost to its breaking point), it's as if I'm about to start work on the Hoover Dam Bridge, but I have to start by getting some rivets. So I first go out and pick up some pieces of iron ore, and start trying to figure out how to build a smelter that can introduce just the right amount of carbon into them to make decent steel. Knowing a bit about smelter designs, I decide to use a Japanese tatara furnace. I still don't have rivets, but I figure that in a few weeks I'll have enough steel to start making them. (After I make the buy-or-build-build decision on the furnace, that is. I'm not allowed to reuse the one I made for my last project of building music stands, and the commonly-used open-source design has been known to mysteriously turn iron ore into lead instead.) The rest of the bridge has yet to be tackled...
Point being: this is not a "we know enough, it's just complicated" problem. We know almost nothing and our tools are appallingly primitive. To quote some classic hacker humor: "I was taught assembler / In my second year of school / It's kinda like construction work / With a toothpick for a tool." Assembler and Haskell have much more in common than a toothpick and a Caterpillar D9.
> The problem is that software in an order of magnitude
> more complex to reason about than e.g. a bridge, and
> as a result developing software to the same level of
> quality would in most cases be a waste of time.
That's only true while we continue to develop software the way we do. If we change the way we develop software, if we create new tools, if we follow different processes, if the discipline grows up and leaves adolescence, then the cost-benefit balance will change, and maybe we can get software developed at a more reasonable cost, and also with fewer bugs.
This is already happening with unit tests, regression tests, warnings in compilers, and languages that remove entire classes of errors, but still some remain. More, some of those that remain are catchable, if only we extended our checking and processes to capture them. That seems to me what this article is about - finding ways of enhancing existing techniques in a way that gives more benefit than costs.
Say what you like about "move fast and break things", following that idea, things are getting broken and need fixing! If the balance is changed then we can move just as fast, break fewer things, and then re-allocate those resources that would have spent fixing them, deploying them instead in more profitable ways.
Comments
No. We don't. As Alan Kay said, programmers don't have a solid concept like an arch to build on. In real engineering, when you build a bridge, a single rusty rivet won't cause it to fall down. In software, a single pair of misplaced braces give you critical "goto fail" bugs. (And don't fool yourself into thinking that static analysis catches all these errors.)
It has become fashionable to say that automated tests somehow turn programming into a rigorous discipline. This is bullshit. Tests are a decent band-aid which kinda help sometimes if you spend half your programming time writing them. And will still miss plenty of horrible bugs. In truth, programming remains a poorly-understood endeavor. Checklists as auto-written automatic tests might actually be a step in the right direction, especially if compilers start to analyze the kinds of mistakes individual programmers tend to make.
In real engineering, you design things to be built out of a whole bunch of tiny, redundant components, each of which might or might not fail catastrophically with a given probability, but where the failure of any given component won't bring down anything else. You also source your components such that the failure of one component won't increase the likelihood of a global failure of all instances of that component in your design--you don't get all your rivets from the same manufacturing batch, etc.
So, if we wanted to do software engineering the way we did other kinds of engineering, we'd have to have our computer systems resemble the physical, componentized systems that engineering principles actually apply to. I think this would involve, at least, two things:
1. Building only tiny components that could fail/crash without bringing down the entire project. Most usually, this is accomplished by separating the components into independent networked services.
2. Building several independent implementations of each component, and multiplexing requests to them using worker-pools (with client retry) and quorum-agreement protocols.
I was trying to think of what in software is the equivilent as a 'rivet'. Something small, with a near singular purpose, one who's design has changed relatively little in decades. The only thing I could argue we have in software engineering is perhaps machine operands. For example, just about every processor will have a register and can a operand that will increment that register by one.
The result of this is that most software projects are as complex as the most advanced 'real engineered' projects. As software engineers we design whole "space shuttles" in a month. The 'buggyness' of software engineering compared to 'real engineering' is the manifestation of the complexity of what software engineering does. The tools and processes of 'real engineer' are completely ill-adapted to the rapid development of the massive levels of complexity that is software.
However, there are certain projects where a developer examines every instruction, at the detail that the 'real engineering' principles can help, however, the tools at the forefront in software engineering enable the addition of billions of washers to billions of rivets in days if not hours of design.
No. We don't. As Alan Kay said, programmers don't have a solid concept like an arch to build on. In real engineering, when you build a bridge, a single rusty rivet won't cause it to fall down. In software, a single pair of misplaced braces give you critical "goto fail" bugs. (And don't fool yourself into thinking that static analysis catches all these errors.)
Yes, we do. Misplaced braces can cause catastrophic failure, but that's precisely because engineering principles were not followed; the system was left with a critical vulnerability which meant that a typo could cause catastrophic failure.
We can, however, build systems that don't suffer from these problems. Look at the Shuttle, for instance; multiple redundant machines making the same calculations, and additional machines running totally independently developed software. AFAIK the system has never failed.
So I dispute that there's a problem here in terms of our ability to develop critical software with engineering-level quality. The problem is that software in an order of magnitude more complex to reason about than e.g. a bridge, and as a result developing software to the same level of quality would in most cases be a waste of time.
I think you're trying to flatter our profession. Software is not more complex; it's just that we don't know what we're doing well enough to have useable, reliable components. Like arches. Or even just rivets.
To stretch the bridge analogy further (probably almost to its breaking point), it's as if I'm about to start work on the Hoover Dam Bridge, but I have to start by getting some rivets. So I first go out and pick up some pieces of iron ore, and start trying to figure out how to build a smelter that can introduce just the right amount of carbon into them to make decent steel. Knowing a bit about smelter designs, I decide to use a Japanese tatara furnace. I still don't have rivets, but I figure that in a few weeks I'll have enough steel to start making them. (After I make the buy-or-build-build decision on the furnace, that is. I'm not allowed to reuse the one I made for my last project of building music stands, and the commonly-used open-source design has been known to mysteriously turn iron ore into lead instead.) The rest of the bridge has yet to be tackled...
Point being: this is not a "we know enough, it's just complicated" problem. We know almost nothing and our tools are appallingly primitive. To quote some classic hacker humor: "I was taught assembler / In my second year of school / It's kinda like construction work / With a toothpick for a tool." Assembler and Haskell have much more in common than a toothpick and a Caterpillar D9.
This is already happening with unit tests, regression tests, warnings in compilers, and languages that remove entire classes of errors, but still some remain. More, some of those that remain are catchable, if only we extended our checking and processes to capture them. That seems to me what this article is about - finding ways of enhancing existing techniques in a way that gives more benefit than costs.
Say what you like about "move fast and break things", following that idea, things are getting broken and need fixing! If the balance is changed then we can move just as fast, break fewer things, and then re-allocate those resources that would have spent fixing them, deploying them instead in more profitable ways.