But one thing needs to be clarified here: the fear over "killer robots" is that the capacity to make war could be placed largely on autopilot. The danger isn't so much a Terminator-style robotic uprising, which is so farfetched in today's world as to be sensationalistic (if not laughable). Rather, the danger -- a very real one, at that -- lies in an accidental war as the result of malfunctions or data-interpretation errors.
We have more to fear from dumb AI than from smart AI.
As it turns out, the US and the Soviet Union came perilously close to nuclear war on multiple occasions over the result of computer errors. (For an interesting look into the subject, I recommend Eric Schlosser's "Command and Control," a recently released history of nuclear weapons policy over the last 50 years).
Banning autonomous or semi-autonomous tactical systems ("Terminators") seems misguided and impractical. These things will get built, and some of them are being built already. The jury is still out on the ethics of building them (Do they save lives by taking humans off the front lines? Or do they take lives by turning war into a sort of game?). But they're being built, and that genie's not going back in the bottle.
But banning fully autonomous strategic systems ("Skynet") seems more fruitful and worthwhile. Again, this is not because we expect "Skynet" to become self-aware and intentionally initiate a nuclear holocaust. It's because "Skynet" might misinterpret something and accidentally initiate a nuclear holocaust.
(I don't mean to dismiss the very real role of human error in this same domain. But fail-safes, in the form of humans having the final decisionmaking authority over warfare at a strategic level, make a lot of sense.)
I'm a layman here, and I'm way out of my depth in discussing the intricacies of war making AI. But the way I see it, we need to avoid a sort of dangerous middle ground here. Either we keep AI dumb and intentionally non-autonomous, or we make AI a heck of a lot smarter (therefore reducing the risk of fatal errors).
In a weird way, building Skynet might be less risky than getting halfway to Skynet. A relatively dumb AI with full autonomy is a frightening thing. An extremely smart AI with the same degree of autonomy is a little unnerving, perhaps, but it's significantly less dangerous in practical terms.
Agreed. The really interesting point of the "How NOT to build a Terminator" talk was not just that we had to avoid building evil machines, but that we have the potential to build machines with better ethics than human soldiers. Avoid atrocities committed to angry soldiers, avoid accidental wars started by twitchy trigger fingers, avoid intentional wars started by people who weren't authorized to start those wars, that kind of thing. The devil is always in the details, but a ban can only ensure that the people building these weapons are the ones that won't put in the proper effort to make them good.
A strategic AI system commanding human troops is reasonable; the deployment of tactical AI (more honestly dumb targeting and fire-control weapons systems) is not.
Comments
Fascinating stuff, and a great comment overall.
But one thing needs to be clarified here: the fear over "killer robots" is that the capacity to make war could be placed largely on autopilot. The danger isn't so much a Terminator-style robotic uprising, which is so farfetched in today's world as to be sensationalistic (if not laughable). Rather, the danger -- a very real one, at that -- lies in an accidental war as the result of malfunctions or data-interpretation errors.
We have more to fear from dumb AI than from smart AI.
As it turns out, the US and the Soviet Union came perilously close to nuclear war on multiple occasions over the result of computer errors. (For an interesting look into the subject, I recommend Eric Schlosser's "Command and Control," a recently released history of nuclear weapons policy over the last 50 years).
Banning autonomous or semi-autonomous tactical systems ("Terminators") seems misguided and impractical. These things will get built, and some of them are being built already. The jury is still out on the ethics of building them (Do they save lives by taking humans off the front lines? Or do they take lives by turning war into a sort of game?). But they're being built, and that genie's not going back in the bottle.
But banning fully autonomous strategic systems ("Skynet") seems more fruitful and worthwhile. Again, this is not because we expect "Skynet" to become self-aware and intentionally initiate a nuclear holocaust. It's because "Skynet" might misinterpret something and accidentally initiate a nuclear holocaust.
(I don't mean to dismiss the very real role of human error in this same domain. But fail-safes, in the form of humans having the final decisionmaking authority over warfare at a strategic level, make a lot of sense.)
I'm a layman here, and I'm way out of my depth in discussing the intricacies of war making AI. But the way I see it, we need to avoid a sort of dangerous middle ground here. Either we keep AI dumb and intentionally non-autonomous, or we make AI a heck of a lot smarter (therefore reducing the risk of fatal errors).
In a weird way, building Skynet might be less risky than getting halfway to Skynet. A relatively dumb AI with full autonomy is a frightening thing. An extremely smart AI with the same degree of autonomy is a little unnerving, perhaps, but it's significantly less dangerous in practical terms.
Agreed. The really interesting point of the "How NOT to build a Terminator" talk was not just that we had to avoid building evil machines, but that we have the potential to build machines with better ethics than human soldiers. Avoid atrocities committed to angry soldiers, avoid accidental wars started by twitchy trigger fingers, avoid intentional wars started by people who weren't authorized to start those wars, that kind of thing. The devil is always in the details, but a ban can only ensure that the people building these weapons are the ones that won't put in the proper effort to make them good.
I come at it from the other end:
A strategic AI system commanding human troops is reasonable; the deployment of tactical AI (more honestly dumb targeting and fire-control weapons systems) is not.