This is such a fascinating topic because the field of robotics outside of industrial applications is still so nascent. I'm not here to weigh in, just to drop off some useful tidbits for other people interested in this ethical gray area as well.
This theme of "terminators" and "killer robots" has been really prevalent in the field lately because of the DARPA Robotics Challenge[1], the latest in DARPA's Grand Challenges, which I'm currently participating in on one of the Track B teams. Many people see the break away from bomb-squad bots and factory floor robotic arms in to humanoids as a really scary thing, and the DRC seems to amplify that (if a robot can hold a sawzall, it can hold a rifle).
Just last month in Atlanta, the IEEE Humanoids conference took place. Dr. Ronald C. Arkin[2] gave a talk exactly on this topic from an ethics perspective, titled "How to NOT build a Terminator", and it was exceptional. It was a plenary talk and not a paper, so I can't really find any record of it to share. Unfortunate.
Tangentially, the lab that I work in (the robotics group at the Florida Institute for Human and Machine Cognition[3][4]) has employed and continues to employ a principle that we call "co-active design"[5] where we actively work to keep a human in the loop at all times; we're definitely not looking to build a "killer robot". It's an interesting design problem that overlaps a lot with UI and UX, popular topics here on HN.
And lastly, a shameless plug for the field itself; a lot of people don't realize just how software-oriented robotics research (especially humanoids, where the fun problems are) is. A lot of people are stuck on it being a hardware endeavor. While it's true that a chunk of robotics falls in the mechanical engineering domain, there's plenty of room for hackers from tons of different disciplines to get involved. There's interesting people solving interesting problems, from the Open Source Robotics Foundation (the Willow Garage spin-off) to private groups like Boston Dynamics, yet so many people still see the field as a black-box that only opens up for the hardware inclined. I could see a talented group with the right hacker mindset doing some really interesting stuff in robotics with the right impetus and execution.
But one thing needs to be clarified here: the fear over "killer robots" is that the capacity to make war could be placed largely on autopilot. The danger isn't so much a Terminator-style robotic uprising, which is so farfetched in today's world as to be sensationalistic (if not laughable). Rather, the danger -- a very real one, at that -- lies in an accidental war as the result of malfunctions or data-interpretation errors.
We have more to fear from dumb AI than from smart AI.
As it turns out, the US and the Soviet Union came perilously close to nuclear war on multiple occasions over the result of computer errors. (For an interesting look into the subject, I recommend Eric Schlosser's "Command and Control," a recently released history of nuclear weapons policy over the last 50 years).
Banning autonomous or semi-autonomous tactical systems ("Terminators") seems misguided and impractical. These things will get built, and some of them are being built already. The jury is still out on the ethics of building them (Do they save lives by taking humans off the front lines? Or do they take lives by turning war into a sort of game?). But they're being built, and that genie's not going back in the bottle.
But banning fully autonomous strategic systems ("Skynet") seems more fruitful and worthwhile. Again, this is not because we expect "Skynet" to become self-aware and intentionally initiate a nuclear holocaust. It's because "Skynet" might misinterpret something and accidentally initiate a nuclear holocaust.
(I don't mean to dismiss the very real role of human error in this same domain. But fail-safes, in the form of humans having the final decisionmaking authority over warfare at a strategic level, make a lot of sense.)
I'm a layman here, and I'm way out of my depth in discussing the intricacies of war making AI. But the way I see it, we need to avoid a sort of dangerous middle ground here. Either we keep AI dumb and intentionally non-autonomous, or we make AI a heck of a lot smarter (therefore reducing the risk of fatal errors).
In a weird way, building Skynet might be less risky than getting halfway to Skynet. A relatively dumb AI with full autonomy is a frightening thing. An extremely smart AI with the same degree of autonomy is a little unnerving, perhaps, but it's significantly less dangerous in practical terms.
Agreed. The really interesting point of the "How NOT to build a Terminator" talk was not just that we had to avoid building evil machines, but that we have the potential to build machines with better ethics than human soldiers. Avoid atrocities committed to angry soldiers, avoid accidental wars started by twitchy trigger fingers, avoid intentional wars started by people who weren't authorized to start those wars, that kind of thing. The devil is always in the details, but a ban can only ensure that the people building these weapons are the ones that won't put in the proper effort to make them good.
A strategic AI system commanding human troops is reasonable; the deployment of tactical AI (more honestly dumb targeting and fire-control weapons systems) is not.
I have heard that Gazebo(Simulator) is used extensively for the DARPA challenge, have you had much experience with it? Does it save you any time and how practical/applicable is the simulation compared to the real world?
I had the opportunity to build my own robot in it, the furthest I was able to go was getting 2D navigation working based on some libraries in ROS using a laser scanner. Unfortunately I can't upload that video since my computer was too slow, but I made another one.
We were required to use ROS/Gazebo during the first phase of Track B, but we mostly lean on a simulation environment that was developed by one of our PI's years ago and is maintained/improved in house.
The robotics field has been in the "about to explode" phase for maybe something like 20 years... At any rate, when you get that hacker group together, count me in!
Comments
This is such a fascinating topic because the field of robotics outside of industrial applications is still so nascent. I'm not here to weigh in, just to drop off some useful tidbits for other people interested in this ethical gray area as well.
This theme of "terminators" and "killer robots" has been really prevalent in the field lately because of the DARPA Robotics Challenge[1], the latest in DARPA's Grand Challenges, which I'm currently participating in on one of the Track B teams. Many people see the break away from bomb-squad bots and factory floor robotic arms in to humanoids as a really scary thing, and the DRC seems to amplify that (if a robot can hold a sawzall, it can hold a rifle).
Just last month in Atlanta, the IEEE Humanoids conference took place. Dr. Ronald C. Arkin[2] gave a talk exactly on this topic from an ethics perspective, titled "How to NOT build a Terminator", and it was exceptional. It was a plenary talk and not a paper, so I can't really find any record of it to share. Unfortunate.
Tangentially, the lab that I work in (the robotics group at the Florida Institute for Human and Machine Cognition[3][4]) has employed and continues to employ a principle that we call "co-active design"[5] where we actively work to keep a human in the loop at all times; we're definitely not looking to build a "killer robot". It's an interesting design problem that overlaps a lot with UI and UX, popular topics here on HN.
And lastly, a shameless plug for the field itself; a lot of people don't realize just how software-oriented robotics research (especially humanoids, where the fun problems are) is. A lot of people are stuck on it being a hardware endeavor. While it's true that a chunk of robotics falls in the mechanical engineering domain, there's plenty of room for hackers from tons of different disciplines to get involved. There's interesting people solving interesting problems, from the Open Source Robotics Foundation (the Willow Garage spin-off) to private groups like Boston Dynamics, yet so many people still see the field as a black-box that only opens up for the hardware inclined. I could see a talented group with the right hacker mindset doing some really interesting stuff in robotics with the right impetus and execution.
1:http://theroboticschallenge.org
2:http://en.wikipedia.org/wiki/Ronald_C._Arkin
3:http://ihmc.us
4:http://robots.ihmc.us
5:http://www.jeffreymbradshaw.net/publications/20101008_Coacti...
Fascinating stuff, and a great comment overall.
But one thing needs to be clarified here: the fear over "killer robots" is that the capacity to make war could be placed largely on autopilot. The danger isn't so much a Terminator-style robotic uprising, which is so farfetched in today's world as to be sensationalistic (if not laughable). Rather, the danger -- a very real one, at that -- lies in an accidental war as the result of malfunctions or data-interpretation errors.
We have more to fear from dumb AI than from smart AI.
As it turns out, the US and the Soviet Union came perilously close to nuclear war on multiple occasions over the result of computer errors. (For an interesting look into the subject, I recommend Eric Schlosser's "Command and Control," a recently released history of nuclear weapons policy over the last 50 years).
Banning autonomous or semi-autonomous tactical systems ("Terminators") seems misguided and impractical. These things will get built, and some of them are being built already. The jury is still out on the ethics of building them (Do they save lives by taking humans off the front lines? Or do they take lives by turning war into a sort of game?). But they're being built, and that genie's not going back in the bottle.
But banning fully autonomous strategic systems ("Skynet") seems more fruitful and worthwhile. Again, this is not because we expect "Skynet" to become self-aware and intentionally initiate a nuclear holocaust. It's because "Skynet" might misinterpret something and accidentally initiate a nuclear holocaust.
(I don't mean to dismiss the very real role of human error in this same domain. But fail-safes, in the form of humans having the final decisionmaking authority over warfare at a strategic level, make a lot of sense.)
I'm a layman here, and I'm way out of my depth in discussing the intricacies of war making AI. But the way I see it, we need to avoid a sort of dangerous middle ground here. Either we keep AI dumb and intentionally non-autonomous, or we make AI a heck of a lot smarter (therefore reducing the risk of fatal errors).
In a weird way, building Skynet might be less risky than getting halfway to Skynet. A relatively dumb AI with full autonomy is a frightening thing. An extremely smart AI with the same degree of autonomy is a little unnerving, perhaps, but it's significantly less dangerous in practical terms.
Agreed. The really interesting point of the "How NOT to build a Terminator" talk was not just that we had to avoid building evil machines, but that we have the potential to build machines with better ethics than human soldiers. Avoid atrocities committed to angry soldiers, avoid accidental wars started by twitchy trigger fingers, avoid intentional wars started by people who weren't authorized to start those wars, that kind of thing. The devil is always in the details, but a ban can only ensure that the people building these weapons are the ones that won't put in the proper effort to make them good.
I come at it from the other end:
A strategic AI system commanding human troops is reasonable; the deployment of tactical AI (more honestly dumb targeting and fire-control weapons systems) is not.
I have heard that Gazebo(Simulator) is used extensively for the DARPA challenge, have you had much experience with it? Does it save you any time and how practical/applicable is the simulation compared to the real world?
I had the opportunity to build my own robot in it, the furthest I was able to go was getting 2D navigation working based on some libraries in ROS using a laser scanner. Unfortunately I can't upload that video since my computer was too slow, but I made another one.
http://www.youtube.com/watch?v=WhNGUJjV1zQ
We were required to use ROS/Gazebo during the first phase of Track B, but we mostly lean on a simulation environment that was developed by one of our PI's years ago and is maintained/improved in house.
The robotics field has been in the "about to explode" phase for maybe something like 20 years... At any rate, when you get that hacker group together, count me in!