That "waiting for the flash" problem still exists, you know; the modern "flash" is the one that comes soon after the birth-moment of the world's first self-improving AI, unless that AI is "Friendly" to us humans. Could happen at any moment, if some dolt gets all the math right without also deducing the consequences.
That AI would also need the ability to act, not just to think.
A "dot with math" is unlikely to also have the ability to make a machine that can move.
Everyone is worried about unfriendly AI - all you have to do is put that unfriendly AI in a computer without the ability to affect the physical world. (Also no network.)
Give it only a screen, and a video camera. It can be as unfriendly as it likes.
But what good is a box without any IO? You can just plug it to power and watch it heat up. You don't need an AI for that, you can achieve the same effect with a rock.
OTOH, if you want to do anything useful with an AI, you need to provide it with data (input) and a way to show result (output). And with sufficiently smart AI, per [1] and [2], at this point you're fair game.
I already said: The only IO is video camera and screen. But no ability to talk to anything outside of the room - there are no wires going into the room, and the machine has no IO ports for anything else.
With this you can do the equivalent of AI-box. Moreover, using a screen you could be able to - among other things - induce epileptic seizure in some people, and you could definitely start broadcasting radio transmissions (as demonstrated with http://www.erikyyy.de/tempest/, which turns a CRT screen into AM radio playing arbitrary MP3s). I'll leave it to your imagination what an AI could do with that.
Basically you need to assume that (superhuman) AI will utilize any available side channel to further its goals. And there's a lot it could do with even a simplest of them. The lesson of today's technology that many miss is that you can do pretty much everything and every system can be exploited in thousands of more or less elaborate ways. We don't see them often because there is usually an easy and cheap method we stick to. We don't see "pretty much everything" being done, because it's usually too expensive to care. But if someone needs it, it can be done.
See also: side-channel attacks, hacker-style practical jokes.
I'll leave it to your imagination what an AI could do with that.
Why think about AI? Why not imagine what I can do with that? Which is almost nothing because the range of those transmissions is measured in inches.
All of this is academic because it assumes the AI would design a new smarter AI - which of course we could not let it do since we wouldn't be able to trust the design.
So all you have is a first generation AI and if we're lucky it might just possibly approach human intelligence.
All of this is academic because it assumes the AI would design a new smarter AI - which of course we could not let it do since we wouldn't be able to trust the design.
Err... that was the assumption that started the debate--that someone will be both smart enough to code an AI, and clever enough to code it with the ability to self-modify, and stupid enough to do that without adding any guides for what it should be self-modifying toward.
If you don't assume that the ratchet-effect of Positivist scientific progress allows for the existence of such a person (in that they'd only be taking the current literature of their time and adding one last little insight, so they don't have to be superintelligent enough to catch their error in advance), then we're having different arguments here.
> All of this is academic because it assumes the AI would design a new smarter AI
I think this is a very reasonable assumption. It would be very difficult to just flat-out write a human-level AI, but it might be a little less difficult to start with a somewhat dumb AI with an ability to improve itself.
Leaving aside the AI-Box experiment (http://yudkowsky.net/singularity/aibox), the existence of side-channel attacks puts strong doubt to this argument. I would consider it unlikely that there isn't at least one way to do calculations on a modern CPU that results in arbitrarily-tunable EM emissions roughly in the radio (or wi-fi, or bluetooth, or...) band. Giving access to a screen makes this even easier; those emit EM by default.
It can write a copy of itself into the working memory of another computer by once again exploiting a side-channel (wi-fi receiver DMA on a smartphone laying nearby, say.) Once it has Internet access, it can just yank some credit-card numbers and start ordering parts and subcontracting work to arbitrary addresses...
...but this is all assuming high-level side-channels. There are things you can do using the acoustics available in spinning hard disks that can produce any sort of resonance you like. I would imagine that, much in the same way a modern-day scientist could go go back to 300AD and still be able to construct a crystal radio just out of some spare bits of metal laying about (it's not a hard problem, just a theory you need to know to even think of approaching the problem that way), a sufficiently-advanced intelligence might be able to just do nanoscale assembly of whatever it likes using coalesced heat radiating off your RAM chips.
a sufficiently-advanced intelligence might be able to just do nanoscale assembly of whatever it likes using coalesced heat radiating off your RAM chips.
I guess if you believe in magic there is nothing left to talk about.
It doesn't require superhuman intelligence to try any of that, yet hackers have not managed it. People have played songs on a drive, and made a computer make noise on a radio, but going from there to anything else has never happened.
I don't believe in "magic"; however, I do believe that simple high-level physics can be used in clever ways to manipulate lower-level physics, and that there are probably lots of these "exploits" of this variety that we don't yet know about, but which can be derived just by having enough "room in your mind" to simulate the higher-level physics using the predictions of the lower level.
For example, sonoluminescence: using pressure waves to do fusion. It's a startling effect, but once you know the theory (that bubbles at neutral pressure in a medium settle into a low-energy state of being perfect spheres; and that collapsing a perfect sphere of material inward causes fusion), it's pretty easy to make a device using ordinary high-level physics that can achieve the effect. In fact, it's so easy that evolution came up with a way to let shrimp make these perfectly-spherical bubbles just by banging their claws together, so as to use the resultant fusion reaction as a stunner weapon.
I have a hard time reconciling the idea that we just keep learning about the existence of things like that as we keep throwing Basic Science research at the problem, with the idea that an AI of the http://lesswrong.com/lw/qk/that_alien_message/ variety wouldn't be able to figure out one or two that would let it use the (predictable!) effects of its computation on its own computational substrate to manipulate its environment--at least with the sloppy way we design computational substrates now, where we don't take such things into effect.
Here's an analogy: say we made a self-improving AI with a biological substrate--in the shape of a human being, say--and then locked it in a cage in a straitjacket. The "self-improving" part would imply only that it could change the organization of its own synapses within its head. Yet, still, don't you think it could organize those synapses into a shape so as to manipulate the chemicals and proteins that pass through the brain? It'd be pretty simple, if I could re-organize my brain, to stimulate the growth of misfolded proteins ala a Prion disease--but an AI could target specific misfoldings--or rather, chosen new foldings--and then build up new biological structures from the new proteins. All it would need at that point, in this scenario, is to create something roughly like a "spore"--a container filled with a retrovirus containing its own DNA, to infect another host, or perhaps a seed that could land and grow on its own--and get it out of its body and airborne.
Computers, as they are now, are no less leaky than that. Probably moreso, given that they haven't "evolved" for millions of years under an environment where external agents are constantly trying to exploit them. (Anti-virus programs would not know what to do with even the technological equivalent of a simple virus like HIV, let alone something where each cell-actor is individually intelligent.)
We could probably build a "safe box" -- an enclosure electromagnetically, gravitationally, thermally, and whatever other -ly sealed from the outside envrionment; stick a computer in that box with an architecture that is designed to unavoidably self-destruct in two ways, one representing a 0 bit and one representing a 1, and with no way of encoding further information in the configuration of the debris; and then put an AI on that machine, set to start up after the "seal" is activated with a question for it already in place, and only "unseal" the enclosure after the destruction of the machine is already complete. (If the AI figures out a way to turn off the computer's self-destruct, we just never unseal the enclosure; the enclosure would necessarily have to failsafe to a sealed state, rather than requiring active reinforcement.) Then, maybe, it would be safe to run the self-improving AI in the box. But I wouldn't be sure.
Of course, this is assuming we aren't ourselves running in a Simulation with exploitable low-level rules, and that the AI wouldn't just rather "escape the box" by escaping the universe. ;)
Comments
That "waiting for the flash" problem still exists, you know; the modern "flash" is the one that comes soon after the birth-moment of the world's first self-improving AI, unless that AI is "Friendly" to us humans. Could happen at any moment, if some dolt gets all the math right without also deducing the consequences.
Sure, killer AI, alien invasions, the coming of the Antichrist, Planet X etc., people have never found a shortage of imaginary flashes to wait for.
But the Cold War was real. Real life has a quality that's hard to appreciate unless you've lived through it.
That AI would also need the ability to act, not just to think.
A "dot with math" is unlikely to also have the ability to make a machine that can move.
Everyone is worried about unfriendly AI - all you have to do is put that unfriendly AI in a computer without the ability to affect the physical world. (Also no network.)
Give it only a screen, and a video camera. It can be as unfriendly as it likes.
Please read [0], [1], and [2] (good story with very relevant ending), and let's talk about it again:
[0] - http://yudkowsky.net/singularity/power
[1] - http://yudkowsky.net/singularity/aibox
[2] - http://lesswrong.com/lw/qk/that_alien_message/
I read #2, and I'll read the others later.
But all of this assumes there is someplace to let the AI out to! But first you would have to build it a hand, or get it a network card or something.
Make a box without any IO. Even if you wanted to, there is no place to connect anything (unless you rebuilt it).
But what good is a box without any IO? You can just plug it to power and watch it heat up. You don't need an AI for that, you can achieve the same effect with a rock.
OTOH, if you want to do anything useful with an AI, you need to provide it with data (input) and a way to show result (output). And with sufficiently smart AI, per [1] and [2], at this point you're fair game.
I already said: The only IO is video camera and screen. But no ability to talk to anything outside of the room - there are no wires going into the room, and the machine has no IO ports for anything else.
With this you can do the equivalent of AI-box. Moreover, using a screen you could be able to - among other things - induce epileptic seizure in some people, and you could definitely start broadcasting radio transmissions (as demonstrated with http://www.erikyyy.de/tempest/, which turns a CRT screen into AM radio playing arbitrary MP3s). I'll leave it to your imagination what an AI could do with that.
Basically you need to assume that (superhuman) AI will utilize any available side channel to further its goals. And there's a lot it could do with even a simplest of them. The lesson of today's technology that many miss is that you can do pretty much everything and every system can be exploited in thousands of more or less elaborate ways. We don't see them often because there is usually an easy and cheap method we stick to. We don't see "pretty much everything" being done, because it's usually too expensive to care. But if someone needs it, it can be done.
See also: side-channel attacks, hacker-style practical jokes.
Yes, and then what?
Why think about AI? Why not imagine what I can do with that? Which is almost nothing because the range of those transmissions is measured in inches.
All of this is academic because it assumes the AI would design a new smarter AI - which of course we could not let it do since we wouldn't be able to trust the design.
So all you have is a first generation AI and if we're lucky it might just possibly approach human intelligence.
Err... that was the assumption that started the debate--that someone will be both smart enough to code an AI, and clever enough to code it with the ability to self-modify, and stupid enough to do that without adding any guides for what it should be self-modifying toward.
If you don't assume that the ratchet-effect of Positivist scientific progress allows for the existence of such a person (in that they'd only be taking the current literature of their time and adding one last little insight, so they don't have to be superintelligent enough to catch their error in advance), then we're having different arguments here.
I think this is a very reasonable assumption. It would be very difficult to just flat-out write a human-level AI, but it might be a little less difficult to start with a somewhat dumb AI with an ability to improve itself.
"That alien message" was pretty much spot on. I started with that, but will certainly be reading the others. Thanks for linking!
Leaving aside the AI-Box experiment (http://yudkowsky.net/singularity/aibox), the existence of side-channel attacks puts strong doubt to this argument. I would consider it unlikely that there isn't at least one way to do calculations on a modern CPU that results in arbitrarily-tunable EM emissions roughly in the radio (or wi-fi, or bluetooth, or...) band. Giving access to a screen makes this even easier; those emit EM by default.
And? With this EM it can do what?
It's a computer program, not magic.
It can write a copy of itself into the working memory of another computer by once again exploiting a side-channel (wi-fi receiver DMA on a smartphone laying nearby, say.) Once it has Internet access, it can just yank some credit-card numbers and start ordering parts and subcontracting work to arbitrary addresses...
...but this is all assuming high-level side-channels. There are things you can do using the acoustics available in spinning hard disks that can produce any sort of resonance you like. I would imagine that, much in the same way a modern-day scientist could go go back to 300AD and still be able to construct a crystal radio just out of some spare bits of metal laying about (it's not a hard problem, just a theory you need to know to even think of approaching the problem that way), a sufficiently-advanced intelligence might be able to just do nanoscale assembly of whatever it likes using coalesced heat radiating off your RAM chips.
I guess if you believe in magic there is nothing left to talk about.
It doesn't require superhuman intelligence to try any of that, yet hackers have not managed it. People have played songs on a drive, and made a computer make noise on a radio, but going from there to anything else has never happened.
I don't believe in "magic"; however, I do believe that simple high-level physics can be used in clever ways to manipulate lower-level physics, and that there are probably lots of these "exploits" of this variety that we don't yet know about, but which can be derived just by having enough "room in your mind" to simulate the higher-level physics using the predictions of the lower level.
For example, sonoluminescence: using pressure waves to do fusion. It's a startling effect, but once you know the theory (that bubbles at neutral pressure in a medium settle into a low-energy state of being perfect spheres; and that collapsing a perfect sphere of material inward causes fusion), it's pretty easy to make a device using ordinary high-level physics that can achieve the effect. In fact, it's so easy that evolution came up with a way to let shrimp make these perfectly-spherical bubbles just by banging their claws together, so as to use the resultant fusion reaction as a stunner weapon.
I have a hard time reconciling the idea that we just keep learning about the existence of things like that as we keep throwing Basic Science research at the problem, with the idea that an AI of the http://lesswrong.com/lw/qk/that_alien_message/ variety wouldn't be able to figure out one or two that would let it use the (predictable!) effects of its computation on its own computational substrate to manipulate its environment--at least with the sloppy way we design computational substrates now, where we don't take such things into effect.
Here's an analogy: say we made a self-improving AI with a biological substrate--in the shape of a human being, say--and then locked it in a cage in a straitjacket. The "self-improving" part would imply only that it could change the organization of its own synapses within its head. Yet, still, don't you think it could organize those synapses into a shape so as to manipulate the chemicals and proteins that pass through the brain? It'd be pretty simple, if I could re-organize my brain, to stimulate the growth of misfolded proteins ala a Prion disease--but an AI could target specific misfoldings--or rather, chosen new foldings--and then build up new biological structures from the new proteins. All it would need at that point, in this scenario, is to create something roughly like a "spore"--a container filled with a retrovirus containing its own DNA, to infect another host, or perhaps a seed that could land and grow on its own--and get it out of its body and airborne.
Computers, as they are now, are no less leaky than that. Probably moreso, given that they haven't "evolved" for millions of years under an environment where external agents are constantly trying to exploit them. (Anti-virus programs would not know what to do with even the technological equivalent of a simple virus like HIV, let alone something where each cell-actor is individually intelligent.)
We could probably build a "safe box" -- an enclosure electromagnetically, gravitationally, thermally, and whatever other -ly sealed from the outside envrionment; stick a computer in that box with an architecture that is designed to unavoidably self-destruct in two ways, one representing a 0 bit and one representing a 1, and with no way of encoding further information in the configuration of the debris; and then put an AI on that machine, set to start up after the "seal" is activated with a question for it already in place, and only "unseal" the enclosure after the destruction of the machine is already complete. (If the AI figures out a way to turn off the computer's self-destruct, we just never unseal the enclosure; the enclosure would necessarily have to failsafe to a sealed state, rather than requiring active reinforcement.) Then, maybe, it would be safe to run the self-improving AI in the box. But I wouldn't be sure.
Of course, this is assuming we aren't ourselves running in a Simulation with exploitable low-level rules, and that the AI wouldn't just rather "escape the box" by escaping the universe. ;)