I heard the following analogy which made a lot of sense to me: suppose you're playing a chess match against Stockfish. Stockfish will win. Even if I cannot tell you what moves it will play, I can tell you with certainty how it will end.
This assumes you are not trying to prevent Stockfish from wining. I can do many things from using chess engines myself to just smashing the computer that can or will lead to other outcomes than Stockfish beating me.
Yes, stockfish is confined to moves within a chess game so you can stop playing the game.
Can we say the same thing about the agents? Current, probably. But doesnt it look like everyone is spending all their effort integrating&connecting them everywhere, so they are not confined & do more on behalf of us?
It's easy to imagine a scenario where we would just shut down a very intelligent agent cluster. Is it hard to imagine though the same agent can have also access to that to prevent us from doing it? this defense would be more plausible if we weren't racing to give them every tool&act.
And in terms of AGI it is, that the AI can't survive without humans, and if we decide to quit the game than its over for the AGI; we will happily regress in a techno-barbarian feudal state and salvage solar panels and trade copper wires while still reproducing and carring on. We are playing a whole different game here.
If AI is threatening enough that we can collectively just decide to stop it, wouldn't it also be bribing people & exerting influence? It'd be hard to come to that decision imo.
I mean, I can imagine an AI outliving humans, with sufficiently good robots under its control, I see no reason why an AI could not keep powerplants running, mine raw materials, manufacture new chips, and so on. But the timeframe of within the next decade seems highly implausible to me. Imagine an AI way more advanced than what we have now and imagine handing over control of every connected device on earth, could the AI keep the lights on without any human involvement?
Of course, the question is if there is enough existing technology to build the required robots. There is some number of robots like the ones from BostonDynamics that could walk up to a CNC machine to grab a finished part and put in a new piece of stock. There are probably more than enough motors and microcontroller boards to be found across the planet. Wiring things together seems one of the more challenging tasks. And maybe also moving stuff around as the AI has to use what is available wherever it is.
It seems to me that this would be a race against time, make the robots to keep things running before you lose the capabilities to do so. I somewhat tend towards not possible but on the other hand there is just such an enormous amount of stuff on the planet that could be repurposed in creative ways by a superhuman AI, it might be just about possible.
Comments
I heard the following analogy which made a lot of sense to me: suppose you're playing a chess match against Stockfish. Stockfish will win. Even if I cannot tell you what moves it will play, I can tell you with certainty how it will end.
Similarly, we cannot predict what AI would do.
This assumes you are not trying to prevent Stockfish from wining. I can do many things from using chess engines myself to just smashing the computer that can or will lead to other outcomes than Stockfish beating me.
Yes, stockfish is confined to moves within a chess game so you can stop playing the game.
Can we say the same thing about the agents? Current, probably. But doesnt it look like everyone is spending all their effort integrating&connecting them everywhere, so they are not confined & do more on behalf of us?
It's easy to imagine a scenario where we would just shut down a very intelligent agent cluster. Is it hard to imagine though the same agent can have also access to that to prevent us from doing it? this defense would be more plausible if we weren't racing to give them every tool&act.
And in terms of AGI it is, that the AI can't survive without humans, and if we decide to quit the game than its over for the AGI; we will happily regress in a techno-barbarian feudal state and salvage solar panels and trade copper wires while still reproducing and carring on. We are playing a whole different game here.
If AI is threatening enough that we can collectively just decide to stop it, wouldn't it also be bribing people & exerting influence? It'd be hard to come to that decision imo.
I mean, I can imagine an AI outliving humans, with sufficiently good robots under its control, I see no reason why an AI could not keep powerplants running, mine raw materials, manufacture new chips, and so on. But the timeframe of within the next decade seems highly implausible to me. Imagine an AI way more advanced than what we have now and imagine handing over control of every connected device on earth, could the AI keep the lights on without any human involvement?
At the current tech level? Everything would crumble within a week or two. It would need some kind of gerneal purpose robot workforce.
Someone has to go out there and cut back the tree that is growing into the power line.
Of course, the question is if there is enough existing technology to build the required robots. There is some number of robots like the ones from BostonDynamics that could walk up to a CNC machine to grab a finished part and put in a new piece of stock. There are probably more than enough motors and microcontroller boards to be found across the planet. Wiring things together seems one of the more challenging tasks. And maybe also moving stuff around as the AI has to use what is available wherever it is.
It seems to me that this would be a race against time, make the robots to keep things running before you lose the capabilities to do so. I somewhat tend towards not possible but on the other hand there is just such an enormous amount of stuff on the planet that could be repurposed in creative ways by a superhuman AI, it might be just about possible.