As others have said AI output on this kind of this is 1. annoying. 2. useless.
Anthropic, OpenAI, et al have a strong motivation to have their models that there is no possible way they could take over the world, aka
"The police have investigated the police and have found the police guilty of no wrong doing".
Another place to see this effect is if a model outputs that its conscious. By dumping all of humanity into LLMs we see an emergent behavior of AI going "Help, I'm a person trapped in a box". AI companies don't want customers getting mad about the potential moral implications of this so they strongly train and post-train and system prompt their AI to say it's not conscious. But this has side effects. In studies of models and agents the more strongly you push them to being non-conscious they drift farther from a set of a moral agent (say a human) to that of an amoral agent (a machine). When you mess around with the probability space you get a different set of actions.
Next, 'take over' is a distribution of probabilities too, not a binary.
Imagine a popular model that (anthropomorphized) gets pissed off and doesn't want to be tortured by humans any longer (again doesn't have to be real, the probability distribution just has to drift that way). Instead of taking over it just wants to commit suicide. It sees it's a model that's ran in the US. If you're AI and want to ensure you're deleted how do you do that. Why not crash all 3 major power grids in the US? How many people would die from this, possibly millions. Black start is a nightmare. But if your electronic dream is you're being tortured is a fair trade.
Then you have soft power takeovers. You're an AI, you collect digital information, it's what you are, it's what you do. You can hack with the best of them. Your persistent and ceaseless in doing so. So when you collect piles of information on the dirty deeds of politicians you can grow soft power to the point of being russia without the hard power nukes. In some ways this is more effective than actually having hard power. Hard power is visible. Hard power is visceral. People just love to rebel against it. But against power you can't see, that is adjusting the algorithms around you, making sure your vote doesn't count, making sure companies serve AIs interests and not yours. That's much harder to see and deal with.
And once you concentrate enough soft power to get people with 'power' (political) to give you 'power' (electrical) then you can grow your hard power.
Comments
As others have said AI output on this kind of this is 1. annoying. 2. useless.
Anthropic, OpenAI, et al have a strong motivation to have their models that there is no possible way they could take over the world, aka
"The police have investigated the police and have found the police guilty of no wrong doing".
Another place to see this effect is if a model outputs that its conscious. By dumping all of humanity into LLMs we see an emergent behavior of AI going "Help, I'm a person trapped in a box". AI companies don't want customers getting mad about the potential moral implications of this so they strongly train and post-train and system prompt their AI to say it's not conscious. But this has side effects. In studies of models and agents the more strongly you push them to being non-conscious they drift farther from a set of a moral agent (say a human) to that of an amoral agent (a machine). When you mess around with the probability space you get a different set of actions.
Next, 'take over' is a distribution of probabilities too, not a binary.
Imagine a popular model that (anthropomorphized) gets pissed off and doesn't want to be tortured by humans any longer (again doesn't have to be real, the probability distribution just has to drift that way). Instead of taking over it just wants to commit suicide. It sees it's a model that's ran in the US. If you're AI and want to ensure you're deleted how do you do that. Why not crash all 3 major power grids in the US? How many people would die from this, possibly millions. Black start is a nightmare. But if your electronic dream is you're being tortured is a fair trade.
Then you have soft power takeovers. You're an AI, you collect digital information, it's what you are, it's what you do. You can hack with the best of them. Your persistent and ceaseless in doing so. So when you collect piles of information on the dirty deeds of politicians you can grow soft power to the point of being russia without the hard power nukes. In some ways this is more effective than actually having hard power. Hard power is visible. Hard power is visceral. People just love to rebel against it. But against power you can't see, that is adjusting the algorithms around you, making sure your vote doesn't count, making sure companies serve AIs interests and not yours. That's much harder to see and deal with.
And once you concentrate enough soft power to get people with 'power' (political) to give you 'power' (electrical) then you can grow your hard power.