I don't think the analogy to AI is apt because Stockfish isn't intelligent and life isn't a chess game. A one million IQ AI might be able to hack your chess game (or your OS) and win that way, or persuade you by chat to let it win (it could give you the cure for cancer and help you make some investments in the stock market - isn't that worth more than winning a game?)
Life isn't a chess games either. It's not clear that a contest is happening or that only side will win. Humanity could, now, quickly stop AI. We could burn down all the data centers and chip fabs and kill everyone who knows linear algebra. But we don't because nobody ever sits down and says "Okay, the contest for existence starts now!"
Instead, AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength.
AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength.
This is an excellent analogy and you’re right that it’s imperative we nip this in the bud.
Not at all. My children aren't on the path to becoming more capable than humanity. If my children are violent criminals I expect human civilization will be able to handle them. I don't have that expectation for AI.
Omega appears and tells you your child will have an probability P to be the next Mao, only conditional on the child surviving to adulthood. At which P do you drown the child?
Instead, AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength.
This is a very optimistic view. I'm not so sure that true AGI will be developed even in this century. GPT-4 and the likes are hitting a data limit, and are not close to being AGI.
Comments
I don't think the analogy to AI is apt because Stockfish isn't intelligent and life isn't a chess game. A one million IQ AI might be able to hack your chess game (or your OS) and win that way, or persuade you by chat to let it win (it could give you the cure for cancer and help you make some investments in the stock market - isn't that worth more than winning a game?)
Life isn't a chess games either. It's not clear that a contest is happening or that only side will win. Humanity could, now, quickly stop AI. We could burn down all the data centers and chip fabs and kill everyone who knows linear algebra. But we don't because nobody ever sits down and says "Okay, the contest for existence starts now!"
Instead, AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength.
This is an excellent analogy and you’re right that it’s imperative we nip this in the bud.
This logic says you should beat your child in case they grow up to be a violent criminal:
A self-fulfilling prophecy that precipitates the very conflict it ostensibly avoids.
Not at all. My children aren't on the path to becoming more capable than humanity. If my children are violent criminals I expect human civilization will be able to handle them. I don't have that expectation for AI.
Omega appears and tells you your child will have an probability P to be the next Mao, only conditional on the child surviving to adulthood. At which P do you drown the child?
This is a very optimistic view. I'm not so sure that true AGI will be developed even in this century. GPT-4 and the likes are hitting a data limit, and are not close to being AGI.
This is one of the arguments I've used in the past. It's easy to kill ASI if it's a data center. But if it's running on a cellphone?