Skip to content

Comment on Pausing AI Developments Isn't Enough. We Need to Shut It All Downparent

Comments

Intelligence is a limited, partial substitute for knowledge. The problem with Aristotelian physics wasn't that Aristotle wasn't smart enough. It was that Aristotle lacked the ground truth knowledge to understand where he was wrong. An AI that can't run experiments will end up with elegant hallucinations of nanofactory blueprints but no actual nanofactory. (That's even assuming that Drexlerian nanotech is actually something that can be built. It's still an open question.)

All the computing power in the world is insufficient to simulate a living bacterium at the molecular level. We also don't have a molecular resolution map of bacteria in the first place. The AI will have to figure out how to make one. The AI can try to develop faster high fidelity approximations for molecular simulation, but that too requires calibration against experiments as well as a lot of simulation resources. If it seizes control of public cloud computing for its simulations people will notice and can just unplug cables. If its intrusions are subtle, only using little unnoticeable compute slices when machines are largely idle, then its simulations are going to get less compute resources in aggregate than researchers have now.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.