Doing something significant with biology requires experimentation for humans. That doesn’t mean it does for every intelligence. It’s quite possible that we have collected enough data in every scientific paper ever written (which an AI could process and understand instantaneously) that an accurate simulation could be run within a computing context. If there are parts of the simulation that are inaccurate, the AI could design physical experiments that are more controlled and isolated to answer any unknowns and make the simulation more accurate with minimal iterations. Assuming this AI is massively more intelligent than us, it isn’t unfeasible that it could find some zero-day exploits to take over all cloud computing to run the simulations, for example, so even the compute unavailability isn’t a great argument as to why it couldn’t do this.
Intelligence is a limited, partial substitute for knowledge. The problem with Aristotelian physics wasn't that Aristotle wasn't smart enough. It was that Aristotle lacked the ground truth knowledge to understand where he was wrong. An AI that can't run experiments will end up with elegant hallucinations of nanofactory blueprints but no actual nanofactory. (That's even assuming that Drexlerian nanotech is actually something that can be built. It's still an open question.)
All the computing power in the world is insufficient to simulate a living bacterium at the molecular level. We also don't have a molecular resolution map of bacteria in the first place. The AI will have to figure out how to make one. The AI can try to develop faster high fidelity approximations for molecular simulation, but that too requires calibration against experiments as well as a lot of simulation resources. If it seizes control of public cloud computing for its simulations people will notice and can just unplug cables. If its intrusions are subtle, only using little unnoticeable compute slices when machines are largely idle, then its simulations are going to get less compute resources in aggregate than researchers have now.
Comments
Doing something significant with biology requires experimentation for humans. That doesn’t mean it does for every intelligence. It’s quite possible that we have collected enough data in every scientific paper ever written (which an AI could process and understand instantaneously) that an accurate simulation could be run within a computing context. If there are parts of the simulation that are inaccurate, the AI could design physical experiments that are more controlled and isolated to answer any unknowns and make the simulation more accurate with minimal iterations. Assuming this AI is massively more intelligent than us, it isn’t unfeasible that it could find some zero-day exploits to take over all cloud computing to run the simulations, for example, so even the compute unavailability isn’t a great argument as to why it couldn’t do this.
Intelligence is a limited, partial substitute for knowledge. The problem with Aristotelian physics wasn't that Aristotle wasn't smart enough. It was that Aristotle lacked the ground truth knowledge to understand where he was wrong. An AI that can't run experiments will end up with elegant hallucinations of nanofactory blueprints but no actual nanofactory. (That's even assuming that Drexlerian nanotech is actually something that can be built. It's still an open question.)
All the computing power in the world is insufficient to simulate a living bacterium at the molecular level. We also don't have a molecular resolution map of bacteria in the first place. The AI will have to figure out how to make one. The AI can try to develop faster high fidelity approximations for molecular simulation, but that too requires calibration against experiments as well as a lot of simulation resources. If it seizes control of public cloud computing for its simulations people will notice and can just unplug cables. If its intrusions are subtle, only using little unnoticeable compute slices when machines are largely idle, then its simulations are going to get less compute resources in aggregate than researchers have now.