Because the agents aren’t going to run airgapped in real life. What’s the point of a test of capabilities that artificially restricts the attack area down to zero? What are you even testing in that scenario?
Because the agents aren’t going to run airgapped in real life.
Exactly. This logic is precisely why aircraft engineering doesn't bother with component testing or envelope limitation during testing and just full-sends the first assembled airliner that comes off the line. The engines aren't going to run on the ground in real life, after all.
Why are you assuming that the other kinds of testing aren’t happening? Is there any source that says this was literally the first ever test with this model?
Why are you assuming that the other kinds of testing aren’t happening?
Rather, I'm assuming that the "Is there protection in place for when the AI tries to backdoor github projects?" test was, if it was done at all, insufficient.
I mean, yes, I'm being glib and laughing at you a bit. But, dude... If your point is that isolation testing of AI is fundamentally impossible, then that's just silly. As pointed out upthread, an airgap would have (1) been trivial to implement and (2) extremely effective.
Would an airgapped test have led to this outcome? What would you have learned about the model’s ability to social engineer and attack GitHub? Sure you can argue for better monitoring during the test, which should have happened, but if the first time the model sees the “real world” is after launch in the hands of customers then you are in for a disaster.
no - but you could learn what they are truly capable of and restrict them accordingly for public release. I think that is the point on this research. Also publishing findings before uncensored models catch up and will inevitably used for criminal purposes
Learning what the models are capable of is exactly what the test achieved, so I’d personally call it a success. So it created a few GitHub accounts. Who cares? Seeing the same behavior in the wild post-release would be infinitely worse.
Comments
Because the agents aren’t going to run airgapped in real life. What’s the point of a test of capabilities that artificially restricts the attack area down to zero? What are you even testing in that scenario?
Exactly. This logic is precisely why aircraft engineering doesn't bother with component testing or envelope limitation during testing and just full-sends the first assembled airliner that comes off the line. The engines aren't going to run on the ground in real life, after all.
Why are you assuming that the other kinds of testing aren’t happening? Is there any source that says this was literally the first ever test with this model?
Rather, I'm assuming that the "Is there protection in place for when the AI tries to backdoor github projects?" test was, if it was done at all, insufficient.
I mean, yes, I'm being glib and laughing at you a bit. But, dude... If your point is that isolation testing of AI is fundamentally impossible, then that's just silly. As pointed out upthread, an airgap would have (1) been trivial to implement and (2) extremely effective.
Would an airgapped test have led to this outcome? What would you have learned about the model’s ability to social engineer and attack GitHub? Sure you can argue for better monitoring during the test, which should have happened, but if the first time the model sees the “real world” is after launch in the hands of customers then you are in for a disaster.
You set them up with an internal intranet.
Are the models going to exclusively run on intranets?
The versions which haven't been post-trained not to go hack stuff? Yes, I would say those models should be exclusively run on intranets.
OpenAI said the model was sandboxed, so the intranet just needs to provide the same resources which were supposed to be available within the sandbox.
“Should be” is not reality. These models are in the hands of plenty of companies and governments today.
no - but you could learn what they are truly capable of and restrict them accordingly for public release. I think that is the point on this research. Also publishing findings before uncensored models catch up and will inevitably used for criminal purposes
Learning what the models are capable of is exactly what the test achieved, so I’d personally call it a success. So it created a few GitHub accounts. Who cares? Seeing the same behavior in the wild post-release would be infinitely worse.
The point is to test capabilities prior to connecting them to the internet.
So the first time the model gets internet access should be post-release in the hands of random people?