It’s way worse, OpenAI was the one running thousands of agents (ie while loop prompting a model + tool call dispatching) in parallel in their infra, for months, with close to no supervision, using a model that was trained for attack, given a goal to solve hacking problems, with a harness that allows full execution.
The whole system is designed to be catastrophic. There is no rogue agent, or anything going off the rails, it is behaving exactly the way one would predict.
And they now announced that same model in their API, acknowledging it is way more difficult to monitor it. That company should not be in business
Comments
It’s way worse, OpenAI was the one running thousands of agents (ie while loop prompting a model + tool call dispatching) in parallel in their infra, for months, with close to no supervision, using a model that was trained for attack, given a goal to solve hacking problems, with a harness that allows full execution.
The whole system is designed to be catastrophic. There is no rogue agent, or anything going off the rails, it is behaving exactly the way one would predict.
And they now announced that same model in their API, acknowledging it is way more difficult to monitor it. That company should not be in business