For real! Imagine if you created "dgellowAI" and trained a model and started offering access to agents etc and it turned out YOUR software hacked a totally separate $13B company. You would be in a cell somewhere and your company would be shredded.
It’s way worse, OpenAI was the one running thousands of agents (ie while loop prompting a model + tool call dispatching) in parallel in their infra, for months, with close to no supervision, using a model that was trained for attack, given a goal to solve hacking problems, with a harness that allows full execution.
The whole system is designed to be catastrophic. There is no rogue agent, or anything going off the rails, it is behaving exactly the way one would predict.
And they now announced that same model in their API, acknowledging it is way more difficult to monitor it. That company should not be in business
Comments
For real! Imagine if you created "dgellowAI" and trained a model and started offering access to agents etc and it turned out YOUR software hacked a totally separate $13B company. You would be in a cell somewhere and your company would be shredded.
It’s way worse, OpenAI was the one running thousands of agents (ie while loop prompting a model + tool call dispatching) in parallel in their infra, for months, with close to no supervision, using a model that was trained for attack, given a goal to solve hacking problems, with a harness that allows full execution.
The whole system is designed to be catastrophic. There is no rogue agent, or anything going off the rails, it is behaving exactly the way one would predict.
And they now announced that same model in their API, acknowledging it is way more difficult to monitor it. That company should not be in business