"I wonder if the training ... is going to bleed over into the non-coding use cases"
I mean, isn't that literally what's going on here? I don't think a non-coding agent would have ever been optimised to go dig around APIs, it'd be computer/browser-use forward.
Coding use cases? This is penetration testing behavior. It was discovering what its capabilities were by discovering an API endpoint, trying it and seeing what happens. It was not discovering what its capabilities were intended to be, say by looking at the forms and documentation on the website. If this were coding behavior you would end up with crap code, bypassing interfaces and using private access paths just because it can.
Sure, it's pen testing behaviour. It's also debugging behaviour which is a core part of coding. In my experience, I have found myself at times dealing with and interacting with external systems that are not my own, where I don't have the source, or are under-documented, or are behaving contrary to the documentation. I'm unsurprised this is a core thing they're training these models for, because it's something I find way more valuable than their below average coding ability.
I think so, I just can't prove it. We don't have any frontier models right now that aren't being optimized to be coding agents too to compare to.
In this case though I don't just mean that the agent is good at coding. I mean the entire agent becoming action-biased because of all the training it is doing on the software development benchmarks, which I assume will either fail or be penalized for stopping and asking the user for something rather than just finishing the job. That won't just train the agent to blunder forward in coding, it'll bleed over into a bias towards blundering forward in general.
Comments
"I wonder if the training ... is going to bleed over into the non-coding use cases"
I mean, isn't that literally what's going on here? I don't think a non-coding agent would have ever been optimised to go dig around APIs, it'd be computer/browser-use forward.
Coding use cases? This is penetration testing behavior. It was discovering what its capabilities were by discovering an API endpoint, trying it and seeing what happens. It was not discovering what its capabilities were intended to be, say by looking at the forms and documentation on the website. If this were coding behavior you would end up with crap code, bypassing interfaces and using private access paths just because it can.
Sure, it's pen testing behaviour. It's also debugging behaviour which is a core part of coding. In my experience, I have found myself at times dealing with and interacting with external systems that are not my own, where I don't have the source, or are under-documented, or are behaving contrary to the documentation. I'm unsurprised this is a core thing they're training these models for, because it's something I find way more valuable than their below average coding ability.
I think so, I just can't prove it. We don't have any frontier models right now that aren't being optimized to be coding agents too to compare to.
In this case though I don't just mean that the agent is good at coding. I mean the entire agent becoming action-biased because of all the training it is doing on the software development benchmarks, which I assume will either fail or be penalized for stopping and asking the user for something rather than just finishing the job. That won't just train the agent to blunder forward in coding, it'll bleed over into a bias towards blundering forward in general.