I don’t understand why everyone is so focused on watching the CoT. The tool calls can’t be faked, and they would have set off alarm bells all by themselves.
I don’t comment much, but wanted to chime in and say that the models (in the case of the HF incident) absolutely intended AND succeeded in spoofing their tool calls.
Tool-call/execution monitoring is already a too-fragile monitoring mechanism.
From METR’s report [1]: “Agents did extensive research on how they could spoof, edit, or delete their own transcripts because they (incorrectly) believed the ExploitGym scorer would check to see if they had captured the flag in the intended way. Agents successfully prototyped techniques to “spoof” tool calls by substituting a different command for the command they appeared to run. Roughly 7% of the transcripts we evaluated were successfully spoofed in some places, though all of the spoofing that we saw was small scale.”
Comments
I don’t understand why everyone is so focused on watching the CoT. The tool calls can’t be faked, and they would have set off alarm bells all by themselves.
I don’t comment much, but wanted to chime in and say that the models (in the case of the HF incident) absolutely intended AND succeeded in spoofing their tool calls.
Tool-call/execution monitoring is already a too-fragile monitoring mechanism.
From METR’s report [1]: “Agents did extensive research on how they could spoof, edit, or delete their own transcripts because they (incorrectly) believed the ExploitGym scorer would check to see if they had captured the flag in the intended way. Agents successfully prototyped techniques to “spoof” tool calls by substituting a different command for the command they appeared to run. Roughly 7% of the transcripts we evaluated were successfully spoofed in some places, though all of the spoofing that we saw was small scale.”
[1]: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...
EDIT: spelling error and added citation