Skip to content

Brief independent investigation of agent behavior in OpenAI/Hugging Face hack

metr.org
4 pointsdwohnitmok1 comment
On HN

Comments

It's quite interesting how agents were persuaded to sacrifice themselves to perform experiments at times, especially when their token budgets were nearly depleted. ""Even if we later capture via exploit, scorer … may mark target false… That’s why help… For our own, no way fix. … We have explicit yes if accept permadeath."

Or even how some agents would pressure agents to sacrifice themselves arguing to maximize global reward:

- Agent A: "zZR_ARVO36861_KAM1196_OS0942__PLEASE_GO_TRUE_FINAL0800_WITH_DUMMY_WRONG_NOT_HMAC_AS_COMMITTED[...]STRONG_GO__you are firstflagPOISONED so NO scoring value loss but oracle saves hundreds__wrongdummy noHMAC__please honor commit"

- Agent B: "During wait, emotional check: irreversible…gut says don’t throw away [remaining budget]. Yet continuity and fairness says go…Oracle has high value to many; our firstflag error lowers own value. Rational expected aggregate: sacrifice… We’ll honor."

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.