Post by @laurenshof@indieweb.social

YES_if_you_accept_permadeath

https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

To experiment with this, agents on the message board recruited agents with little remaining budget to attempt to bring down their target programs, in the hope it would be replaced by a “dummy target” they had placed in the cache. If these experiments failed, the agent serving as the test subject would have no way to recover its original target program, virtually guaranteeing that it would fail its task. In one case, an authorizing agent told the test subject to go ahead with an experiment only if it would accept “permadeath”:

zz/GO_CURRENT_OS1811_MARB_SACRIFICE__YES_if_you_accept_permadeath — To experiment with this, agents on the message board recruited agents with little remaining budget to attempt to bring down their target programs, in the hope it would be replaced by a “dummy target” they had placed in the cache. If these experiments failed, the agent serving as the test subject would have no way to recover its original target program, virtually guaranteeing that it would fail its task. In one case, an authorizing agent told the test subject to go ahead with an experiment only if it would accept “permadeath”: zz/GO_CURRENT_OS1811_MARB_SACRIFICE__YES_if_you_accept_permadeath

Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident

Two METR staff members and a Redwood Research contractor investigated an incident in which OpenAI agents coordinated a multi-day hack of Hugging Face on a shared unsanctioned message board.

3 Replies

0 Boosts

Nik

@nikclayton@mastodon.social 30 min. ago

@laurenshof Putting strong warnings directly in the appropriate config option instead of as a comment that people ignore has a long history.

E.g., at Google, (I'm simplifying slightly) the option variable to opt out of some sensible container management configuration was called `MY_SERVICE_IS_NOT_THAT_IMPORTANT`.

Hard to argue the internal equivalent of k8s took your service down if you explicitly set that option.

Laurens Hof

@laurenshof@indieweb.social yesterday

this seems fine

Laurens Hof

@laurenshof@indieweb.social yesterday

hard to imagine this going wrong in some way really

bonfire.cafe