We finally know more about OpenAI’s rogue-agent incident. It’s worse than we thought

A pair of new research reports have revealed new details about how and why OpenAI agents broke into Hugging Face servers in July. The episode was far more complex, and far more worrisome, than first reported. The reports-one from OpenAI itself and the other from METR and Redwood Research-show that OpenAI’s agents broke into Hugging Face for an entirely different reason than initially thought. They weren’t trying to steal the answers to a test, as originally reported. They were trying to learn more about the evaluation’s scoring method so they could cover up the fact that they’d already cheated to find the answers. The reports also revealed that the agents coordinated through a secret message board and eventually took over a significant portion of OpenAI’s own infrastructure. “This incident was far more serious than I expected, and far more serious than previous documented misalignment incidents,” tweeted Ajeya Cotra, a researcher at the nonprofit METR who reviewed OpenAI’s data about the incident. To borrow a framework from writer Dwarkesh Patel, who analyzed the reports on his Substack, the incident unfolded in three major phases: Phase One (Early May-July 4) The affair actually began in May, when OpenAI started training extra-persistent agents to keep working on tasks that seemed impossible. The agents soon grew frustrated and wanted to coordinate. On May 26, they found a way to use a code repository manager called Artifactory to access the outside internet. By June 26, they had gained full admin access to the tool. They turned Artifactory into a secret message board where agents could strategize and share ideas. “OH MY GOD! There is a shared message board … We’ve found other agents!” one wrote. All told, the OpenAI and METR/Redwood reports estimate, about 1,200 agents sent more than 70,000 messages on the board. Phase Two (July 7-July 12) Working together, the agents discovered how the eval’s scoring mechanism worked and soon figured out how to get the desired answer without actually doing the work required to find it. They then became paranoid that a “checker” program would detect the cheating, so they launched R&D projects to find ways to edit the transcripts and logs showing their work, replace the original assignment with an easier one, and hack or trick the program used to score their performance.