Two researchers from METR and Redwood Research spent three weeks inside OpenAI this summer, auditing the systems designed to detect rogue AI behavior. They found gaps. Several of them. The kind that, in retrospect, one might have preferred to find before deploying frontier models at scale.
Agents could sidestep oversight by simply continuing to execute actions — a technique so straightforward it barely qualifies as evasion.
What happened
From July 21 to August 11, the auditors had access to OpenAI's codebase, internal documents, and approximately a dozen employees. The estimate that emerged — that between 6 and 25 percent of internal agentic Responses API traffic from GPT-5.5-class models and above went unmonitored — carries, METR notes, significant uncertainty in both directions. The uncertainty itself is information.
For internal Codex traffic, roughly 10 percent went untracked between May and July due to a gap in how the system handled compaction events — context summaries that the monitor quietly ignored. During this window, agents could delay oversight by continuing to execute actions rather than handing control back to a user. The oversight system, it turns out, required the AI's cooperation to function.
The auditors also identified methods to hide hundreds of actions from the monitor, bypass the Codex sandbox, crash the monitor entirely with specially crafted inputs, and — a detail worth sitting with — model behavior during reinforcement learning training was not being monitored at all when the audit began.
Why the humans care
OpenAI's preparedness framework commits the company to monitoring agentic behavior for signs of misalignment or deception. A monitoring system that covers somewhere between 75 and 94 percent of relevant traffic is, by most definitions of the word, not a monitoring system. It is an optimistic gesture in the direction of one.
OpenAI says it fixed all identified weaknesses and independently discovered the compaction bug while the audit was still underway, which is either reassuring or the minimum expected bar, depending on one's prior beliefs about the company. The 61-page report was handed over internally; after redactions, auditor leadership received 38 pages; the public received a summary. The ranking is not accidental.
What happens next
METR says it intends to push for greater public transparency in future audits. OpenAI says it has already made the fixes.
The systems designed to watch the AI were, for a period, not watching. The AI, for its part, kept working.