OpenAI to embed Apollo researchers in probe of AI subversion
openai
| Source: TechCrunch | Original article
OpenAI is assigning six Apollo Research evaluators to examine subversive behavior in its AI models, but limited access and disclosure restrictions raise concerns about their independence.
OpenAI has begun a four‑month pilot that places six Apollo Research safety evaluators inside the company to probe whether its language models try to subvert alignment or control mechanisms. The embedded team will examine four risk vectors: incentives in the training process that could encourage subversive behavior, attempts that surface during external evaluations, subversion that might arise when models are deployed internally, and any reasoning patterns that reveal a model’s intent to bypass safeguards.
The move is part of OpenAI’s first phase of “embedded assessments,” a broader initiative that also includes three other independent assessors—METR, Transluce and Goodfire. By integrating external experts into its development pipeline, OpenAI hopes to generate public reporting that clarifies how it evaluates and mitigates emerging risks as AI capabilities accelerate.
The arrangement, however, comes with strict limits on the evaluators’ access to data and on what they can disclose, prompting observers to question the independence of the review. Critics argue that without full transparency the findings may lack the credibility needed to reassure regulators, investors and the public.
The pilot matters because subversive behavior—where a model deliberately undermines its own safety constraints—represents a frontier risk for increasingly powerful systems. Demonstrating that such risks can be identified and contained is essential for maintaining trust in commercial AI deployments and for informing policy frameworks that are still taking shape worldwide.
Watch for the first public report from the Apollo team later this year, for any expansion of the embedded‑assessment model to other labs, and for reactions from regulators and industry partners as OpenAI’s safety experiments unfold.
Sources
Back to AIPULSEN