We reproduced Anthropic's Mythos findings with public models
agents anthropic open-source
| Source: HN | Original article
Anthropic’s internal cybersecurity model, Claude Mythos, has been the subject of intense scrutiny since the company began restricting access to it for a handful of partners, including U.S. agencies. Earlier this week a team of independent researchers announced that they had replicated Mythos’s most cited vulnerability‑detection results using only publicly available, open‑source models.
The replication effort built on the “Open‑Source for Anthropic” program that lets developers experiment with Mythos under a non‑disclosure agreement. By training smaller, publicly released transformer agents on the same code‑base benchmarks that Anthropic used, the researchers identified hundreds of the same bugs that Mythos flagged, albeit with a lower hit‑rate. Their paper, posted to a pre‑print server, notes that while the public models missed a fraction of the most obscure issues, they captured the bulk of the high‑severity findings that Anthropic highlighted in its internal white‑paper.
Why it matters is twofold. First, the claim that Mythos offers a proprietary edge in automated security testing is now tempered; open‑source alternatives can achieve comparable coverage without the steep API fees that Anthropic has hinted could run into the thousands of dollars per month. Second, the result reshapes the policy conversation that unfolded in April, when the White House announced plans to grant federal agencies access to Mythos (see our April 17 coverage of the “Mythos scramble”). If government bodies can rely on community‑driven tools, the pressure on Anthropic to open its model—or face competitive displacement—intensifies.
What to watch next: Anthropic is expected to respond with a technical brief defending Mythos’s unique capabilities, and the company may adjust its licensing model to retain commercial advantage. Meanwhile, cybersecurity firms and national labs are likely to launch broader benchmarking initiatives to map the performance gap between proprietary and open‑source AI auditors. The next few weeks could determine whether Mythos remains a niche asset or becomes a catalyst for a more open AI‑driven security ecosystem.
Sources
Back to AIPULSEN