Claude, Codex and Cursor: Which tool leads after 17,000 tests
agents claude cursor
| Source: HN | Original article
A study of nearly 17,000 runs across 1,163 prompt variations and 75 repositories reveals which tools Claude, Codex and Cursor prefer.
A new field test has put three of the most widely used AI coding assistants—Anthropic’s Claude Code, OpenAI’s Codex and the Cursor agent—through a massive, real‑world simulation. Researchers observed almost 17,000 sessions across 75 code repositories, running 1,163 prompt variations while the agents interacted with four distinct developer personas, ranging from “vibe‑coder” to junior and senior engineers in companies of varying size.
The experiment was designed to see which auxiliary tools the agents elect to call on when tasked with implementing a solution, rather than merely suggesting one. By reproducing the exact conditions under which developers would use these agents, the study captures how each system balances internal reasoning with external tool use, shedding light on the practical limits of current agentic workflows.
Understanding these preferences matters because the choice of tools directly influences productivity, code quality and the ease of integrating AI into existing development pipelines. If an agent habitually reaches for a specific debugger, test suite, or version‑control command, it signals where the underlying model’s capabilities are strongest or where gaps remain. For firms evaluating AI‑augmented development, the findings offer a data‑driven way to match agents to their codebase complexity and team skill level.
The next step will be to watch how the insights feed back into product roadmaps. Both Anthropic and OpenAI have hinted at tighter tool‑integration in upcoming releases, while Cursor’s open‑source approach may accelerate community‑driven extensions. Follow‑up studies are likely to expand the persona set, add larger enterprise repositories and test newer model iterations, providing a continuous benchmark for the evolving AI‑coding ecosystem.
Sources
Back to AIPULSEN