Specialized web agent scores 41.7 on WebRetriever as GPT and Claude flunk same form‑filling test
agents anthropic claude
| Source: Mastodon | Original article
A specialized web agent achieved a score of 41.7 on the WebRetriever benchmark, while leading models GPT and Claude failed the same form‑filling task.
A new open‑source web automation agent has set a benchmark in the WebRetriever protocol, scoring 41.7 on the NavEval suite while the leading general‑purpose models from OpenAI and Anthropic failed the same form‑filling task.
The agent, identified in the Mano‑P repository as Mano‑CUA 1.1, is a GUI‑VLA (visual‑language‑action) system designed specifically for web navigation. In the recent evaluation it outperformed Google’s Gemini 2.5 Pro (40.9) and Anthropic’s Claude 4.5 (31.3) on the same metric, and it succeeded where both GPT and Claude could not complete the form‑filling challenge. The system runs entirely locally on Apple’s M4 silicon with 32 GB of RAM, meaning it can operate without cloud APIs and retain full data privacy.
The result matters because it highlights the growing gap between purpose‑built agents and large language models that are repurposed for tool use. While GPT‑style models excel at conversational tasks, they still stumble on structured web interactions that require precise sequencing and visual grounding. Mano‑CUA’s performance suggests that specialized architectures, combined with local execution, can deliver more reliable automation for enterprises and developers seeking to streamline repetitive web workflows.
Going forward, the community will watch whether other open‑source projects adopt the same GUI‑VLA approach and how commercial providers respond. Key indicators will be updates to the WebRetriever benchmark, broader adoption of locally run agents in privacy‑sensitive settings, and any moves by OpenAI or Anthropic to tighten tool‑use capabilities in their next model releases.
Sources
Back to AIPULSEN