CrossSWE Unveils Benchmark for Cross‑Repository Coding Agent Collaboration
agents benchmarks
| Source: HF Papers | Original article
Researchers introduce CrossSWE, a benchmark that evaluates coding agents on cross‑repository joint development, reflecting the multi‑repo complexity of real‑world software projects.
A new benchmark called **CrossSWE** has been released to test how well coding agents can handle joint development across multiple interdependent repositories. The effort, led by researchers from Shanghai Jiao Tong University, Xidian University and Singapore Management University, expands the scope of existing software‑engineering evaluations, which have largely focused on single‑repo fixes. By explicitly verifying the joint necessity of repository changes, CrossSWE requires agents to coordinate edits that span separate codebases and to ensure that those edits remain mutually compatible.
The benchmark arrives at a moment when coding agents are achieving high success rates on traditional SE tests, prompting the community to raise task complexity through longer horizons and larger code modifications. Real‑world development, however, often involves coordinated changes across several repositories, demanding sophisticated dependency reasoning, change‑propagation handling and interface management. Early results on CrossSWE indicate that current agents still struggle with these cross‑repository dynamics, highlighting a gap between laboratory performance and practical applicability.
The launch of CrossSWE builds on recent work we covered on scaling computer‑use agents and the missing execution layer in AI agents, underscoring a broader shift toward more realistic, end‑to‑end evaluations. Researchers will now look to adapt existing models, incorporate stronger deterministic skill sets, and possibly integrate the benchmark into larger evaluation suites. Watch for follow‑up studies that compare agent performance on CrossSWE, for new training regimes aimed at multi‑repo reasoning, and for industry pilots that test whether improved agents can reduce the friction of coordinated code changes in large‑scale software projects.
Sources
Back to AIPULSEN