Language Models Iteratively Build, Grade, and Revise Data Science Projects
agents
| Source: ArXiv | Original article
Researchers assess how agentic AI models can autonomously execute full data‑science projects, showing performance gains when iterative feedback is applied.
A team led by Rohan Alexander has released a new arXiv pre‑print, “Iterated creation, grading, and revision of data science projects by language models” (arXiv:2611.05548v1). The paper describes a closed‑loop experiment in which an “agentic” large language model (LLM) acts as a writer, using specialised tools to download a dataset, clean and analyse it, and produce a short research report. The output is then executed, and a separate “grader” LLM evaluates the results against a set of criteria. Based on the feedback, the writer revises its code and narrative, iterating until the grader’s score improves.
The work matters because it pushes LLMs beyond single‑turn question answering toward end‑to‑end autonomous data‑science pipelines. By embedding a feedback loop, the authors show that performance can rise with iterative refinement, echoing earlier findings on LLMs as reviewers of scientific literature (see our Nov 5 coverage). If such agentic systems can reliably generate, test, and polish analyses, they could lower the barrier to reproducible research, accelerate exploratory work, and free human analysts for higher‑level interpretation.
The study was funded by Canada’s Natural Sciences and Engineering Research Council (NSERC) Alliance Grant and the Social Sciences and Humanities Research Council (SSHRC), underscoring growing public interest in AI‑driven scientific productivity.
What to watch next: the authors have released only the abstract and high‑level description, so the community will be looking for detailed methodology, benchmark datasets, and quantitative results. Follow‑up work may explore scaling the loop to larger, more complex projects, integrating domain‑specific tools, or deploying the approach in commercial data‑science platforms. Success could reshape how research teams prototype analyses and how institutions assess reproducibility in AI‑augmented science.
Sources
Back to AIPULSEN