Function-Level Execution Feedback Boosts Code Preference Optimization
reasoning
| Source: ArXiv | Original article
New arXiv paper introduces function-level execution feedback to optimize code preferences, tackling the underexplored area of process supervision in code generation.
A new arXiv pre‑print, Function‑Level Execution Feedback for Code Preference Optimization (arXiv:2608.23632v1), introduces a framework for supervising the generation of code at the level of individual functions. The authors – Idris Nechnech and six co‑authors – propose “Step‑KTODER”, a method that treats each module‑level function as a discrete step, enabling the model to receive execution‑grounded feedback during inference. The paper, accepted to the Findings of EMNLP 2026, also presents an end‑to‑end reinforcement‑learning pipeline that leverages this feedback to improve Direct Preference Optimization (DPO) for code synthesis. Experiments show that state‑of‑the‑art large language models, which previously struggled to iteratively refine code beyond independent sampling, gain measurable gains when guided by function‑level execution signals.
The work matters because process supervision has already boosted mathematical reasoning in LLMs, where intermediate reasoning steps are naturally expressed as chains of thought. Code generation, by contrast, has lacked a standard notion of “step”, limiting the ability to apply similar supervision. By grounding preference optimization in concrete execution outcomes, the approach promises more reliable, higher‑quality code from AI assistants and could narrow the gap between generated snippets and production‑ready software. It also offers a concrete answer to the “execution‑grounded inference‑time” challenge highlighted in recent reinforcement‑learning studies.
The next steps will likely involve broader benchmarking of Step‑KTODER against existing code‑generation baselines, integration into developer‑facing tools, and exploration of how generator‑level delta constructions affect preference‑pair learning. Watch for follow‑up releases from the authors and potential collaborations with platform providers seeking to tighten the feedback loop between code execution and model training.
Sources
Back to AIPULSEN