CaPtain Unveils Pipeline for Post‑Training Code‑as‑Policy Robot Agents
agents fine-tuning reinforcement-learning training
| Source: HF Papers | Original article
Researchers introduce CaPtain, a pipeline that applies post‑training supervised fine‑tuning and multi‑turn reinforcement learning to turn code‑as‑policy models into robot manipulation agents.
A new research pipeline called **CaPtain** shows that robot manipulation can be taught with the same two‑step recipe that has become standard for language‑model agents in coding, web browsing and tool use. The authors first apply supervised fine‑tuning (SFT) to a small, open‑source vision‑language model, teaching it to interpret raw sensor inputs and to emit Python calls to low‑level perception and motion primitives. In a second phase the model is refined through multi‑turn reinforcement learning against a “physical critic” that evaluates the robot’s actual movements. The result is a **code‑as‑policy (CaP)** robot agent that writes its own control code, reads the execution receipts of each call, and decides autonomously when a task is complete.
The significance lies in extending the SFT‑plus‑RL paradigm—already dominant in LLM‑driven agents—to embodied robotics without resorting to massive vision‑language‑action backbones or frontier‑scale models. By leveraging code as the policy representation, CaPtain inherits the compositional generalisation of programming languages, enabling stronger performance on manipulation tasks while keeping the model footprint modest. The approach also demonstrates that a physical critic can replace costly simulated environments, offering a more direct path from data to real‑world competence.
Going forward, the community will watch for broader evaluations of CaPtain on diverse manipulation benchmarks, its integration with existing agent toolkits, and whether the code‑as‑policy concept scales to more complex, multi‑step robotic workflows. Success could reshape how researchers and industry players build execution layers for embodied AI, echoing earlier discussions on deterministic machine skills in AI agents.
Sources
Back to AIPULSEN