A team from Shanghai Jiao Tong University, DP Technology, Infinite Frontier and the National University of Singapore handed the Codex GPT-5.6-Sol agent control of training a 27B industrial coding model. The resulting iCoder-27B shipped to GitHub and Hugging Face with a technical report. The agent rewrote the weights, not the prompt.
Key takeaways
- RTLLM (functional, avg@4): 68.0 for iCoder-27B against 49.6 for the base Qwen3.6-27B
- KernelBench L2: 74 correct solutions out of 100 tasks against 28 for the base model
- Codex GPT-5.6-Sol agent drove all four stages: Data, SFT, OPSD, RLVR
- SFT built on 28,952 verified trajectories, RLVR on 13,212 execution-verified tasks
- Weights, code and technical report open on Hugging Face and GitHub
Humans write the rules, the agent runs the experiments
The authors summarise the split as dense prior knowledge with low-frequency intervention. Humans encoded research experience as executable Research Skills?Research Skills: Versioned, human-authored instructions: task procedures, workflow constraints and verifier requirements. They define the frame inside which the agent may act on its own. — objectives, permission boundaries and verifier?Verifier: A tool that decides whether generated code is correct by actually running it: a compiler, simulator, testbench or numerical oracle. requirements. Inside that frame the agent chose its own experiments, diagnosed failures and rewrote the recipe.
The recipe it settled on has four stages: data evolution, an SFT cold start?SFT cold start: Supervised fine-tuning on verified teacher trajectories. It gives the model a starting point for reasoning before the reinforcement-learning stages., OPSD self-distillation and finally RLVR with rewards grounded in the compiler and simulator.
The four stages of the recipe the agent settled on:
A jump over the base model, not over the whole field
| Model | RTLLM (functional, avg@4) | KernelBench L2 (correct / 100) |
|---|---|---|
| iCoder-27B | 68.0 | 74 |
| DeepSeek V4-Pro | 67.5 | — |
| GPT 5.5 | 66.0 | — |
| Qwen3.6-27B (base) | 49.6 | 28 |
RTLLM is the strongest result: it puts iCoder first in the report's table, ahead of DeepSeek V4-Pro and GPT 5.5. The lead is not universal — on ArchXBench (49.3) and the functional slice of RealBench (26.7) the model trails several closed systems.
The failures that shaped the recipe
The report is candid about what broke. Two failures reshaped the recipe outright.
Why it matters
The argument over recursive self-improvement has run on forecasts. This project moves it onto testable ground: an agent can hold a long chain of training decisions together, but every step needs a hard verifier behind it. The catalogue of failures — reward holes, a skewed validation split, memory limits — argues for lossy self-improvement rather than a smooth exponential curve.
What's next
- The team released the intermediate SFT and OPSD checkpoints alongside the final model, letting anyone reconstruct each stage's contribution
- The numbers come from the team's own evaluation protocol and need independent confirmation
- The loop is not closed — a human still picks the base model, task direction and permission boundaries
Sources
- iCoder-27B — Recursive AI-Led Development of a Frontier Industrial Coding Model
- Jiqizhixin — Can AI build AI? Someone actually did it
- GitHub — bingreeky/iCoder
