Reasoning-to-Readout Self-Distillation
How it works
For a given query the model first unrolls its reasoning and arrives at a probability distribution over the allowed options — this is the teaching signal (target). The same model is then trained so that its first-token readout — restricted to the allowed option labels — reproduces that post-reasoning distribution immediately. Because the target comes from the same model after reasoning, and not from an external teacher or gold labels, this is reasoning-to-readout self-distillation. At inference only the First-Token Decision Readout is used: a single forward pass returns the decision and its probability. The same readout handles all three JEVal formats (choice, judgment, score). Full-weight and LoRA variants are released.
Problem solved
Reasoning-based decision models (chain-of-thought) are slow and costly because they must generate a long token chain before deciding, and classic distillation needs gold labels or a stronger teacher. This technique removes both constraints: it obtains "post-reasoning" decision quality in a single forward pass and trains without gold labels and without an external teacher.
Components
An inference mechanism that restricts the next-token distribution to the allowed option labels and returns the decision with its probability in a single forward pass.
The probability distribution over options that the same model reaches after unrolling reasoning; it serves as the teaching target in self-distillation (the model as its own teacher).
Evolution
The SII team presents Reasoning-to-Readout Self-Distillation and the First-Token Decision Readout, training SII-Decision-4B/27B without gold labels and without a teacher.