Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?

Self-distillation trains a model with itself as teacher. The teacher copy sees privileged context, such as the worked solution or environment feedback; the student copy sees only the problem, writes an answer, and is trained to match the teacher's next-token probabilities on that answer. Recent methods (SDPO among them) reported faster gains and shorter reasoning than RL with verifiable rewards (GRPO) in chemistry, tool use and coding.

On math the authors saw the opposite: responses got shorter and accuracy fell. Their explanation is epistemic verbalization: reasoning models write "wait", "hmm", "maybe" and similar words at points of uncertainty, and those moments are where they catch and repair mistakes. A teacher that already knows the solution has no reason to hedge. Measured on DeepSeek-R1-Distill-Qwen-7B and 100 math problems, giving the model the full solution cut answers from 13,054 tokens to 1,873 and uncertainty markers from 182.5 to 8.8. Less informative context (the solution without its reasoning) left both near the unguided level.

SFT test. Fine-tuning on 800 correct unguided answers left benchmark scores about where they were; fine-tuning on 800 correct solution-guided answers dropped AIME24 from 54.8 to 20.2 and MATH500 from 92.2 to 65.5. Every answer was right; only the style differed.

On-policy test. SDPO with the full solution cut AIME24 by about 40% where GRPO improved it slightly. Penalizing the ten markers inside GRPO reproduced part of the drop, which the authors take as causal evidence.

When it works. Self-distillation helped where the task set is small and repeated (six chemistry problem types; 131 coding problems used for both training and evaluation). As the number of distinct training problems grows and tests use unseen problems, confident, hedge-free reasoning stops paying.

Teams fine-tuning reasoning models for a customer often build training data the obvious way: show a strong model the reference solution and ask it to write a clean worked answer. This paper shows that exact recipe can wreck a reasoning model, because the clean answers teach the model to stop checking its work.

Clean is not the same as good training data. Answers written with the solution in view are short and confident because the writer already knew where it was going. A model trained on them learns that style without the knowledge behind it. The drop here (AIME24 54.8 to 20.2) came from correct answers only.

Shorter outputs can be a warning, not a win. Lower token counts look like an efficiency gain on a dashboard. Track the uncertainty markers and accuracy on unseen problems alongside length; a fall in all three together is the failure this paper describes.

Narrow tasks are the exception. If the customer's task really is a few repeated problem types (a fixed extraction, a standard calculation), confident short reasoning trained by self-distillation can be fine, and faster. If users bring varied, novel problems, keep the model's habit of checking.