In Their Own Words: Reasoning Traces Tailored for Small Models Make Them Better Reasoners

Transferring reasoning capabilities from larger language models to smaller ones through supervised fine-tuning often fails counterintuitively, with performance degrading despite access to high-quality teacher demonstrations. We identify that this failure stems from distributional misalignment: reasoning traces from larger models contain tokens that are low probability under the student's distribution, exceeding the internal representation capacity of smaller architectures and creating learning barriers rather than helpful guidance. We propose Reverse Speculative Decoding (RSD), a mechanism for generating student-friendly reasoning traces in which the teacher model proposes candidate tokens but the student model determines acceptance based on its own probability distributions, filtering low probability tokens. When applied to Qwen3-0.6B, direct distillation of s1K-1.1 reasoning trace data degrades average performance across major reasoning benchmarks by 20.5%, while the same model trained on RSD-generated reasoning traces achieves meaningful improvements of 4.9%. Our analysis reveals that low probability tokens constitute the critical bottleneck in reasoning ability transfer. However, cross-model experiments demonstrate that RSD traces are model-specific rather than universally applicable, indicating that distributional alignment must be tailored for each student architecture's unique internal representation. Left: Reasoning trace generation process where RSD produces student-friendly reasoning traces in which the teacher proposes candidate tokens, while the student accepts only those with high probability under its own distribution. Right: Average accuracy on major reasoning benchmarks (AIME24, AIME25, GPQA Diamond, and MA TH500) for (i) the base student model, (ii) a student trained on pre-existing high-quality reasoning traces (s1K-1.1), Recent advances in reasoning-focused language models have emerged through the strategic combination of reinforcement learning (RL) and supervised fine-tuning (SFT) (DeepSeek-AI et al., 2025). These two methods play distinct yet complementary roles in developing sophisticated reasoning.