EP148: How AI masters math through self-correction

"Dual-Phase LLM Reasoning: Self-Evolved Mathematical Frameworks" proposes a novel two-stage training framework designed to enhance the mathematical reasoning capabilities of large language models (LLMs) through supervised fine-tuning (SFT) rather than traditional reinforcement learning.The framework addresses the limitations of existing research that often relies on external model distillation or complex reinforcement learning by focusing on the model's own self-generated data. The two stages include:Stage 1: Long CoT Data Construction and Fine-tuning: The model uses a multi-turn dialogue strategy to self-generate long chain-of-thought (CoT) data that inherently embeds four critical reasoning habits: verification, backtracking, subgoal decomposition, and backward reasoning. High-quality samples are filtered using predefined rules to fine-tune the model and activate its intrinsic reasoning abilities.Stage 2: Difficulty-Aware Rejection Sampling: An iterative sampling mechanism is employed to progressively focus on complex, unsolved problems. This dynamic optimization balances the data distribution, ensuring the model receives more training signals for difficult tasks.Key Results and Impact:Performance Gains: The approach yielded significant improvements across mathematical benchmarks, including a 149% relative improvement on AIME24 and notable gains on GSM8K and MATH500.Reasoning Depth: The fine-tuned models generated reasoning chains over 4× longer than baselines, demonstrating a capacity for detailed, olympiad-level proofs.Efficiency: The method provides a resource-efficient pathway for optimization, matching the accuracy of distillation-based methods while utilizing significantly shorter response lengths and requiring no external teacher models.