On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models

This paper details a controlled experimental framework used to examine the interaction between pre-training, mid-training, and reinforcement learning (RL) on the reasoning abilities of language models (LMs). Researchers from Carnegie Mellon University and the Language Technologies Institute utilized a synthetic dataset with explicitly defined reasoning complexity and contextual templates to isolate the causal effect of each training stage. Key findings indicate that RL yields true capability gains only when targeting the model's "edge of competence," where tasks are difficult but still within reach of generalization. Furthermore, minimal pre-training exposure to long-tail contexts is critical for RL to induce robust contextual generalization, and incorporating a mid-training phase substantially improves performance under a fixed computational budget. Finally, the study confirms that process-aware rewards effectively mitigate reward hacking and enhance reasoning fidelity.