Is one layer enough? Training a single transformer layer can match full-parameter RL training

This paper explores a surprising structural property of large language models: most reinforcement learning (RL) gains are concentrated in a very small subset of transformer layers. By isolating and training individual layers, researchers discovered that optimizing just a single middle layer can match or even exceed the performance of full-parameter RL training. This phenomenon was remarkably consistent across multiple model families like Qwen3 and Qwen2.5, various RL algorithms, and diverse tasks including mathematics, coding, and agentic decision-making. The study reveals that layers near the input and output ends contribute significantly less to post-training improvements than those in the 40%–60% depth range. Leveraging these insights, the authors developed layer-aware training strategies that prioritize these high-contribution layers to outperform standard uniform training methods. Additionally, the findings suggest that different layers capture complementary problem-solving behaviors, which can be combined through majority voting for further accuracy gains. Overall, the work challenges the assumption that RL adaptation must be distributed throughout a network and offers a more efficient, targeted approach to LLM post-training.