Agent RL Scaling for Mathematical Problem Solving

This academic paper explores ZeroTIR, a novel method for training Large Language Models (LLMs) to spontaneously use external tools, specifically Python code execution, for mathematical problem-solving through Reinforcement Learning (RL). The authors identify and characterize Agent RL Scaling Laws, demonstrating that as RL training progresses, there are predictable increases in code execution frequency, response length, and task accuracy. They propose and implement an efficient framework (ARL) for this training, showing that ZeroTIR models (ZTRL)significantly outperform traditional RL methods without tool integration on challenging math benchmarks. The research highlights that while increased interaction potential improves results, models tend to converge on strategies that favor fewer, high-utility code calls for successful solutions.