Agent Lightning: Training Any AI Agents with Reinforcement Learning

This paper introduces **Agent Lightning**, a novel framework designed to enhance the training of **Large Language Models (LLMs)** within **AI agents** using **Reinforcement Learning (RL)**. A key innovation is the **complete decoupling** of agent execution from the RL training process, allowing for seamless integration with existing agents without significant code changes. This is achieved by formulating agent execution as a **Markov Decision Process (MDP)**, which defines a **unified data interface** to transform agent trajectories into training transitions. The framework also proposes **LightningRL**, a hierarchical RL algorithm, and a **Training-Agent Disaggregation architecture** to standardize the training service, proving its efficacy across various tasks like text-to-SQL and retrieval-augmented generation.