Multi-Agent Evolve: LLM Self-Improvement Through Co-Evolution

This research paper introduces Multi-Agent Evolve (MAE), a novel reinforcement learning framework designed to enable large language models (LLMs) to self-improve their general reasoning abilities without relying on human-curated datasets or verifiable external rewards. MAE accomplishes this through a system where a single LLM is instantiated into three interacting roles—a Proposer that creates challenging questions, a Solver that attempts to answer them, and a Judge that evaluates both the questions and answers. This triad operates in a closed-loop co-evolution process, driven by domain-agnostic self-rewarding mechanisms like difficulty-aware and quality rewards, which allows the model to continuously generate better training material and enhance its capabilities across diverse benchmarks like mathematics, coding, and general knowledge. The experiments demonstrate that this multi-agent, self-play approach outperforms traditional Supervised Fine-Tuning (SFT), particularly highlighting its stability and effectiveness in generating a self-improving training signal.