RLVMR: Verifiable Meta-Reasoning for Long-Horizon Agents

The document introduces RLVMR (Reinforcement Learning with Verifiable Meta-Reasoning Rewards), a novel framework designed to enhance the performance and generalization of AI agents tackling complex, multi-step tasks. It addresses the "inefficient exploration" problem prevalent in standard reinforcement learning, where agents achieve success but through flawed or redundant actions. RLVMR integrates dense, process-level rewards for explicit cognitive behaviors like planning, exploration, and reflection, alongside the traditional final outcome reward. Experiments on benchmarks like ALFWorld and ScienceWorld demonstrate that RLVMR significantly improves success rates, reduces repetitive actions, and enhances error recovery, ultimately leading to more robust and efficient agents, even enabling smaller models to outperform larger ones. The research confirms that supervising the reasoning process itself is crucial for developing truly intelligent and adaptable AI.