Agentic Reward Modeling_Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems

This paper proposes a new reward system for large language models (LLMs) called agentic reward modeling, which aims to create more reliable rewards by integrating human preferences with verifiable correctness signals. An empirical implementation, named REWARDAGENT, is presented, which combines human preference rewards with signals related to factuality and instruction following. Extensive experiments show that REWARDAGENT outperforms traditional reward models on benchmarks and in practical applications like inference-time searches and training LLMs with DPO. The authors suggest that incorporating additional verification agents for specific scenarios could lead to more robust reward systems.