Advantage-Weighted Regression: Simple and Scalable Off-Policy RL

This paper introduces and explains Advantage-Weighted Regression (AWR), a simple and scalable off-policy reinforcement learning algorithm that utilizes standard supervised learning techniques. The paper details AWR's theoretical basis, highlighting its connection to constrained policy optimization and its ability to effectively handle off-policy data through experience replay. The authors demonstrate AWR's competitive performance against existing methods on benchmark tasks and complex simulated character control, also showing its strength in learning from purely static datasets. Overall, the work presents AWR as a promising and straightforward approach to reinforcement learning.