Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning

This research introduces Posterior Behavioral Cloning (POSTBC), a novel pretraining method designed to enhance the reinforcement learning (RL) finetuning of robotic policies. Traditional behavioral cloning (BC) often fails because it overfits to specific demonstration data, resulting in poor action coverage and limited exploration during subsequent online learning. By modeling the posterior distribution of demonstrator behavior rather than simply mimicking actions, POSTBC injects uncertainty-aware entropy into the policy's action distribution. This ensures the robot maintains high performance in familiar scenarios while exploring a diverse range of actions in low-density data regions. Experimental results across simulation and real-world robotics demonstrate that this approach significantly improves the efficiency of RL finetuning without sacrificing initial pretraining quality. Ultimately, POSTBC provides a more robust initialization for autonomous systems, allowing them to adapt to new tasks with fewer samples.