Personalized Policy Learning from Heterogeneous Data

This document introduces a novel framework for offline reinforcement learning (RL), focusing on optimizing individual policies when data comes from diverse or heterogeneous populations. The authors propose using individualized latent variables within a shared heterogeneous model to efficiently estimate unique Q-functions for each individual. Their Penalized Pessimistic Personalized Policy Learning (P4L) algorithm offers theoretical guarantees for a fast average regret rate under a weak partial coverage assumption. The research highlights the limitations of traditional RL methods that assume population homogeneity, which often lead to suboptimal policies for diverse groups. Simulation studies and a real-world application in intensive care demonstrate the superior performance of their proposed method compared to existing approaches.