RL for Image Generation: DPO vs GRPO

This source evaluates and compares two reinforcement learning algorithms, GRPO and DPO, for their effectiveness in generating images from text descriptions. The research investigates how different reward models, designed to assess image quality and human preferences, influence the performance and generalization capabilities of these algorithms. Additionally, the study examines the impact of various scaling strategies—specifically, increasing the number of sampled images per prompt and augmenting the diversity of training data—on the in-domain proficiency and out-of-domain generalization of both GRPO and DPO in text-to-image generation. The paper presents empirical results and visual examples to illustrate the findings and contributions to the field of autoregressive image generation.