Accelerating Unbiased LLM Evaluation via Synthetic Feedback

This paper proposes Control Variates Evaluation, a method for efficiently evaluating large language models (LLMs) that reduces reliance on expensive human annotations. While synthetic feedback from other LLMs is cheaper, it introduces bias. This new approach combines human and synthetic feedback to achieve unbiased win-rate calculations with significantly fewer human annotations. Experiments demonstrate a considerable reduction in human annotations and show that fine-tuning synthetic evaluators can further improve these savings. The method also offers a predictable measure of potential annotation reduction based on the correlation between human and synthetic judgments.