Value-Guided Search for Efficient Chain-of-Thought Reasoning

This paper introduces Value-Guided Search (VGS), a novel method for improving the reasoning capabilities and efficiency of large language models (LLMs) on complex tasks like competition math. Unlike prior methods that rely on fine-grained, step-by-step feedback, VGS uses a token-level value model trained on large datasets of reasoning traces. This model guides a block-wise search process, selecting the most promising continuations at intervals rather than individual steps. The paper demonstrates that VGS significantly enhances performance and reduces the computational resources required compared to existing techniques like majority voting or search guided by process reward models. The authors also release their dataset, model, and codebase to support future research.