EP167: Why AI models ignore visual evidence

Paper Link: https://arxiv.org/abs/2603.00873Summary:MC-SEARCH is a new benchmark designed to evaluate and improve multimodal large language models (MLLMs) as they transition from simple retrieval to complex, agentic reasoning. While older datasets focus on short, single-step tasks, this framework provides 3,333 high-quality examples featuring long reasoning chains that average nearly four hops in length. These examples are categorized into five distinct reasoning structures, such as image-initiated or parallel forks, to test how models coordinate text and visual data. The researchers also introduced HAVE, a verification process that ensures every step in a reasoning chain is necessary and grounded in evidence. To move beyond final answer accuracy, the benchmark uses process-level metrics like Hit per Step and Rollout Deviation to identify specific errors like over-retrieval or planning misalignment. Finally, the authors present SEARCH-ALIGN, a fine-tuning method that uses these verified chains to significantly boost the planning and retrieval fidelity of open-source models.