The Illusion of Thinking in Reasoning Models

This academic paper explores the strengths and limitations of Large Reasoning Models (LRMs) compared to standard Large Language Models (LLMs), specifically in problem-solving scenarios of varying complexity. The authors introduce controllable puzzle environments like Tower of Hanoi and River Crossing to systematically evaluate model performance and internal reasoning processes, contrasting with traditional, potentially contaminated mathematical benchmarks. Key findings indicate that while LRMs show advantages in medium-complexity tasks, their accuracy collapses at high complexity, a point at which their reasoning effort counterintuitively declines despite ample token budgets. The study reveals that LRMs struggle with exact computation and consistent logical execution, even when provided with explicit algorithms, highlighting fundamental limitations in their current reasoning capabilities.