Algorithmic Thinking Theory

This paper introduce a theoretical framework for studying "algorithmic thinking" in Large Language Models (LLMs), focusing on how iterative refinement and the aggregation of multiple solutions improve performance on complex reasoning tasks, like advanced mathematics problems. This framework formalizes the LLM as a **"reasoning oracle"** that generates new solutions based on a context of previous attempts, modeled by a **transfer function**. The authors define and analyze several algorithmic approaches—including **Branching**, **Genetic**, and **Random Sampling** algorithms—and establish that for certain model types, these iterative methods achieve the **maximum achievable success probability** by favoring solution independence and synthesis over simple selection. Ultimately, the work aims to move beyond empirical successes to provide a **rigorous theory** for designing highly effective, resource-efficient reasoning procedures.