Test-Time Scaling Makes Overtraining Compute-Optimal

The final episode of Topic 3 closes the loop by picking a fight with Topic 2. Chinchilla's compute-optimal recipe balances model size against training tokens under a training budget — but deployed reasoning systems don't pay for one forward pass per task. They sample candidates and vote, run verifiers, search, retry. Today's paper, Test-Time Scaling Makes Overtraining Compute-Optimal, writes down Train-to-Test — T-squared — scaling laws: one end-to-end budget, three knobs turned jointly — model size, training tokens, and inference samples. Maya builds the accounting through a math olympiad that suddenly allows unlimited submissions per problem, then the two meters every model runs: a training meter that spins once and a serving meter that spins on every query. Leo mounts the canon's defense — training compute is a number you know, deployment forecasts are guesses — and the debate resolves on what would settle it: a stated deployment profile. The evidence: across eight downstream tasks, counting inference cost shifts optimal pretraining into an overtraining regime well outside what standard scaling suites explore. The topic ends where it started, with the sixty-four-GPU team — now asking whether the hundred-billion model was the right model to train at all. Sources: • Test-Time Scaling Makes Overtraining Compute-Optimal: https://arxiv.org/pdf/2604.01411 • Embarrassingly Simple Self-Distillation Improves Code Generation: https://arxiv.org/pdf/2604.01193 • Training Compute-Optimal Large Language Models (Chinchilla): https://arxiv.org/abs/2203.15556