Welcome to the Era of Experience

This "bitter lesson"-style position paper by David Silver and Richard S. Sutton introduces a shift in artificial intelligence from the "Era of Human Data" to the "Era of Experience." The authors argue that AI progress is slowing because the available pool of human-generated data is being exhausted, necessitating a new approach where agents learn predominantly from their own self-generated experience and interaction with the environment. This transition is characterized by agents that inhabit streams of experience rather than short interactions, utilize richly grounded actions and observations, optimize for grounded rewards from the environment, and engage in non-human planning and reasoning to achieve truly superhuman capabilities. The paper positions this new era as a critical next step that will reconcile the task-generality achieved by large language models with the self-discovery of knowledge exemplified by earlier reinforcement learning systems.