Sutton and Silver: The Era of Experience: Learning Beyond Human Data

Along with Sutton's recent conversation with Hannah Fry, we review Sutton and Silver's recent paper on Era of Experience in AI, contrasting it with the current Era of Human Data where models learn primarily by imitating human-generated content. The authors argue that progress based solely on human data is plateauing, necessitating a shift towards agents learning from their own interactions with the environment. This new era will involve agents experiencing continuous streams of data, acting autonomously with richer observations and actions, and learning from grounded rewards derived from the environment itself, enabling reasoning and planning beyond human limitations and potentially leading to superhuman intelligence. The transition to experiential learning, while promising unprecedented capabilities, also presents novel challenges and safety considerations that require careful attention.