RL Was Broken at Every Level - With Joseph Suarez (PufferAI)

In this episode, Joseph Suarez from PufferAI explains why he thinks RL never had an algorithm problem, but it had a code problem. Every part of the standard RL stack was running about a thousand times slower than it should have been, and once that got fixed, problems that used to take months started getting solved in seconds on one GPU. We talk about what makes a simulator good for RL, why most of their sims run on CPU, what he wants to do with scientific simulation, and why he open sources all of it instead of writing papers. Key topicsTypes of RL and their applicationsChallenges in scaling reinforcement learningThe role of simulators and hardware in RLRL in gaming: from chess to complex games like NetHack and RuneScapeFuture directions: scientific simulation and biological modelingChapters00:00 - Introduction to RL and Puff AI01:50 - Different settings for RL: Games, Robots, Finance04:10 - RL in LM and other domains07:00 - Challenges and solutions in RL scaling09:55 - Building fast, efficient simulators15:10 - RL for scientific research and simulation19:57 - RL in complex games: NetHack, RuneScape, Dwarf Fortress29:55 - Future of RL: Scientific discovery and beyondResourcesPuff AI - Official Site - https://puffer.aiNetHack - https://www.nethack.org/RuneScape - https://www.runescape.com/Dwarf Fortress - http://www.bay12games.com/dwarves/OpenAI Gym - https://github.com/openai/gymMusic"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.