RL, Reasoning, Reward Hacking, AI Timeline and Post AGI | Will Brown (Research Lead at Prime Intellect)

Will Brown is Research Lead at Prime Intellect.TIMESTAMPS(00:00:00) - INTRO(00:02:00) - Career Trajectory, Love for Maths and Algorithms(00:06:24) - Early Motivation and PhD experience(00:12:22) - Teaching, Open Source, GenAI Handbook(00:17:27) - Rubrik Engineering and Approach to Design(00:23:06) - RL over Instruction Tuning(00:26:07) - Journey behind VERIFIERS (00:33:30) - RL experiments on SLMs(00:37:06) - Decentralized Training and Inference(00:40:28) - Self-Improving agents(00:44:05) - GRPO in Multimodal settings, Robotics(00:49:00) - Challenges in Curriculum Learning(00:52:02) - Kimi Dev 72B?(00:54:55) - Reinforcement Pre-training(00:57:53) - Do LLMs have hit a wall (in novelties)?(01:02:06) - LLM and Classical Agents(01:05:10) - LLMs as Evaluators or Doers(01:07:19) - Do models know when they're Reward Hacking?(01:12:42) - Training agents in diverse environments(01:14:55) - What if solving reward hacking accidentally gives us alignment?(01:16:28) - Will AGI collapse decentralized system into centralized one?(01:19:24) - The Illusion of the Illusion of Thinking(01:20:21) - LLMs in Quant Trading(01:22:21) - Thoughts on Pattern Matching in Intelligence?huhhh, let's relax!(01:23:55) - Kalomaze's hidden secret(01:24:30) - Which frontier lab Will is bullish on?(01:26:57) - Finest teams in AI Research(01:30:45) - Is xAI late compared to other frontier labs?(01:32:35) - How to become an Experimentalist?(01:33:49) - Underrated mindset that has been an edge in Will's research?(01:34:41) - "Cheat Code" in RL Research that has been game changing?(01:35:45) - What Will think about AGI?(01:38:01) - Will's advice to 20yo high school/college grads(01:39:23) - Classical ML in current age of AI Research(01:41:10) - What Will have to say for a general audience?(01:42:29) - Concluding!