Nathan Lambert: Inside Post-Training and the Open Model Fight

Nathan Lambert spent three years as post-training lead at Ai2, where he built the OLMo models, and he writes Interconnects, one of the most-read technical newsletters in AI. He left Ai2 in June and is now working on a new project. He's also the author of the RLHF book. We talked a lot about open models, their capabilities, and why they are better than he expected. We get into what that means over the next two to five years, why he thinks recursive self-improvement is overblown, what the market for training environments actually looks like now, and why he expects Anthropic's famously open internal culture to break after its IPO.Key TopicsOpen vs closed models and who actually captures the valueAnthropic and OpenAI as opposite cultures, and the talent concentration problemBoom vs bubble, and why token spend hasn't produced 10x better productsContinual learning, RSI skepticism, and what Nathan wants to work on nextWhat the open ecosystem needs economically to surviveTimeline 00:00 Intro00:27 Open vs closed models, and who actually captures the value05:12 China, harnesses, and where the real training leverage sits08:40 Sovereign compute and the national security case for building models11:18 Uncensored open weights and the bioweapon question14:29 Anthropic vs OpenAI, ideology and politics19:35 The Mythos ban and the Fable 5 delays24:30 The AGI narrative, the talent drain, and antitrust28:12 Why researchers join Anthropic, and the open Slack culture34:04 Nathan's next 12 months: character training and big RL runs37:55 Continual learning, RSI, and why Nathan is skeptical43:19 Boom or bubble, tokens vs GPUs45:12 Why all that token spend never produced 10x products48:38 Job displacement and the small-business future52:49 Robotics, world models, and why multimodal lags57:44 What the open ecosystem should actually do1:03:17 Why NVIDIA isn't building a frontier model1:07:34 The RLHF book, and whether RLHF still matters1:11:06 GRPO vs PPO and on-policy distillationMusic"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.