Building a School Where AI Models Learn About Humanity

If scaling laws hold—and Surge AI CEO Edwin Chen believes they do—we’re hurtling toward a future where there’s nothing humans can do that AI can’t do better. When OpenAI’s models disproved an open conjecture posed by mathematician Paul Erdős using novel algebraic geometry techniques, Fields medalist Timothy Gowers felt the shift acutely. He initially thought the model had proved an upper bound, and braced himself: that would mean it was “all over for mathematicians very soon.” When he realized it had only found a counterexample, he was relieved—it bought him another year or two before the thing he’s devoted his life to becomes something AI does better.As founder and CEO of the company behind the data environments and evals the major model companies use to train their models, Chen has a unique perspective on how quickly AI models are absorbing tasks we used to think of as uniquely human.Dan Shipper talked with Chen for AI & I about what the act of creating or building means when AI can do it better—and whether an answer to that question already exists within science fiction.If you found this episode interesting, please like, subscribe, comment, and share!Join the membership for Where You Live at ⁠https://www.joinbilt.com/danTo hear more from Dan Shipper:Subscribe to Every: https://every.to/subscribeFollow him on X: https://twitter.com/danshipperTimestamps:00:00:54 Introduction00:01:49 Surge as a "school for AGI"00:04:46 What AI's capacity for novel mathematics says about human achievement00:07:29 Motivation in an era when AI can do everything00:14:34 The trap of optimizing AI models for engagement00:29:34 Training using datasets versus training using environments00:35:09 The value of personal data00:39:40 Why models are bad at writing00:42:00 Chen's AGI timelineLinks to resources mentioned in the episode:Edwin Chen on X: https://x.com/echenSurge: https://surgehq.aiRiemann-bench (research-level math benchmark): https://surgehq.ai/leaderboards/riemann-benchHemingway-bench (creative writing benchmark): https://surgehq.ai/leaderboards/hemingway-benchTalkie-1930 (language model trained on pre-1930 text): https://huggingface.co/talkie-lm/talkie-1930-13b-itTed Chiang, “What’s Expected of Us”: https://www.nature.com/articles/436150aEvery is the most AI-native startup on the internet. Through ideas, software and education, subscribers get the tools to work at the frontier of AI. Start your free trial today: https://every.to/subscribe?utm_source=youtubeFollow Every: https://x.com/everyFollow Dan Shipper: https://x.com/danshipper