PODCAST · technology
The Information Bottleneck
by Ravid Shwartz-Ziv & Allen Roush
Two AI Researchers - Ravid Shwartz Ziv, and Allen Roush, discuss the latest trends, news, and research within Generative AI, LLMs, GPUs, and Cloud Systems.
Episodes
-
What’s Left for Mathematicians? | Daniel Litt
In this episode, we talked with Daniel Litt, a mathematician at the University of Toronto. In March 2025, Daniel bet that AI would not be able to autonomously produce an Annals of Mathematics-level paper by 2030. He has since conceded. He told us what changed and why. The models became superhuman...
-
The Political Economy of AI: Governance, Bias, and the Future | Andy Hall (Stanford/Anthropic)
In this episode of the the information bottleneck podcast, we talked with Andy Hall, a Stanford professor and MTS at Anthropic. The conversation explores the political economy of AGI, analyzing how superintelligence could reshape job markets, wealth redistribution, and public trust in government....
-
John Platt (Google): AI for Climate and Science
John Platt is a Google Fellow and leads the Applied Science team at Google Research, the group working on climate and science. He's also the person behind SMO and Platt scaling. In this episode, we talk with him about how Google decides which scientific problems are worth its time, and what…
more -
Yuandong Tian on Recursive Self-Improvement
Yuandong Tian spent many years at Meta FAIR and recently left to co-found Recursive Superintelligence, a company building AI that improves itself. In this episode, he tells us why. We start with his work on computer Go, where he built DarkForest before AlphaGo came out. A few years later came...
-
Chris Manning: Language Is the Real Unlock for Intelligence
Chris Manning is a legend in NLP. He's a professor of linguistics and computer science at Stanford and ran the Stanford AI Lab. His course CS224N, Natural Language Processing with Deep Learning, is where a whole generation of researchers learned the field. If you've worked on anything in NLP over...
-
Why AI Still Can't See wtih Andrew Dai
Andrew Dai spent over a decade at Google Brain and DeepMind, where he co-wrote the 2015 paper that introduced language model pre-training followed by fine-tuning, and later co-led pre-training data for Gemini. He's now co-founder and CEO of Elorian AI, which is building models for visual…
more -
Can AI Replace Doctors? Zachary Lipton on the Future of Healthcare and AI
Zachary Lipton is an associate professor at Carnegie Mellon University and a co-founder of Abridge, a healthcare AI company (and a jazz saxophonist!). His research spans machine learning, healthcare, and the broader impact of AI. In this episode, we discuss what AI can (and cannot yet) do in...
-
Sara Hooker on the End of Static AI
What comes after scaling? We talk with Sara Hooker, co-founder and CEO of Adaptation Lab, about why the next generation of AI may look very different from today's static models. Sara argues that models should continuously adapt to new tasks, data, users, and environments—and that doing this...
-
Tiny Recursive Models Beat the Giants - Alexia Jolicoeur-Martineau (Microsoft)
Alexia Jolicoeur-Martineau is a Principal Researcher at Microsoft and the author of "Less is More: Recursive Reasoning with Tiny Networks," the paper behind the Tiny Recursive Model that hit about 45% on ARC-AGI-1 with a fraction of the parameters of frontier systems. It won the 2025 ARC Prize...
-
Continual Learning Is the Next Bottleneck | Rohan Anil (Core Automation )
Rohan Anil spent eleven and a half years at Google, where he went from writing memory allocators to large-scale linear solvers, then optimization at Google Brain, where he co-developed distributed Shampoo and led optimization for PaLM and Gemini pre-training, including the work that produced…
more -
World Models | John Langford (Microsoft AI Labs)
John Langford, one of the heads of Microsoft's AI Labs, the creator of Vowpal Wabbit, and a co-inventor of CAPTCHA, joins us to talk about world models. Transformers need orders of magnitude more data than humans to learn the same thing, and John argues a compact, implicit world model is how you...
-
Which Tabular Model Should You Actually Use? | David Holzmüller (INRIA)
Description Tabular data is still where most of machine learning actually happens in industry, and the field has changed a lot in the last few years. In this episode we talk with David Holzmüller, a researcher at INRIA and one of the people behind TabArena, TabICL and RealMLP, about what the…
more -
Why You Can't Just Rent 1,000 GPUs | Charles Frye (Modal)
Charles Frye (Modal, ex-Weights & Biases, Berkeley PhD) joins Ravid and Allen to explain why modern AI research is bottlenecked by compute, and why simply buying more GPUs doesn't solve it. We cover the three problems every lab hits (underutilization, saturation, resource sharing), when companies...
-
Stella Biderman (EleutherAI) - Open Source, AI Safety, and Who We Can Trust
Stella Biderman, Executive Director of EleutherAI, joins us the week an OpenAI model autonomously broke out of its sandbox and hacked Hugging Face. Stella calls it what she thinks it is, an offensive cyber operation, and argues it's part of a pattern: this is not the first containment failure at…
more -
Why Deep Learning Finally Works on Tables | Frank Hutter (Prior Labs)
In this episode, Frank Hutter joins us to talk about TabPFN and why tabular data is suddenly the hottest problem in deep learning. Frank is a professor at the University of Freiburg and spent 15 years building the AutoML field before founding Prior Labs, which SAP just acquired for over a billion...
-
Surya Ganguli: The Physics of Intelligence
Surya Ganguli is a professor at Stanford and VP at General Catalyst, working at the intersection of physics, neuroscience, and AI. He started in string theory, moved to theoretical neuroscience, and now uses tools from statistical physics to understand both brains and neural networks. We talk…
more -
Text Diffusion Models with Brendan O'Donoghue (Google DeepMind)
Brendan O'Donoghue, research director at Google DeepMind, makes the case for text diffusion as a real alternative to autoregressive generation. He walks through how discrete diffusion works, why diffusion samples are far more diverse and what that unlocks for RL, where the Gemma diffusion model...
-
Nathan Lambert: Inside Post-Training and the Open Model Fight
Nathan Lambert spent three years as post-training lead at Ai2, where he built the OLMo models, and he writes Interconnects, one of the most-read technical newsletters in AI. He left Ai2 in June and is now working on a new project. He's also the author of the RLHF book. We talked a lot about open...
-
Daphne Koller - The Future of AI in Biology and Drug Discovery
Daphne Koller wrote the book that many of us learned probabilistic graphical models from, founded Coursera, and now runs insitro, which is trying to make drug discovery a machine-learning problem. We start with the bitter lesson. She agrees with most of it and then says where it stops working:...
-
RL Was Broken at Every Level - With Joseph Suarez (PufferAI)
In this episode, Joseph Suarez from PufferAI explains why he thinks RL never had an algorithm problem, but it had a code problem. Every part of the standard RL stack was running about a thousand times slower than it should have been, and once that got fixed, problems that used to take months...
-
The Model Found a Way Out - with Florian Brand (Prime Intellect)
Florian Brand builds evals at Prime Intellect. The premise of the conversation is that writing a benchmark is the easy part now. Keeping the model from cheating it is the job, and it takes longer than the benchmark itself. We get into why he thinks you can't evaluate a model apart from the CLI it...
-
Pierre-Carl Langlais on Building Models from Data You Can Account For
Most labs build language models by scraping the web and filtering afterward. Pierre-Carl Langlais runs it the other way around. At Pleias, the French-German lab he co-founded, the models are built from data he can actually account for, which in practice means open and public-domain sources plus a...
-
Dhruv Batra: The Browser Is a Robotics Problem - From Embodied AI at Meta to Web Agents at Yutori
Dhruv Batra spent years leading Embodied AI at Meta, training virtual robots to navigate photorealistic 3D scans of real buildings with pure reinforcement learning. Then he left to co-found Yutori and build agents for a very different environment: the web browser. In this episode, Dhruv explains...
-
How to Turn Research Into Billion-Dollar Companies, with Ion Stoica
Ion Stoica has done what almost no academic ever does — repeatedly turned university research into billion-dollar companies. He co-founded Databricks (now valued at over $100 billion), Anyscale, Arena AI and Conviva, while his Berkeley lab produced the open source projects the entire AI industry...
-
Kaggle Grandmasters, Agent Skills, and Why Everyone Is Overfitting with Jean-Francois Puget (NVIDIA)
Jean-Francois Puget is a Director and Distinguished Engineer at NVIDIA, where he leads the Kaggle Grandmasters team, and he's ranked third on Kaggle's all-time list. We caught him on the day NVIDIA announced Nemotron Ultra and its new agent skills repo. We talk about what skills actually are, why...
-
AI Agents and The Golden Age of Asking Questions with Dimitris Papailiopoulos (MSR/UW-Madison)
In this episode, we talked with Dimitris Papailiopoulos, researcher at Microsoft Research's AI Frontiers lab and professor at the University of Wisconsin, about doing research in the age of agents. Dimitris told us about the Sunday morning that changed how he works: he handed Claude Code and…
more -
Why All Models Learn the Same Thing with Phillip Isola (MIT)
Phillip Isola, professor at MIT, joins us to talk about representation learning: what makes a representation good, why different models seem to converge on similar representations, and whether pre-training is really over. We discuss the platonic representation hypothesis and its limits, why...
-
AI for Science with Qichao Hu (Molecular Universe / SES AI)
Most AI-for-science companies are selling shovels. Qichao Hu wants the gold. In this episode, we talk with Qichao, the founder and CEO of Molecular Universe, the AI-for-science platform that grew out of SES AI, a high-energy-density battery developer he's run for fourteen years. His core...
-
Infrastructure for AI at Scale - With Benny Chen (Fireworks AI)
We talk a lot on this show about RL, agents, and the move between pre-training and post-training, but not enough about the layer everything actually runs on. Benny Chen, co-founder of Fireworks AI, one of the largest inference platforms around, walks us through what it takes to serve models at...
-
Broken Peer Review, AI, and Worms — with Oded Rechavi
Oded Rechavi is a biologist at Tel Aviv University and the co-founder of QED, a company building AI to review scientific work. He's also spent years studying worms. We start with what's wrong with peer review and grant funding: why it takes years to publish, why reviewers are often your own...
-
Will AI Take Our Jobs? With Alex Imas (Google/University of Chicago)
Will AI take our jobs? We put the question to Alex Imas, the new Director of AGI Economics at Google DeepMind and a professor at Chicago Booth, whose entire job now is studying how frontier AI reshapes the economy. His short answer: probably some of them, but the popular story is mostly wrong…
more -
Why AI Benchmarks Are Lying to You - with Wenhu Chen (Meta/University of Waterloo)
In this episode, we sit down with Wenhu Chen, research scientist at Meta MSL, assistant professor at the University of Waterloo, and the person behind MMLU-Pro and MMMU. If you've read a frontier model release in the last two years, you've seen his benchmarks. That makes him one of the best…
more -
Jürgen Schmidhuber - Part 2: JEPA, the Road to AGI, and Who Really Invented Modern AI
In the second half of our conversation with Jürgen Schmidhuber, we focus on the key ideas he's pursued since the early 1990s and discuss why he believes these concepts are only now being rediscovered. We start with JEPA. Jürgen argues that the method LeCun named in 2022 is the same family he...
-
Jürgen Schmidhuber - World Models, RL, and the Year that changed AI (Part 1)
In this episode, we host Jürgen Schmidhuber - the man, the legend, one of the godfathers of modern AI. His lab worked out many ideas behind today’s systems (LSTM, world models, artificial curiosity, Transformer variants, and even GAN-style setups) decades before they became fashionable, and he’s...
-
AI for Science and the Thermodynamics of Generative AI - with Max Welling (UvA, CuspAI)
In this episode, we sit with Max Welling, Professor of Machine Learning at the University of Amsterdam, co-founder and CTO of CuspAI, and a foundational figure behind variational autoencoders (VAEs), equivariant networks, and Bayesian deep learning. We talk about AI for science, the physics...
-
After Math Falls, What's Next? with Julia Kempe (NYU/Meta)
Julia Kempe on Why Math Will Fall Next, Superhuman Provers, and the Return of the Renaissance Researcher In this episode, we sit down with Julia Kempe, a Professor at NYU's Center for Data Science and researcher at Meta FAIR's Foundations of Reasoning team, for a wide-ranging conversation on the...
-
Intelligence in an Open World - with Mengye Ren (NYU)
We talk with Mengye Ren, Assistant Professor at NYU's Center for Data Science, about what intelligence actually means once you step outside a benchmark, and why scaling a single centralized model isn't the whole story. We get into why intelligence has to be defined in open environments, not…
more -
Language, Cognition, and the Limits of LLMs - with Tal Linzen (NYU/Google)
We host Tal Linzen, Associate Professor at NYU and Research Scientist at Google, for a conversation on the intersection of cognitive science and large language models. We discussed why children can learn language from around 100 million words while LLMs need trillions, and the surprising finding...
-
The Principles of Diffusion Models - with Jesse Lai (Sony AI)
We host Chieh-Hsin (Jesse) Lai, Staff Research Scientist at Sony AI and visiting professor at National Yang Ming Chiao Tung University, Taiwan, for a conversation about diffusion models, the technology behind tools like Stable Diffusion, and most of the AI image and video generators you've seen…
more -
Inside xAI, and the Bet on AI Math - with Christian Szegedy (Math Inc)
We talked with Christian Szegedy, co-inventor of Inception and Batch Normalization, founding scientist at xAI, now at Math Inc, about what it takes to build a frontier lab, and why he left xAI to work on formal mathematics. Christian thinks Lean and auto-formalization are the missing piece for...
-
Reasoning Models and Planning - with Rao Kambhampati (Arizona State)
We sat down with Rao Kambhampati, a Professor of CS at Arizona State University and former President of AAAI, to talk about reasoning models: what they are, when they work, and when they break. Rao has been working on planning and decision-making since long before deep learning, which makes him…
more -
What Actually Matters in AI? - with Zhuang Liu (Princeton)
In this episode, we hosted Zhuang Liu, Assistant Professor at Princeton and former researcher at Meta, for a conversation about what actually matters in modern AI and what turns out to be a historical accident. Zhuang is behind some of the most important papers in recent years (with more than…
more -
The Future of Coding Agents with Sasha Rush (Cursor/Cornell)
We talked with Sasha Rush, researcher at Cursor and professor at Cornell, about what it actually feels like to we in the heart of the AI revolution and build coding agents right now. Sasha shared how these systems are changing day-to-day work and how it feels to develop these systems. A big part…
more -
The Hidden Engine of Vision with Peyman Milanfar (Google)
How Denoising Secretly Powers Everything in AI Peyman Milanfar is a Distinguished Scientist at Google, leading its Computational Imaging team. He's a member of the National Academy of Engineering, an IEEE Fellow, and one of the key people behind the Pixel camera pipeline. Before Google, he was a...
-
Reinventing AI From Scratch with Yaroslav Bulatov
Yaroslav Bulatov helped build the AI era from the inside, as one of the earliest researchers at both OpenAI and Google Brain. Now he wants to tear it all down and start over. Modern deep learning, he argues, is up to 100x more wasteful than it needs to be - a Frankenstein of hacks designed for…
more -
Why Healthcare Is AI's Hardest and Most Important Problem with Kyunghyun Cho (NYU)
We talk with Kyunghyun Cho, who is a Professor of Health Statistics and a Professor of Computer Science and Data Science at New York University, and a former Executive Director at Genentech, about why healthcare might be the most important and most difficult domain for AI to transform. Kyunghyun...
-
Diffusion LLM & Why the Future of AI Won't Be Autoregressive - Stefano Ermon (Stanford /Inception)
In this episode, we talk with Stefano Ermon, Stanford professor, co-founder & CEO of Inception AI, and co-inventor of DDIM, FlashAttention, DPO, and score-based/diffusion models, about why diffusion-based language models may overtake the autoregressive paradigm that dominates today's LLMs. We…
more -
Training Is Nothing Like Learning with Naomi Saphra (Harvard)
Naomi Saphra, Kempner Research Fellow at Harvard and incoming Assistant Professor at Boston University, joins us to explain why you can't do interpretability without understanding training dynamics, in the same way you can't do biology without evolution. Naomi argues that many structures...
-
EP28: How to Control a Stochastic Agent with Stefano Soatto (VP AWS/ Pro. UCLA)
Stefano Soatto, VP for AI at AWS and Professor at UCLA, the person responsible for agentic AI at AWS, joins us to explain why building reliable AI agents is fundamentally a control theory problem. Stefano sees LLMs as stochastic dynamical systems that need to be controlled, not just prompted. He...
-
EP27: Medical Foundation Models - with Tanishq Abraham (Sophont.AI)
Tanishq Abraham, CEO and co-founder of Sophont.ai, joins us to talk about building foundation models specifically for medicine. Sophont is trying to be something like an OpenAI or Anthropic but for healthcare - training models across pathology, neuroimaging, and clinical text, to eventually fuse...
We're indexing this podcast's transcripts for the first time — this can take a minute or two. We'll show results as soon as they're ready.
No matches for "" in this podcast's transcripts.
No topics indexed yet for this podcast.
Loading reviews...
ABOUT THIS SHOW
Two AI Researchers - Ravid Shwartz Ziv, and Allen Roush, discuss the latest trends, news, and research within Generative AI, LLMs, GPUs, and Cloud Systems.
HOSTED BY
Ravid Shwartz-Ziv & Allen Roush
CATEGORIES
Loading similar podcasts...
Frequently Asked Questions
How many episodes does The Information Bottleneck have?
The Information Bottleneck currently has 75 episodes available on PodParley. New episodes are automatically indexed when they're published to the podcast feed.
What is The Information Bottleneck about?
Two AI Researchers - Ravid Shwartz Ziv, and Allen Roush, discuss the latest trends, news, and research within Generative AI, LLMs, GPUs, and Cloud Systems.
How often does The Information Bottleneck release new episodes?
The Information Bottleneck has 75 episodes. Check the episode list to see recent publication dates and frequency.
Where can I listen to The Information Bottleneck?
You can listen to The Information Bottleneck on PodParley by clicking any episode. We provide an embedded audio player for direct listening, and you can also subscribe via your preferred podcast app using the RSS feed.
Who hosts The Information Bottleneck?
The Information Bottleneck is created and hosted by Ravid Shwartz-Ziv & Allen Roush.
URL copied to clipboard!