Skip to content
Why AI Inference Is So Expensive (And How Positron Is Solving It) episode artwork

Apr 3, 2026 29 min

Why AI Inference Is So Expensive (And How Positron Is Solving It)

from Built This Week Hosted by Jordan Metzner, Samuel Nadler

Training gets the headlines. Inference is where the money is. In Episode 37 of Built This Week, we sit down with Mitesh, CEO of Positron AI, to break down one of the biggest bottlenecks in AI today: inference infrastructure. While the world focuses on trillion-parameter models and frontier labs, the real constraint isn’t intelligence — it’s memory, bandwidth, energy, and cost. We cover: • Why inference is where 90% of AI spend happens • The memory wall problem in large models • Why GPUs weren’t designed for text generation • How Positron is building terabyte-plus memory chips • The economics of 10 trillion parameter models • Why memory bandwidth utilization matters • Why CPUs are suddenly back in demand • The difference between speed-optimized and cost-optimized AI systems • The slider bar future of AI infrastructure We also dive into: • OpenAI’s $122B valuation • Anthropic vs OpenAI secondary market dynamics • Why Nvidia isn’t going anywhere • Why commodity memory might beat premium stacks in certain use cases • The rise of agentic workflows and what that means for compute If you care about the future of AI, silicon, infrastructure, or trillion-dollar companies — this episode is for you. New episodes every Friday. ⏱ TIMESTAMPS (0:00) Why inference is the real AI bottleneck (2:00) What Positron AI is building (4:30) The memory problem in trillion-parameter models (6:30) Why GPUs struggle with inference economics (9:00) Energy, bandwidth, and supply chain constraints (12:00) Memory capacity vs memory speed tradeoffs (16:00) The “slider bar” model of AI infrastructure (18:30) OpenAI’s $122B valuation discussion (21:00) Anthropic vs OpenAI secondary markets (23:30) CPUs making a comeback (26:00) Agentic workflows and compute demand explosion (28:00) Closing thoughts on AI infrastructure

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Why AI Inference Is So Expensive (And How Positron Is Solving It)

0:00 29:03

Chapters

  1. 0:00 Why is AI inference so expensive, and how is Positron AI solving it?
  2. 1:07 How does Positron AI's 2 TB memory chip cut inference costs?
  3. 4:19 Why did Lambda cofounder Mitesh Agrawal join Positron AI as CEO?
  4. 6:31 Do GPUs or dedicated inference chips cost less per AI token?
  5. 9:52 Which AI inference workloads can beat Nvidia GPUs on efficiency?
  6. 12:06 Why will AI users always want more memory and context length?
  7. 14:17 Why does Positron AI use commodity memory instead of HBM?
  8. 16:13 Can you trade off AI inference speed against cost?
  9. 19:23 Is OpenAI falling out of favor with secondary market buyers?
  10. 23:52 Why is AI agent demand bringing back the CPU?

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Built This Week?

This episode is 29 minutes long.

When was this Built This Week episode published?

This episode was published on April 3, 2026.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this Built This Week episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.