Inside the AI Bottleneck: Data Movement, Chiplets, and System Scaling

For the last decade AI has been driven by one thing, more compute: bigger models, more accelerators, higher throughput.But as NVIDIA’s Jensen Huang recently highlighted at GTC, the industry is hitting a different kind of wall, one that hasn’t received nearly as much attention.The real constraint is no longer just compute. It’s data movement.To its credit, Nvidia has pushed this frontier with innovations like NVLink Fusion, and continued investment in connectivity AI dataflow architectures. But the challenge is bigger than any one company.As AI systems scale to hundreds - and even thousands - of processors, performance is increasingly defined by the ability to efficiently move, synchronize, and manage data across increasingly distributed architectures that can orchestrate data across chiplets, nodes, and entire racks.In this episode of Tech Threads, we bring together a panel of deeply experienced technologists, architects and leaders from companies like Intel, Arm, Altera, Texas Instruments, and Arteris - individuals who have helped shape modern compute, interconnect standards, and system architecture.Together they explore what is really changing beneath the surface: why traditional scaling approaches are breaking down, how coherent interconnects and network-on-chip architectures are evolving, and why system-level thinking is becoming essential.They also dive into the growing complexity introduced by chiplet-based designs, heterogeneous compute, and distributed memory systems and what it takes to maintain performance, efficiency, and programmability at scale.This is not just a technology shift, it’s an architectural reset.If you’re building or thinking about next-generation AI systems, this conversation gets to the heart of what matters next.