Nemotron-Labs-Diffusion

In this episode: • Welcome and the Autoregressive Bottleneck: Linda introduces the Nemotron-Labs-Diffusion paper and the hosts discuss the fundamental limits of sequential decoding. • The Best of Both Worlds: Joint Training: The hosts dive into how combining Autoregressive and Diffusion objectives creates a complementary, rather than competing, dynamic. • Drafting with Diffusion: Self-Speculation: Linda explains the third mode, self-speculation, where the model uses its diffusion pathway to draft and AR pathway to verify. • Reaching for the Speed of Light: Professor Norris and Linda discuss the SOL analysis, exploring the theoretical limits and untapped potential of diffusion parallelism. • Scaling Up and Wrapping Up: A look at the benchmark results across 3B to 14B parameters and multimodal capabilities before concluding the episode.