EPISODE · May 2, 2026 · 8 MIN
Stanford CS336 2026 L4: Linear Time Attention and Sparse Architectural Alternatives
from Steven AI Talk · host Steven
How do we scale LLMs beyond current limits? This lecture explores the transition from quadratic attention to linear alternatives and the rise of sparse Mixture of Experts (MoE).Topics Covered:The fundamental bottleneck of Transformers.RNN-like inference speed with linear attention.How MoE partitions parameters for efficiency.Optimizing for hardware with shared experts and MLA.Full videos in youtube, tiktok, substack, etcSubscribe for more SOTA AI research summaries! All my links: https://linktr.ee/learnbydoingwithsteven#CS336 #StanfordAI #MoE #Architecture #LearnByDoingWithSteven #StevenDataTalk #数能生智
What this episode covers
How do we scale LLMs beyond current limits? This lecture explores the transition from quadratic attention to linear alternatives and the rise of sparse Mixture of Experts (MoE).Topics Covered:The fundamental bottleneck of Transformers.RNN-like inference speed with linear attention.How MoE partitions parameters for efficiency.Optimizing for hardware with shared experts and MLA.Full videos in youtube, tiktok, substack, etcSubscribe for more SOTA AI research summaries! All my links: https://linktr.ee/learnbydoingwithsteven#CS336 #StanfordAI #MoE #Architecture #LearnByDoingWithSteven #StevenDataTalk #数能生智
NOW PLAYING
Stanford CS336 2026 L4: Linear Time Attention and Sparse Architectural Alternatives
No transcript for this episode yet
Similar Episodes
No similar episodes found.
Similar Podcasts
No similar podcasts found.