Microsoft's Latest $3.2B AI Investment 🇸🇪 // Grokfast Algorithm 💪 // Zipper Decoder Architecture 🎧

Microsoft is investing $3.2 billion in Sweden for cloud and AI infrastructure, deploying 20,000 advanced graphics processing units and training 250,000 Swedes with AI skills over three years. "Grokfast" is a new algorithm that accelerates generalization under the grokking phenomenon in machine learning by amplifying the slow-varying component of gradients, improving performance on tasks like image classification. "Zipper" is a multi-tower decoder architecture that uses cross-attention to flexibly compose multimodal generative models from independently pre-trained unimodal decoders, showcasing superior performance in tasks like speech-to-text generation. "MetRag" is a new framework for retrieval augmented generation that combines similarity and utility-oriented models, using an LLM as a task adaptive summarizer to generate knowledge-augmented text and outperforming existing models on knowledge-intensive tasks like finance and medicine. Contact:  [email protected] Timestamps: 00:34 Introduction 01:49 Microsoft to invest $3.2 bln in Swedish cloud, AI 03:42 State Space Duality (Mamba-2) Part I - The Model 04:47 Sam Altman, Lately 06:08 Fake sponsor 08:39 Grokfast: Accelerated Grokking by Amplifying Slow Gradients 10:11 Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities 11:38 Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts 13:52 Outro