DeepMind's AI Soundtracks 🎥 // Challenges of Training AI Clusters ⚡ // Large Language Model Factual Knowledge 🤯

Google DeepMind's new AI tool that generates video soundtracks by combining text prompts with visual content. Challenges of building large training AI clusters, including power, network topology, and reliability. How large language models acquire factual knowledge during pretraining and their probabilistic reasoning capabilities. LLARVA's vision-action instruction tuning that enhances robot learning. Contact:  [email protected] Timestamps: 00:34 Introduction 01:47 Google DeepMind’s new AI tool uses video pixels and text prompts to generate soundtracks 03:31 100,000 H100 Clusters: Power, Network Topology, Ethernet vs InfiniBand, Reliability, Failures, Checkpointing 05:22 Large language model data pipelines and Common Crawl (WARC/WAT/WET) 06:47 Fake sponsor 08:20 How Do Large Language Models Acquire Factual Knowledge During Pretraining? 10:01 What Are the Odds? Language Models Are Capable of Probabilistic Reasoning 11:22 LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning 13:06 Outro