Meta's AI Instagram Search 🔍 // Grok-1.5V Multimodal Model 🌟 // Scaling CLIP 📉

Meta is testing an AI-powered search bar in Instagram, which could improve the quality of search and help users discover new content on the platform. Grok-1.5V is a new multimodal model that can process a wide variety of visual information and outperforms its peers in the new RealWorldQA benchmark. "Scaling (Down) CLIP" explores the performance of the Contrastive Language-Image Pre-training (CLIP) when scaled down to limited computation budgets, and shows that smaller datasets and models can still achieve comparable performance. "Pre-training Small Base LMs with Fewer Tokens" investigates a simple approach called Inheritune to develop a small base language model (LM) from a larger existing LM, which can effectively match the val loss of their bigger counterparts when trained from scratch for the same number of training steps. Contact:  [email protected] Timestamps: 00:34 Introduction 01:40 Meta is testing an AI-powered search bar in Instagram 03:02 Grok-1.5 Vision Preview 04:56 Visualizing Attention, a Transformer's Heart 06:12 Fake sponsor 08:27 Scaling (Down) CLIP: A Comprehensive Analysis of Data, Architecture, and Training Strategies 10:11 Pre-training Small Base LMs with Fewer Tokens 11:58 Flying with Photons: Rendering Novel Views of Propagating Light 13:57 Outro