Phi-3 from Microsoft 💻 // SoftBank Invests $1B in Nvidia 🤑 // HuggingFace's FineWeb Dataset 🌐

Microsoft has launched its smallest AI model yet, the Phi-3 Mini, which is designed to be smaller and cheaper to run than its larger counterparts. SoftBank plans to invest nearly $1 billion in Nvidia's chips to bolster its computing facilities and develop its own generative AI, giving Japan a strong domestic player in the AI space. HuggingFace has released FineWeb, a dataset consisting of more than 15 trillion tokens of cleaned and deduplicated English web data from CommonCrawl, which outperforms models trained on other commonly used high-quality web datasets. The papers discussed in this episode cover topics such as extending embedding models for long context retrieval, automating graphic design using large multimodal models, and Microsoft's innovative approach to training the Phi-3 Mini AI model. Contact:  [email protected] Timestamps: 00:34 Introduction 01:35 Microsoft launches Phi-3, its smallest AI model yet 03:10 SoftBank will reportedly invest nearly $1 billion in AI push, tapping Nvidia’s chips 05:11 HuggingFace Releases FineWeb: 15 Trillion tokens to train on 06:02 Fake sponsor 08:15 Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone 09:42 LongEmbed: Extending Embedding Models for Long Context Retrieval 11:04 Graphic Design with Large Multimodal Model 12:53 Outro