#192 Lukas Biewald: How Weights and Biases Supercharges Machine Learning

This episode is sponsored by Oracle. AI is revolutionizing industries, but needs power without breaking the bank. Enter Oracle Cloud Infrastructure (OCI): the one-stop platform for all your AI needs, with 4-8x the bandwidth of other clouds. Train AI models faster and at half the cost. Be ahead like Uber and Cohere. If you want to do more and spend less like Uber, 8x8, and Databricks Mosaic - take a free test drive of OCI at https://oracle.com/eyeonai In this episode of the Eye on AI podcast, join us as we sit down with Lukas Biewald, CEO & co-founder of Weights & Biases, the AI developer platform with tools for training models, fine-tuning models, and leveraging foundation models.  Lukas takes us through his journey, from his early days at Stanford and his work in natural language processing, to the founding of CrowdFlower and its evolution into a major player in data annotation. He shares the insights that led him to start Weights and Biases, aiming to provide comprehensive tools for the entire machine learning workflow. Lukas discusses the importance of high-quality data annotation, the shift in AI applications, and the role of reinforcement learning with human feedback (RLHF) in refining large models. Discover how Weights and Biases helps ML practitioners with data lineage and compliance, ensuring that models are trained on the right data and adhere to regulatory standards. Lukas also highlights the significance of tracking and visualizing experiments, retaining intellectual property, and evolving the company's products to meet industry needs. Tune in to gain valuable insights into the world of ML Ops, data annotation, and the critical tools that support machine learning practitioners in deploying reliable models. Don't forget to like, subscribe, and hit the notification bell for more on groundbreaking AI technologies. Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI (00:00) Preview and Intro (01:39) Lukas's Background and Career  (04:09) Founding CrowdFlower and Early Machine Learning  (06:59) Current Trends in Machine Learning (08:46) Reinforcement Learning with Human Feedback (RLHF)  (12:43) Weights and Biases: Origin and Mission   (16:44) Visualizations and Compliance in AI  (22:43) US vs. EU AI Regulations (25:20) Importance of Experiment Tracking in ML  (28:47) Evolving Products to Meet Industry Needs  (30:38) Prompt Engineering in Modern AI  (33:34) Challenges in Monitoring AI Models  (37:25) Monitoring Functions of Weights and Biases (39:33) Future of Weights and Biases