Evaluating AI Models: Cost, Latency, and Quality: A conversation with Ivan Lee, founder/CEO of Datasaur

In this podcast episode of AI Realized, Ivan Lee, founder and CEO of Datasaur, discusses his company's mission to democratize access to natural language processing (NLP). Datasaur introduced a data labeling platform used by major organizations like Netflix and the FBI and launched LLM Labs to facilitate the customization of large language models (LLMs). Ivan elaborates on the importance of evaluating AI models in production, focusing on cost, latency, and quality, and introduces the concept of prompt unit testing to ensure consistent model performance. He highlights the need for data scientists to understand business-side ROI and notes the significant unit costs associated with LLMs. Ivan explores the potential for cost reduction driven by innovations like OpenAI's GPT 4o Mini and open-source models like Lama. He also considers the implications of running LLMs on-device for industries with strict data privacy needs. Lastly, Ivan advises enterprise executives to start small, piloting AI solutions to demonstrate their effectiveness, and emphasizes the future of a multi-model AI solution landscape within organizations.00:00 Introduction to AI Realized Podcast01:04 Meet Ivan Lee, CEO of Datasaur01:18 Datasaur's Mission and Products02:31 Ensuring Quality in AI Models03:51 Calculating ROI for AI Use Cases06:41 Cost Reduction in AI07:47 Future of AI Deployment09:46 Advice for Enterprise Executives12:22 Key Takeaways and Conclusion