The Right Model for the Job

Lauren Gold and Kustomer CTO and co-founder Jeremy Suriel discuss the architectural considerations for building AI into a customer service platform, including model selection, orchestration layer, and privacy obligations. They also explore the debate around training own models versus relying on foundation models from big labs and the evolving role of in-house AI teams in B2B SaaS companies. Takeaways Model Selection: The approach to AI model selection involves a tiered architecture, with each model chosen based on its specific strengths and requirements. Orchestration Layer: Building a flexible orchestration layer allows for the integration of multiple AI models, providing customers with meaningful control over AI configurations. Privacy Obligations: Zero data retention agreements and residency matters are key considerations for maintaining customer trust and privacy obligations. Build vs. Buy: The conversation highlights the debate around training own models versus relying on foundation models from big labs, emphasizing the importance of building around the model rather than the model itself. In-House AI Teams: The evolving role of in-house AI teams in B2B SaaS companies involves integration, domain expertise, and bridging the gap between research and production. Chapters 00:00 Architecting AI for Customer Service Platforms 05:35 Orchestration Layer and Flexibility 17:34 Privacy Obligations and Data Residency