Preventing AI Hallucinations

Anand Kannappan (@anandnk24, CEO @PatronusAI) talks about evaluating AI models for hallucinations, managing data quality, automating the process, and optimizing models. SHOW: 927 SHOW TRANSCRIPT: The Cloudcast #927 Transcript SHOW VIDEO: https://youtube.com/@TheCloudcastNET CLOUD NEWS OF THE WEEK: http://bit.ly/cloudcast-cnotw NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST: "CLOUDCAST BASICS" SPONSORS: [VASION] Vasion Print eliminates the need for print servers by enabling secure, cloud-based printing from any device, anywhere. Get a custom demo to see the difference for yourself. [US CLOUD] Cut Enterprise IT Support Costs by 30-50% with US Cloud SHOW NOTES: Patronus AI website Topic 1 - Welcome to the show, Anand. Give everyone a quick introduction. Topic 2 - Our topic today is Preventing AI Model Hallucinations. Before we dig into that, I wanted to ask about your time as Lead Data Scientist at Meta. What was it like to be early into that organization, and what did you take away from your time there? Topic 3 - Ok, let’s dig into model evaluations and hallucinations. Let’s start at the beginning. How do model hallucinations come about? Topic 4 - When evaluating models for hallucinations, how does a developer or a data scientist know fact from fiction? Due to its size, complexity, and number of parameters, it’s not feasible to simply fact-check and manually verify inputs to outputs. How is this process evaluated and automated with some level of confidence? Additionally, numerous benchmarks are available. What are your thoughts on the usefulness of the benchmarks? Topic 5 - How does the concept of data quality play into this? How would we know when a model was given insufficient or improper data vs. a hallucination Topic 6 - We often hear about how frontier models are running out of training data, and increasingly, synthetic data is being used. Does this impact hallucinations in any way? Topic 7 - The last item I wanted to ask you about, Partonus AI, also pertains to model optimization. Can you explain that process? FEEDBACK? Email: show at the cloudcast dot net Bluesky: @cloudcastpod.bsky.social Twitter/X: @cloudcastpod Instagram: @cloudcastpod TikTok: @cloudcastpod FEEDBACK? Email: show @ the enterprise ai show dot com Bluesky: @TheEntAIShow.bsky.social Twitter/X: @TheEntAIShow Instagram: @TheEntAIShow