LTR 95: The case for software risk management in AI-driven home health devices

Summary“If there is a risk in your design for a device used in a hospital setting, you have qualified people there. But at home, there is no more safety net. So (AI) software risk becomes even more important.”In this Let’s Talk Risk! conversation, Erhan Ilhan shares how software is increasingly being used as a medical device in our industry. As a result, we have to pay more attention to software risk management. Software failure will occur, but we need to understand the sequence of events leading to hazardous situations and think of controls to mitigate their impact. In this episode, Naveen Agarwal and Erhan Ilhan discuss the complexities of software risk management in the medical device industry. They explore the differences between software and hardware risks, the importance of usability testing, the challenges of estimating software failure probabilities, and the integration of cybersecurity risks into overall risk management. The discussion also highlights the significance of post-market surveillance and the impact of emerging technologies like AI and ML on medical devices.Listen to the full 30-minute podcast or jump to a section of interest listed below. Chapters01:05 Introducing Erhan Ilhan02:10 Understanding Software Risk vs. Hardware Risk03:12 Understanding Software Failures and Estimating Probability05:20 Potential Effect of Recent Update to IEC 6230406:36 Importance of Post-Market Surveillance in Software Risk Management07:42 Usability Testing and Human Factors in Software Risk Management12:35 Estimating Probability of Software Failure in FMEA18:30 Cybersecurity Risk Management for Software as a Medical Device22:55 AI/ML, GenAI in Medical Devices and Home Health29:42 Final Thoughts and Key TakeawaysIf you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.Suggested links:LTR: Tips for medical device software risk analysis. LTR: Cloud computing in MedTech. ACHIEVE: Post-market surveillance workshop.Key Takeaways* Software risk management is essential for patient safety.* Software does not pose direct harm like hardware can.* Assuming a software failure probability of one is incorrect.* Real-world data is crucial for improving software safety.* Usability testing should involve diverse user demographics.* Cybersecurity risks must be assessed alongside safety risks.* Post-market surveillance is vital for ongoing risk management.* AI and ML are transforming the medical device landscape.* Risk control measures should be clearly defined and integrated.* Always prioritize patient safety in software development.KeywordsSoftware risk management, medical devices, usability testing, cybersecurity, post-market surveillance, AI, ML, IEC 62304, ISO 14971, quality assuranceAbout Erhan IlhanErhan Ilhan is the Head of Quality and Regulatory at Circadia Health, where he has been building the company’s Quality Management System (QMS), driving ISO 13485 implementation, and leading the regulatory strategy for Circadia’s software and hardware-based medical devices. His work includes preparing 510(k) submissions, managing FDA interactions, and leading AI/ML-focused pre-submissions to help define the regulatory pathway for Circadia’s AI-driven clinical decision support software.At Circadia, Erhan also leads design quality assurance activities, oversees risk management and cybersecurity compliance, and works closely with cross-functional teams to embed quality into every stage of product development — from design to manufacturing and post-market surveillance.Prior to Circadia, Erhan held various engineering and quality leadership roles at GE Healthcare, Abbott, Danaher, Medtronic, and Glooko. With deep expertise in software development, design controls, verification and validation, design transfer, test method development, production and process controls, risk management, and regulatory submissions, he has contributed to a wide range of medical device and digital health products. His work spans implantable pacemakers and neuromodulation devices, Continuous Glucose Monitoring (CGM) systems, insulin pumps, diabetes management platforms, mobile and web health applications, and cloud-based algorithm engines.Erhan holds a Bachelor’s degree in Industrial and Electrical Engineering and a Master’s degree in Computer Science, blending technical depth with a strong foundation in quality and regulatory compliance.Let’s Talk Risk! with Dr. Naveen Agarwal is a weekly live audio event on LinkedIn, where we talk about risk management related topics in a casual, informal way. Join us at 11:00 am EST every Friday on LinkedIn.DisclaimerInformation and insights presented in this podcast are for educational purposes only, and not as legal advice. Views expressed by all speakers are their own and do not reflect those of their respective organizations. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit naveenagarwalphd.substack.com/subscribe