LTR 79: Tips for SaMD development and success with FDA

“Overconfidence in the performance of AI models in the development phase is a common problem.”In this Let’s Talk Risk! conversation, David Giese highlights the growing need for a rigorous, disciplined development process with the goal of meeting regulatory requirements for marketing authorization. He shares how AI developers are often over-confident in the performance of their models, but disappointed when these models are subjected to rigorous scrutiny during the regulatory review. Software as a Medical Device (SaMD), including AI/ML devices, are growing exponentially in MedTech. Rapidly evolving technology offers an opportunity to develop innovative products, but it also presents new risks. Security and Cybersecurity concerns are on the rise, as well as, concerns about transparency and explainability of AI/ML models. David points to new guidance documents from the FDA and emerging standards. As an example, software developers should consider using IEC 81001-5-1 - Health software and health IT system safety, effectiveness and security - along with IEC 62304 - Medical device software - Software life cycle processes. Listen to this Let’s Talk Risk! conversation with David Giese, which also includes an open discussion with a live audience. Jump to a section of interest using these timestamps. 00:01:30 Introduction00:02:30 Current state of AI/ML applications in MedTech00:04:05 Different types of AI models - adaptive vs. generative AI00:06:25 Challenges in meeting regulatory expectations for AI in MedTech00:08:18 Use of synthetic data in AI model development00:09:30 Best practices for AI development and regulatory submissions00:12:02 Cybersecurity challenges for SaMD and AI/ML devices00:14:03 New skills risk practitioners need to develop 00:15:09 Open discussion and audience Q&A00:27:20 Closing comments and key takeawaysIf you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.Suggested links:LTR: EU vs. FDA - Aligned but different in approach to safety of AI/ML devicesLTR: A new paradigm for building connected medical devicesFDA: Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions, Issued September 2023. About J. David GieseJ. David Giese is currently President and Partner at Innolitics, a consulting firm specializing in development of new medical software and regulatory submissions to the FDA. His team has helped develop and receive FDA clearance for over 60 medical devices, both SaMD and SiMD, over the last 12 years. His expertise include application development, custom software development, engineering design, project management, user experience design, cybersecurity, software testing and technical writing. He holds Bachelors and Masters degrees in Biomedical Engineering. Let’s Talk Risk! with Dr. Naveen Agarwal is a weekly live audio event on LinkedIn, where we talk about risk management related topics in a casual, informal way. Join us at 11:00 am EST every Friday on LinkedIn.DisclaimerInformation and insights presented in this podcast are for educational purposes only. Views expressed by all speakers are their own and do not reflect those of their respective organizations. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit naveenagarwalphd.substack.com/subscribe