LTR 84: Think of human factors for AI as levels of automation

“Many of the AI/ML devices are going through De Novo and (in these) human factors considerations are specifically being called out as special controls.”In this Let’s Talk Risk! conversation, Shannon Hoste highlights some of the new concerns emerging for human factor considerations for AI/ML devices in MedTech. She explains how we could consider the level of automation as we think about new hazards related to the user interface. For example, in a clinical decision support system, having appropriate levels of trust with the physician is important. Shannon emphasizes the need for transparency and explainability of AI models, and highlights the potential for over-reliance as a reasonably foreseeably misuse by physicians. FDA is increasingly looking for human factors studies as part of De Novo submissions of novel AI/ML applications. Listen to this Let’s Talk Risk! conversation with Shannon Hoste, which also includes an open discussion with a live audience. Jump to a section of interest using these timestamps. 00:01:35 Introduction00:03:00 Understanding human factors in AI as levels of automation00:07:30 Two useful guidance documents for human factors in AI00:09:23 Current state of AI/ML devices in MedTech00:11:07 FDA questions/concerns related to human factors in AI/ML00:14:40 Handling reasonably foreseeable misuse for AI/ML devices 00:16:10 Open discussion and audience Q&A00:31:20 Closing comments and key takeawaysIf you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.Suggested links:FDA: Clinical Decision Support Software, Final Guidance issued Sep 2022. FDA: Technical Considerations for Medical Devices with Physiologic Closed-Loop Control Technology, Final Guidance issued Sep 2023. LTR: Treat human factors as a driver of customer satisfaction.About Shannon HosteShannon Hoste is currently the Chief Scientific Offer at Pathway for Patient Health, where she is actively creating and sharing knowledge to drive innovation in MedTech. In a career spanning over 25 years, Shannon has served in a variety of roles, including a Human Factors Team Lead at the US FDA. Her extensive experience includes engineering and product development roles, as well as continuing to consult in human factors engineering through Kymanox. She holds a Bachelor’s degree in Mechanical Engineering, and Graduate degrees in Management and Cognitive Systems Engineering, and is currently working on a Graduate Certificate in Epidemiology. Let’s Talk Risk! with Dr. Naveen Agarwal is a weekly live audio event on LinkedIn, where we talk about risk management related topics in a casual, informal way. Join us at 11:00 am EST every Friday on LinkedIn.DisclaimerInformation and insights presented in this podcast are for educational purposes only. Views expressed by all speakers are their own and do not reflect those of their respective organizations. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit naveenagarwalphd.substack.com/subscribe