Case Study: What AI/ML Device Recalls Reveal About Emerging Risks

Note: this is an updated version of the original article first published on Sep 8. It has been corrected for an error in the number of AI/ML enabled medical devices cleared by the FDA mentioned in the original article.

In this audio brief, we unpack recalls data on AI/ML enabled medical devices to gain insights on emerging vulnerabilities from a risk management point of view.

Here are a few key highlights

* The Landscape: 1,247 FDA cleared AI/ML devices across 155 product codes; 38 recall events identified for a deep dive.

* Recall Severity: Mostly Class II recalls, no Class I recalls.

* Leading Causes:

* Software and algorithm errors (e.g., incorrect dose calculations).

* Data integrity issues (e.g., misfiled or missing images).

* Hardware failures (e.g., loose CT table bolts).

* Labeling & approval lapses (e.g., unapproved software versions).Four-tier, risk-based system; stringent requirements for high-risk systems (including many medical devices); compliance timelines of 1–3 years.

* Trends to Watch:

* High rate of recalls within 12 months of clearance.

* Devices without clinical validation face more, and larger, recalls.

* Public companies account for nearly all recalled units, suggesting market pressures for faster launches without adequate clinical validation.

* Takeaways for stakeholders:

* Manufacturers: Strengthen lifecycle controls, prioritize pre-market validation, enhance post-market vigilance.

* Regulators: Consider time-limited approvals and stronger oversight of high-volume AI devices.

* Clinicians: Validate AI results with clinical judgment—trust but verify.

* Patients: Benefit from innovation but remain vulnerable; safety must remain paramount.

AI in MedTech is transformative but not without risk. The challenge is moving from compliance - driven recall response to active risk mitigation for robust safety and effectiveness.

🎧Listen to the audio brief above for an overview of the AI/ML device recalls, emerging vulnerabilities and trends to watch.

Thanks for reading Let's Talk Risk!. This post is public so feel free to share it.

Note:

This audio brief was prepared using Google NotebookLM, an AI-enabled research tool. Here is the list of resources used in our analysis:

* JAMA: Early Recalls and Clinical Validation Gaps in Artificial Intelligence - Enabled Medical Devices, Research Letter | AI in Health Policy, August 2025.

* AI/ML Recalls Analysis - Unpublished report, created using ChatGPT.

This summary was created using ChatGPT-5 (September 2025) with expert review. It distills publicly available information on FDA-cleared AI/ML-enabled devices and related recall patterns. While reviewed for accuracy and relevance, it does not constitute legal, regulatory, or medical advice. AI in healthcare is a rapidly evolving area, and details may change after publication.

We encourage listeners to interpret these findings in the context of these constraints.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit naveenagarwalphd.substack.com/subscribe