LTR 156: Using AI as a Thinking Partner in Product Development Without Losing Control

Summary“AI can be a powerful helper, but it should not be the decision-maker.”In this Let’s Talk Risk! conversation, host Naveen Agarwal speaks with David Grilli about how MedTech teams can use AI responsibly in product development without losing control of risk, quality, or accountability.David brings experience across medical device risk management, system safety, reliability engineering, and regulated product development. He shares practical examples of where AI may help—such as requirements development, software troubleshooting, and early discovery—while emphasizing the need for clear boundaries, human judgment, validation, and leadership oversight.The conversation also explores how teams should define the intended use of AI applications, how to communicate AI proposals to leadership, and why AI should be treated as a cognitive partner rather than an accountable decision-maker.Listen to the full 30-minute podcast or jump to a section of interest listed below. Chapters00:00 – Introduction03:20 – Where AI is showing up in product development05:00 – Using AI for requirements without giving up decision authority08:45 – Treating AI as a helper, not an accountable decision-maker12:00 – AI for debugging, code review, and software troubleshooting16:00 – Data quality, training inputs, and weak answers17:30 – AI in early discovery and idea-stage development18:40 – Making the leadership case for AI adoption20:55 – Intended use, validation, and QMS expectations22:35 – Where human review belongs in AI-enabled workflows25:40 – David’s key takeawaysIf you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.Suggested links:LTR: Building Trustworthy AI and MedTech Readiness.LTR: What AI/ML Device Recalls Reveal About Emerging Risks.LTR: LTR Risk Coach - AI-Powered Decision Support Tool.Key Takeaways* AI should support judgment, not replace it. David emphasizes that AI can help generate options, identify gaps, and accelerate work, but humans remain accountable for decisions.* Start with high-impact, low-risk use cases. Requirements drafting, debugging, and discovery support may be practical starting points when clear review controls are in place.* Define the intended use before deploying AI. Teams should be clear about what AI is allowed to do, what it is not allowed to do, and where human review is required.* Leadership needs more than enthusiasm. A credible AI proposal should include workflow, expected benefit, risks, mitigations, validation expectations, and decision criteria.* AI adoption is a governance challenge. As AI enters product development, teams must build review points, instructions, and accountability into the development process.KeywordsAI in MedTech, artificial intelligence, product development, medical device risk management, ISO 14971, system safety, software validation, design controls, requirements development, human judgment, AI governance, QMS, risk-based decision-making, leadership, product innovationAbout David GrilliDavid Grilli is a senior engineering consultant supporting teams building safety-critical, regulated, and operationally complex systems. His background spans more than 15 years across medical device risk management, aviation reliability engineering, and system safety, with experience in ISO 14971, MIL-STD-882, design reviews, hazard analysis, failure-mode analysis, technical justification, and audit readiness.David is also the founder of North Star Haptics, where he applies systems risk, reliability awareness, and human-interface thinking to early-stage tactile technology development. His prior experience includes senior risk management engineering at Abbott and reliability and system safety work at Honeywell.Let’s Talk Risk! with Dr. Naveen Agarwal is a bi-weekly live audio event on LinkedIn, where we talk about risk management related topics in a casual, informal way. Join us at 11:00 am EST every other Friday on LinkedIn.DisclaimerInformation and insights presented in this podcast are for educational purposes only, and not as legal advice. Views expressed by all speakers are their own and do not reflect those of their respective organizations.Parts of this article were created using AI-generated content, which was subsequently reviewed, edited, and fact-checked by the author to ensure accuracy and alignment with our standards. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit naveenagarwalphd.substack.com/subscribe