Responsible AI in Assessment Development

In this episode, Michael Roscoe, PhD, PA-C, president of the PA Education Association, is joined by Robert Furter, PhD, MBA, chief data officer and principal psychometrician at PAEA, along with Henry Makinde, PhD, for a conversation on the responsible use of artificial intelligence in assessment communications and item development. The discussion explores how PAEA Assessment is incorporating AI into the early stages of item drafting through a proprietary, AI-supported process, while maintaining the same rigorous peer review and validation standards already in place. The guests emphasize that AI-generated content serves only as a starting point, with human oversight remaining central to quality control and decision-making. They also examine common pitfalls organizations face when adopting AI, stressing the importance of clearly defining the problem before implementing new technology. Throughout the conversation, they highlight the need for structured oversight, including rubrics, checkpoints, documentation, and clear guardrails to support responsible use. The group also discusses the differences between open and closed AI systems, including PAEA’s use of OpenAI Enterprise to support data governance and security.    This episode is sponsored by Body Interact. Explore Body Interact’s new PA Virtual Simulation Catalog and see how it supports your curriculum: https://bodyinteract.com/physician-assistant.   All Things PA Education is produced by Association Briefings.