The Challenges of Finetuning a Therapy Bot and the “PeerBot Paradox”: a Conversation with Dr. Christopher R. Larrison

We’re joined by Dr. Christopher R. Larrison,  Associate Professor at the University of Illinois at Urbana-Champaign. Dr. Larrison’s research includes studying the use of technology to improve mental health services and social interventions. We discussed the study he co-authored, titled “From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data.” We discussed:  How the study was conducted, its aim, and the gaps in extant studies that it sought to address.   The study’s findings, including how fine-tuning on social media data improves accessibility and coherence.   Whether there’s a material difference between genuine empathy and “synthetic” empathy.   The steep challenges in developing a “peerbot” that approximates the behavior of a traditional peer in a therapeutic setting.    The potential paradox at the core of a GenAI-based peer bot: “the more you train it, the less like a peer it behaves.” Additional research that needs to be done, and other research projects Chris is working on.  As more people engage GenAI chatbots for counseling, and as they become increasingly integrated into the therapeutic "space," we must get a deeper understanding of their behaviors, fine-tune them to improve the quality and "sensitivity" of the interactions, and, perhaps most importantly, ensure that the proper guardrails are in place. Anyone with an interest or stake in these issues should make it a point to listen to this important conversation.