EP116: Why AI struggles with empathy and interruptions

The ICASSP 2026 HumDial Challenge paper introduces a standardized benchmark for evaluating human-like spoken dialogue systems in the era of advanced Audio-LLMs. While current models excel at task completion, measuring their ability to replicate the subtle nuances of natural human communication requires assessing deep emotional resonance and complex turn-taking. To address this gap, the authors created a sizable dataset using a hybrid approach of LLM-generated scripts performed by professional human actors to preserve authentic conversational dynamics.The challenge evaluates systems across two core dimensions:Track I: Emotional Intelligence, which tests a model's ability to track emotional trajectories over multiple turns, reason about the underlying causes of a user's emotions, and generate empathetic responses.Track II: Full-Duplex Interaction, which assesses real-time decision-making capabilities, specifically focusing on how well a system can handle user interruptions and reject non-instructional background noise while simultaneously listening and speaking.Key findings from the challenge submissions showed that while top systems are highly capable of analyzing emotional logic and reasoning, generating truly empathetic vocal and textual responses remains a significant difficulty. Furthermore, in full-duplex interactions, maintaining silence and distinguishing valid user turns from ambient background noise was identified as the primary hurdle for current systems.