From Telemetry to Empathy: Measuring AI in Your Teams

The Opening ContextThe conversation begins with a critical look at how engineering organizations measure success in the era of AI. Lauren Peate CEO of Multitudes joins Deejay to unpack their recent mixed methods research on the real world impact of agentic coding on software delivery. The duo quickly establishes that the headline grabbing promises of AI often clash with the lived realities of developers. Lauren emphasizes that relying solely on telemetry data can paint a misleading picture necessitating a blend of quantitative metrics and qualitative interviews to uncover what is actually happening within engineering teams.The Technical CoreA significant pivot occurs when Lauren reveals one of the most surprising findings of the study: a 19.6 percent increase in out of hours commits among developers using AI tools. While many leaders assumed this spike was driven by the sheer joy of using new technology the qualitative data told a different story. Developers are struggling to balance existing delivery pressures with the steep learning curve of rapidly evolving AI models. The heart of the episode explores how organizations can mitigate this burnout. Lauren advocates for structured peer to peer learning noting that engineers trust AI demos from their codebase peers far more than top down mandates or static playbooks. Deejay parallels this with the military concept of commanders intent arguing that leaders must clearly communicate the why behind AI adoption rather than just dictating the how.Philosophical and Human ImplicationsThe discussion deepens as they examine the psychological toll of AI mandates and the shifting dynamics of psychological safety. They lament the disconnect between senior leadership who often view AI as a sheer productivity multiplier and the individual contributors on the ground who feel their job security is threatened. Lauren points out that unaddressed fears stifle adoption making it crucial for leaders to be authentic about economic realities and organizational goals. Deejay highlights the importance of creating spaces for developers to voice their anxieties and share their AI failures transforming skepticism into collective problem solving.Future OutlookLooking ahead the conversation addresses the precarious future of the junior developer pipeline. Lauren shares upcoming research indicating that while senior leaders rarely mention the impact of AI on entry level talent individual contributors are deeply worried about who will mentor the next generation. Deejay voices a dystopian concern: if AI abstracts away the foundational coding work we risk building critical societal infrastructure on layers of digital cruft that no human truly understands. Ultimately they conclude that the successful integration of AI relies not on the tools themselves but on preserving the human elements of mentorship intentional leadership and community learning.