Episode 35 | Why Most AI Productivity Claims Are Dangerously Misleading | Ben Waber

MIT researcher and People Analytics author Ben Waber joins Amy and Meg for one of the most myth-busting conversations about AI, productivity, and what actually drives enterprise value. From a 23-year-old grad student who discovered that billion-dollar companies don't know how their own teams communicate, to the lunch table experiment that changed programmer productivity by 20%, Ben brings 15 years of behavioral data to challenge everything you think you know about how organizations work — and why most AI claims should make you very skeptical. ⏰ TIMESTAMPS: 00:00 What percent of your company is a dumpster fire? 00:11 Introduction to Ben Waber 01:11 From Philly to MIT: How Ben started measuring how humans work 04:35 A paper that made a billion-dollar bank reorganize 06:31 The Japanese minor, a bestselling book, and being recognized on the street 08:42 The Academic Run Playlist: 2,500+ talks and counting 13:07 The big idea: Where is the real value in AI? 14:37 Why AI vendors and economists are both getting it wrong 16:20 The calculation machines story: 20 years to get 20% cheaper 18:06 Amazon's box-packing metric and why "quantitative" doesn't mean "objective" 20:11 Jack Dorsey, Block, and the rude awakening ahead 22:11 Klarna's AI rollback and the nuance problem 24:33 "Spin up 100,000 agents doing nothing" — the meaningless metrics trap 27:17 The three things you need to understand before deploying AI 29:37 Tripwires: Building permission to be wrong 31:22 How do you actually model work? Amy's HRIS thesis 35:48 What we're really good at measuring: what's awful 37:12 From dumpster fires to board-level accountability 38:20 AI is a sugar rush — and profit predicts 1% of your future 39:06 If the cows are limping, it's bad 39:28 The lunch table story: a 20% productivity difference from a $50 decision 44:22 Leadership Corner: Breaking through when a peer team is gatekeeping 51:44 Wrap-up: What we learned from Ben 🔑 KEY INSIGHTS: - Most AI productivity claims are measuring activity, not value — "having a seizure on my keyboard outputs more lines of code" - Companies can't define what performance actually means — and that's the root problem - We can't predict what great looks like, but we're really good at identifying what's awful - The "dumpster fire" reframe: measure what percent of your company is broken and put a dollar value on it - AI adoption is a sugar rush — firing 40% of employees boosts quarterly profit but predicts nothing about the future - Current profit predicts only 1% of future profit — people metrics predict far more - A 20% difference in programmer productivity was driven by which cafeteria door people walked through - The financial industry is starting to use workplace behavioral data in investment decisions 📚 RESOURCES: Ben Waber's book, People Analytics: https://www.amazon.com/People-Analytics-Technology-Transform-Business/dp/0133158314 Ben's HBR piece on LLMs and organizational performance: https://hbr.org/2024/01/is-genais-impact-on-productivity-overblown Patty Azzarello, Move: https://www.amazon.com/Move-Decisive-Strategy-Obstacles-Setbacks/dp/1119348374 Nate B. Jones on Klarna: https://www.youtube.com/@NateBJones 🔗 CONNECT: Ben Waber: https://www.linkedin.com/in/benjaminwaber/ Submit Leadership Questions: [email protected] Instagram: https://www.instagram.com/megandamyshow/ LinkedIn: https://www.linkedin.com/company/the-meg-amy-show #Leadership #AI #AITransformation #PeopleAnalytics #FutureOfWork #MegAndAmyShow