You Bought the AI. So Why Aren't You Using It?

“You buy the hiring platform, and then you don’t have these AI workflows enabled... here you are three, four years later. You’re still not able to do that.”— Nicole MundyEpisode OverviewIn this episode I’m joined by Nicole Mundy, Senior Research Analyst at Talentech Labs — a research and advisory firm that helps its enterprise clients evaluate and procure hiring technology systems.Nicole brings a valuable perspective to the table because she sees the dynamic between vendor and buyer up close and personal. When combined with my experience in this same realm from the science side- our discussion shines light on the reality of what is happening in AI tech adoption for TA.Topics Discussed & Key Insights1. Companies are buying AI hiring tools at scale — and then leaving them switched off.Among the world’s top enterprise organizations, Nicole estimates a surprisingly small amount are actually using AI to automatically assess active applicants.Why?Approvals never come - Companies buy the platform intending to enable the AI, “once the right approvals are in place- but years often pass without any change.Pilots underperform - Big companies test these tools and often conclude they can’t really use them, or they just didn’t work.Lack of solid ROI evidence - Despite the best intentions- most companies do not do the follow up work needed to demonstrate the impact of these tools on the bottom line. Legal ambiguity freezes decisions - With regulations constantly in flux, risk management often takes priority over business needs.2. Validation is misunderstood and absent.What vendors without I/O science guidance call validation isn’t what legal compliance actually requires.* Vendors are quick to speak about the validity of their solution and talk endlessly about validating their AI models — running statistical checks that the model predicts consistently and de-biasing its outputs across groups. This is purely empirical work.* But that’s IT-style validation. It confirms the system runs as built; it says nothing about whether the tool is fair or job-related* Validation for legal compliance, and sound science, demands a blend of rational and empirical work to document the job-relatedness of any tool used to make employment decisions3. “Skills” are everywhere, and nowhere.Skills-based hiring is the headline everyone wants. The problem is what counts as a skill.* Most platforms apply the “skill” label with no objective framework to define it. A skill ends up being little more than a tag like “Excel” or “communication”* The definitions behind these labels are usually poorly organized and loosely constructed. * There is no connection between the skills a platform claims to measure and any outcome on the job. Without that link, there is nothing for a buyer to trust or defend.* The companies doing skills-based hiring well are not buying one vendor and flipping a switch. They run multi-year programs: define the skills objectively, inventory what the organization has against what it needs, curate tools carefully on the front end, and collect assessment data at multiple points.4. Cheating is a zero-sum game, so let’s change the rulesAI-assisted candidate fraud, from AI completing assessments to coaching candidates through interviews, is driving enterprises toward more dynamic evaluation that is harder to game. But chasing detection is largely a losing battle.* Trying to catch and block AI use is whack-a-mole. It’s a zero-sum game, and it’s frustrating, because the tools keep getting better and the detection never really gets ahead* There’s a more useful way to think about it. People are going to use AI on the job, so why not let them use it in the application process in a controlled way? The question stops being “did they use AI” and becomes “how well do they use it”* Most large organizations are still just trying to get visibility on how much cheating is happening and what it looks like, which says how early everyone is on this.* Across the approaches Nicole sees, one thing tends to hold up whether a process is locked down or fully AI-assisted: competency-based follow-up questions that make candidates explain their own reasoning in their own words.Final TakeawayEnterprise isn’t slow on AI in hiring because it doesn’t understand the technology. It’s slow because the tools are bought on vendor claims that were never reviewed against real science, and the danger only becomes clear once the tool is in play. The companies getting it right aren’t chasing tools. They’re building programs, science first. Held to that bar, many of the tools on the market today wouldn’t survive the review, and the ones that would wouldn’t be sitting switched off. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit charleshandler.substack.com