The Early Detection Cancer Test That Doesn’t Detect Early
from Research Translation Podcast Hosted by David Newman
In college our physics teacher, Professor McGinnis, held an annual competition. Using a container made only of glue and toothpicks, we had to keep an egg from cracking when dropped out of the classroom’s first-story window. My team built a geometric lattice, buttressed with layers of internal protection. It was beautiful. For our (mandatory) test run, we dropped it empty, to be sure the container would survive and the egg compartment would hold. The container fell apart, and the compartment was gone. So we redesigned. Again the empty drop failed. But the lattice was so pretty. Perhaps the weight of a settled egg would strengthen the wood, we hoped. And with no time for a loaded test, we prepared for Egg Drop Friday. Suzanne, our dropper, had magic hands. With a quick lift and a soft release, our soccer-ball-like structure seemed to fall slower than gravity should allow. All morning, cheers had erupted at each impact. But when our dazzler struck the ground, the crowd fell silent. Toothpicks everywhere. The egg was not cracked, it was obliterated. No piece of shell was larger than a coin. Professor McGinnis wondered aloud whether our structure had somehow funneled the energy straight into the egg. Then, shaking his head, he asked the question we all knew was coming. “What happened when you did the empty drop?” Last week the New England Journal of Medicine published NHS-Galleri, a massive trial that randomized more than 140,000 people in England to have, or not have, their blood run through a multi-cancer early detection (MCED) test once a year for three years. The trial was funded by the test’s manufacturer, Grail, and Grail employees are among the authors. Let’s start with what matters: The point of cancer screening is to keep people from dying. This paper doesn’t even report how many people died in each group, from cancer or from anything else. This is despite the fact that the trial enrolled 140,000 people, easily enough to have a count of cancer and overall mortality, Instead, they offer us a test run: did the early detection test detect cancer earlier? It is the empty egg drop. It failed. The primary outcome was whether annual testing reduced the number of people diagnosed with late-stage (stage III or IV) cancer. It didn’t, and it wasn’t a near miss. The screened people were found to have 706 late-stage cancer diagnoses and the unscreened people were diagnosed with 688. In other words there were MORE, LATER STAGE, cancers in the tested group. Grail’s press release, however, leads with something different. They tout “20% drop in stage IV cancers”! This was a secondary endpoint, and the result’s confidence interval brushes up against no effect at all. Moreover this reduction, they confess was “offset by a higher rate of stage III cancer diagnosis.” The test didn’t lift anyone out of late-stage disease. Why? The companion paper in Nature Medicine shows us. In years two and three, which is what a real annual screening program looks like, the test missed about 73% of cancers, and roughly half of the 12 deadly types it was designed to catch. Worse yet, about half its positive results were false alarms: 864 healthy, cancer-free people were told they might have cancer. The test’s specificity, around 99.5%, sounds superb. But it isn’t. When you screen millions of healthy people for rare diseases, even 99.5% means half of all positive results are wrong. Scale it to 100 million Americans and roughly 450,000 people a year would be told they might have cancer when they don’t, each launched into scans, scopes, and biopsies. And Grail tuned the test for that specificity. The price was sensitivity, which is why it missed MOST cancers. This isn’t a bug to fix in version 2.0. Small, early tumors shed very little DNA into the blood. Crank up the sensitivity to find them and you drown in false positives and overdiagnosis, the treatment of ‘cancers’ that would never have hurt anyone. Cancer is not one disease marching in a straight line from small and curable to big and deadly. Some kill no matter how early you find them. Some never would have mattered. The mythical sweet spot, where finding it early changes everything, has never been proven to exist. And if it does, we know this: It’s way smaller than the brochures suggest. But fortunately for the test manufacturers, and unfortunately for us, early detection is more than a strategy. It’s an ideology, and for many it’s even a faith. And faith can ‘succeed’ even when the empty drop fails. Which brings us to September 23, exactly one day after the NEJM paper went online. An FDA advisory committee met to consider Grail’s application and voted to recommend approving the test. Someone on that panel should have asked Professor McGinnis’s question: What happened when you did the empty drop? Get full access to Research Translation at researchtranslation.substack.com/subscribe
Embed this episode
No transcript yet.