'Perverse Incentives' Perpetuate Use of Disproven Medical Treatments

In what researchers are calling "the largest and most comprehensive study of medical (finding) reversal," they identify 146 medical practices that have been refuted by subsequent studies, but which continue to be used by providers.

Flawed research too often prompts doctors and hospitals to rush in technologies or practices which subsequent studies resoundingly refute. Yet despite solid contradicting evidence, some doctors take years to change their recommended care, and others just don't.

That's the conclusion of researchers who looked at 146 medical practices based on studies published in the New England Journal of Medicine, whose conclusions were reversed by subsequent scientific trials within the next 10 years. The new paper is published in the August issue of the Mayo Clinic Proceedings, and authors wrote that it is "the largest and most comprehensive study of medical (finding) reversal."

See Also: Berwick: Zapping Overtreatment, Costs Takes 'Courage'

"What we found is that when we learn in certain cases that we were wrong, it's much harder to stop doing something that we have been doing, especially when there's money involved, and especially when the finances of the person making the recommendation are tied to the patient going through [with the recommended care]," says Vinay Prasad, MD, an oncologist with the National Cancer Institute and lead author of the report.

"It's a perverse incentive, that's what Don Berwick talks about, and that's what we're identifying."

In one of dozens of examples, the article explains that research attempting to test whether bispectral index (BIS) monitors, designed to calculate whether anesthesia administered to patients undergoing procedures in which they needed to be asleep, was sufficient to eliminate their awareness.

The device received U.S. Food and Drug Administration approval in 1997 based on two trials, one an industry-sponsored study that did not use standard protocols for comparison and the other which wasn't large enough to draw conclusions.

"Nevertheless the monitors' use increased. By July 2007, half of all operating rooms in the United States had a BIS monitor," gaining acceptance "largely through vocal support from prominent advocates and faith that the mechanism of action is sound."

In 2008, a large randomized trial compared the BIS with standard sedation monitoring and "found no benefit for the device on anesthesia awareness."

In an interview, Prasad says that because all too often, poorly designed research studies are reversed, doctors question what care practice evidence they should value and what they should take with a grain of salt.

"Because of these reversals, there's a frustration, that you can't trust a recommendation."

He adds, "All of us in our professional lives should inject a little more reflection in what we're doing, and reappraise regularly on whether what we're doing is based on the most solid evidence. That would go a long way."

Prasad's study was conducted with researchers at Yale University, Northwestern University, Lankenau Medical Center in Philadelphia, George Washington University, University of Maryland Medical Center and the University of Chicago.

They reviewed 2,044 articles published between 2001 and 2010, and selected 1,344 that concerned a medical practice. Of these, 981 examined a new medical practice and 363 tested an established practice. Of these 1,344, 947 had positive findings, but 397 were negative findings.

Of these 1,344, 756 suggested a medical practice should replace an older standard of care, 165 found that a new practice failed to surpass an older practice, 146 found that a current medical practice was actually inferior to a prior standard and the remaining 138 reaffirmed the older practice.

"Looking at all the examples we put together, the unifying theme is that the most common reason for things being found not to work is that they were actually adopted based on poor quality evidence," Prasad says.  

Among some of the examples listed:

• Concerns that vaccinations precipitate flare of multiple sclerosis prompt many doctors to omit this intervention, concerns "largely undermined" by two studies in 2001.
• Worries that oral contraceptives increase lupus flares made doctors reluctant to prescribe them, resulting in a higher rate of unwanted pregnancies and elective abortions among patients with lupus, despite two trials that exonerated the drug as a cause of lupus flares.
• Despite evidence from numerous studies that the benefit of vertebroplasty, a procedure in which doctors inject a kind of cement into the vertebrae, "was contraindicated by two paired articles," the "referrals actually went up after those studies, Prasad says.

"When you're doing something that, psychologically, it's plausible that it could help, and you're getting reimbursed well, together that's really a strong psychological motivation to doing it," despite what the evidence says.

In an accompanying editorial, John Ioannides, MD, of the Stanford Prevention Research Center, called the NCI article impressive and said that "at a minimum, it poses major questions about the validity and clinical utility of a sizeable portion of everyday medical care."

And, he added, Prasad and colleagues "offer some hints about how this dreadful scenario might be aborted. The 146 medical reversals that they have assembled are, in a sense, examples of success stories that can inspire the astute clinician and clinical investigator to challenge the status quo and realize that doing less is more."