After six years, the 252 hospitals that participated in Premier Inc.'s large pay-for performance program, on which Medicare's current value-based purchasing rules are based, did not have lower 30-day mortality rates compared with 3,363 non-participating hospitals, according to a Harvard-based report.
The study calls into question whether pay-for-performance models that mostly use process measures translate to better outcomes—in this case the reduction of 30-day mortality rates, said Ashish K. Jha, MD, of the Harvard School of Public Health Department of Health Policy and Management, and co-authors.
"These findings are sobering for policymakers who hope to use incentives such as those in Premier HQID to improve patient outcomes, e.g. 30-day mortality," the authors wrote.
Premier Inc., a purchasing and quality alliance with 2,400 hospital members, ran the six-year program, called the Hospital Quality Incentive Demonstration (HQID), under a contract from the Centers for Medicare & Medicaid Services. It was based on the hospitals' performance on 33 process measures, such as whether the hospital gave a heart attack patient a fibrinolytic within 30 minutes of arrival, or gave detailed discharge instructions to a heart failure patient.
Over the six years, CMS paid $60 million in incentive payments to 211 of the 242 participating hospitals.
Jha's report was published in Thursday's New England Journal of Medicine.
In a telephone interview, Jha says the data analysis of the HQID study led him to conclude that "Our expectations of what value-based purchasing is going to produce in terms of improvements in outcomes should be pretty limited."
"No matter how you slice it," he adds, "I think what we see is that (among the 252 hospitals participating), improvements for processes were pretty modest. And, then obviously our study says it didn't have any impact on outcomes."
The study found "little evidence" that participation in the Premier program "was associated with declines in mortality above and beyond those reported for hospitals that participated in public reporting alone, even when we examined care over a period of six years after the program's inception," the authors wrote.
Throughout the duration of the experiment, from 2003 to 2009, public reporting on Medicare's Hospital Compare may have incentivized all hospitals to improve quality performance. So the improvement at all hospitals' may have been, to some extent, a result of that pressure.
Additionally, Jha says, over this time clinicians were realizing much better ways to care for patients with these conditions, a learning process that reduced mortality as well, but was unrelated to incentive payments.
"Furthermore, we found no difference in trends in mortality between conditions for which outcomes were explicitly linked to incentives and conditions for which outcomes were not linked to incentives," the authors wrote.
The conditions for which the process measures were applied were congestive heart failure, acute myocardial infarction, and pneumonia, the same ones in the current value-based purchasing logarithm.
What should policymakers do now? Jha was asked.
"In some ways we have to go back to the drawing board," he says. "I'm a big believer in pay for performance, despite this study. But I think we need to figure out the 'pay' part, and the 'performance part. I actually think 1% is not enough. It should be more substantial."
He was referring to the fact that under value-based purchasing rules set forth by the Affordable Care Act, all hospitals in nation this year will receive only 99% of their Medicare base DRG starting Oct. 1. That 1% goes into a pool that is redistributed to hospitals with the best value-based purchasing scores based on 12 process of care measures (70%) and patient responses to patient experience surveys (30%) during a measurement period that began July 1, 2011.
Hospitals with the lowest scores will not earn any of their 1% back.
Jha also thinks value-based purchasing should be re-worked with more emphasis on outcome measures, and that is starting to happen.
Next year, the value-based purchasing formula will introduce a measure for 30-day mortality, the first outcome measure in the rule.
When the six-year project ended last November, Premier said that participating hospitals had "saved an estimated 8,500 heart attack patients" and administered more than 960,000 additional evidence based measures to 2.7 million patients" cared for at those hospitals.
Asked to respond to the conclusions in the Jha paper, Premier said in a statement last week that the HQID "achieved its goal.
The HQID "was designed to test whether incentives would improve care processes and a limited number of outcomes in hospitals beyond what was possible with public reporting alone. HQID hospitals outperformed matched hospitals in improving quality scores and did so more quickly, achieving an 18.6 percent improvement."
However, Premier acknowledged, "the jury is still out on whether these limited process and outcomes measures are driving favorable results in patient outcomes, such as mortality, which is the focus of the NEJM article. HQID did measure mortality in two areas (heart attack and heart bypass surgery), and results show that outcomes in these areas were comparable to non-participants.
"However, mortality affects very few inpatients, creating limited opportunities for performance improvement from the outset. In fact, subsequent research has shown that a focus on the main drivers of mortality (sepsis, respiratory infections, shock and palliative care) is more effective at reducing a hospital’s total mortality numbers," the Premier statement said.
Premier added that "Although there have been a number of studies on HQID and outcomes measures like mortality, results often vary, with some showing positive results, others not.
"What is important and unquestioned is that HQID created a successful framework for performance improvement. Since the inception of HQID, the science of measurement has advanced and matured, and we have newer, better tools to assess performance on both processes and outcomes. We do know that an HQID-type execution strategy is a good one for driving rapid and sustainable improvements over time.
"That is why we used new measures to drive a higher level of performance in our quality and cost reduction collaborative, QUEST. In fact, we have seen strong results in QUEST in reducing mortality, harm rates and readmissions as well, and in three years’ time, QUEST hospitals have saved nearly 25,000 lives while reducing healthcare spending by nearly $4.5 billion."