Skip to main content

It's Impossible to Know What $1B in Federal Quality Spending Buys

 |  By cclark@healthleadersmedia.com  
   February 20, 2014

A pair of quality initiatives say they are preventing harm and saving lives, but their data is lacking, two prominent patient safety and health policy experts contend.

From the jubilant tone of two recent announcements, one might be relieved to know that hospital safety and quality have improved by leaps and bounds in the last few years.

First, some 1,500 hospitals participating in the federally funded Hospital Engagement Network (HEN) say they have prevented hospital-associated harm in 69,000 patients in the program's first two years, with cost savings of $201 million.

This single HEN, one of 26 in the federal Partnership for Patients initiative, has reduced or prevented readmissions, infections, blood clots, and pressure ulcers, says Rich Umbdenstock, president of the American Hospital Association whose trust operates this HEN with a share of $1 billion in federal funds authorized by the Affordable Care Act. Their "hard work is really paying off," he said.

Second, Premier Inc.'s Quest Collaborative of 352 hospitals boasted last week that its programs prevented an expected 136,375 deaths over five years and saved $11.65 billion. Premier officials said this 38% reduction was based on incident rates at the baseline, which "outpaced the national average."

But both claims have two prominent leaders in hospital quality shaking their heads.

"These statements about how many lives and costs are saved are encouraging because it's what policy makers and providers all want," says Peter Pronovost, MD, senior vice president for patient safety and quality at Johns Hopkins Hospital. "But the underlying designs of these programs all have problems that make such statements about savings extremely challenging."

First, he says, many of the measures used in both the HEN project and Premier's Quest, "are of low validity, with data varying among sites and limited quality control." And, he points out, there's no peer review of the results, and "no public reporting of how accurate they are. There's a reason why peer review is important; it assures us that the science passes the smell test."

Second, the projects lack control groups. "Hospitals all over the country are working on this stuff," with dozens of other projects, Pronovost says, "so that makes it difficult to attribute the results to one particular intervention."

For example, while the Quest program receives no federal funds, a large number of its participating hospitals are also in Premier's HEN, which does get federal money.

In planning the design of the HEN programs, which target 10 types of hospital harms, officials with the Centers for Medicare & Medicaid Services "didn't standardize the collection of data, Pronovost says. "It was like 'let 1,000 flowers bloom.' I recall pushing back on that, saying, 'if you do that you'll never be able to say how big of an impact these really had.' "

Ashish Jha, MD, professor of health policy and management at Harvard School of Public Health, quipped, "I'm fine with letting 1,000 flowers bloom. But we'd love to know which flowers actually bloomed, and which ones didn't."

A huge amount of this is taxpayer money, after all.

About Quest, Jha says "the problem is it's really hard to know whether it improved care and reduced costs. We have no idea. Because we don't have good controls. They say they're compared to matched controls, but it would be nice to see how those were picked, and how they looked at data over time. Those things can make a huge difference."

"And until there is peer review and careful evaluation, it's very hard to know."

Premier's chief medical officer, Richard Bankowitz, MD, says that Premier's method is to

"rigorously define and consistently collect measures from all 350 hospitals that are members of the collaborative, and then to objectively analyze the data. We understand there will always be skeptics, but the results are what they are.

"We have researched this and learned that consistent, transparent measurement; sharing of best practices across the cohort; and helping hospitals to effectively execute, all combine to create a sense of urgency and to facilitate improvement. We are continuing to undertake research to better understand which particular interventions work best, and under what circumstances."

But Premier officials acknowledge that many of the hospitals in its Quest program also participate in HENs other than that run by Premier, and in other programs. So how can improvement definitively be attributed to the HEN initiative or to Quest, or to anything at all?

Keep in mind that apart from any of these federal or private quality improvement initiatives, there's the federal sledgehammer of financial penalties for higher rates of readmissions and negative payment adjustments for higher 30-day mortality rates, low patient experience scores, and hospital-acquired conditions.

Might those incentives be prompting improvements over baseline rates all by themselves?

We just don't know. It might be that some initiatives at one hospital were so successful, they masked other initiatives at that same hospital that actually had a negative impact. Like taking every drug in the medicine cabinet to get over a cold; one never knows which pill did the trick, if any. And some drugs might have exacerbated symptoms.

"The Joint Commission, professional societies like the American College of Surgeons, and our bloodstream infection work has been implemented all over the country," all [these groups] have projects to reduce readmissions and reduce adverse events and costs," and many are working in the same hospitals, Pronovost says.

Like I said, much of this effort is funded by taxpayers. In all, some $500 million in federal funds is funding organizations in 102 communities through meals, transportation and home health agencies to help hospitals improve care after discharge, and reduce mortality and readmissions.

The Centers for Disease Control and Prevention has programs to reduce hospital acquired infections such as C. diff and MRSA, and other projects are run from the Agency for Healthcare Research and Quality. Numerous private endowments such as the Robert Wood Johnson Foundation work with hospitals as well. And don't forget the Quality Improvement Organizations, which work under CMS contracts with hospitals around the country too. And there are many others.

Some 25 other HENs are working with state hospital associations and other networks across the country using different strategies as well. And some hospitals might work with multiple HENs.

Don Goldmann, MD, chief medical and scientific officer for the Institute of Healthcare Improvement, agrees that there are problems with some of these programs, but says a lot of good will come from them eventually.

"The way some of these projects were set up, they weren't really designed to collect data in a standardized way, and they weren't set up for people to learn in real time. But I suspect that at the end of the day, they will find certain core practices that are really important, and there'll be a lot of emphasis on adapting them in a flexible way," so they work in multiple hospital settings.

"A lot of people wanted to tackle these tough problems at once, and that was a good thing…And at the end of the day we will have some really great lessons people can run with," Goldmann told me.

I asked Pronovost if it might all be worth it if at the very least, all these programs forced hospital leadership and their trustees to take quality improvement more seriously. Yes and no, he says.

One risk is that "it disengages physicians who don't believe there's science behind them, that this is just marketing. The early quality improvement movement had all these hyperbolic 'we are filling the football stadium with all the people whose lives we saved' statements, and that turned physicians away," Pronovost says.

"All of us want to help, not harm patients, so there's an amazingly strong bias that we want the data to get better. We all want the story to be true.

The problem is at the end of the day, if it's not better, [patients] are still dying, and yet we all clap our hands and say look, what a good job we've done."

Is it a case of the emperor having no clothes?

"The emperor may be wearing clothes," Pronovost says, "but you certainly can't say he's wearing a gold robe."

Pages

Tagged Under:


Get the latest on healthcare leadership in your inbox.