To truly evaluate quality, hospitals would do better to look at avoidable events such as bloodstream infections, which kill 31,000 people a year in the U.S., rather than at mortality rates, a prominent safety expert wrote in a new report.
That's because all too often, patients arrive at the hospital too sick to be saved by the time they are admitted, said Peter Pronovost, MD, professor of anesthesiology and critical care medicine at the Johns Hopkins University School of Medicine.
Mortality could be one factor considered in rating some divisions of quality care as good or bad, he says, but it should not be the only outcome measure.
Pronovost authored the paper "Using hospital mortality rates to judge hospital performance: A bad idea that just won't go away" in the British Medical Journal along with Richard Lilford, professor of clinical epidemiology at the University of Birmingham in England.
When statistical analyses compare hospitals, they usually use a risk adjustment formula. But that doesn't work, the authors said, for two reasons.
"First, risk adjustment can only adjust for factors that can be identified and measured accurately . . . Secondly, risk adjustment can exaggerate the very bias that it is intended to reduce," they wrote.
"For example, if diabetes is a more powerful prognostic factor in Glasgow than in Four Oaks, then adjusting for the average effect of diabetes will deflate expected diabetic deaths in Glasgow and inflate them in Four Oaks....This effect could tilt the playing field against Glasgow."
A key point the authors make is that there is little correlation between how well a hospital performs on one standard of safe and effective care and how well it does on another. "While commercial organisations such as Enron fail corporately, hospitals are more likely to fail on specifics–pathology in Liverpool; paediatric cardiac surgery in Bristol; radiation therapy in Missouri."
Mortality rates for the hospital overall "are silent about where any problem might lie. This combination of unfairness and non-specificity is a toxic mix, inducing what has been called ‘institutional stigma' "
Pronovost and Lilford suggested that when they know they are being graded, officials may resort to "gaming the system – for example, by upgrading risk assessments. Furthermore, a focus on hospital mortality may lead to overly aggressive care, which is inhumane and drives up costs."
Instead of mortality rates, the authors propose these adjustments to better reflect hospital quality:
- Mortality rates associated with high risk procedures that are heavily dependent on technical skill, such as intrauterine transfusion and heart surgery will detect poor practice quickly.
- Non-mortality outcomes, such as rates of hospital acquired bloodstream infections, are "heavily influenced by quality of care.
- Greater use of monitoring of clinical processes known to affect quality of care. "For example, if (an) audit of clinical processes shows that anticoagulants are not being given before hip replacement surgery, then it is clear where the hospital should direct its attention."
In an accompanying editorial, Nick Black, professor of health services research at the London School of Hygiene and Tropical Medicine, agreed that mortality should not be so heavily weighted as a measure of quality.
"Hospitals have taken on the role of providing a place for people to die," he said. "This makes it perverse to use a hospital's mortality statistics to judge its quality of care, given that deaths are often an expected and accepted outcome."
And, he said, mortality in a hospital "is exacerbated by geographical variation in the proportion of deaths that occur in a hospital, (40-65%) which reflect not only the availability of alternative forms of end of life care, such as hospices and community palliative services, but also cultural, religious, and socioeconomic characteristics of the local population."