We need to measure the hell out of healthcare to help us compare one organization or system with others. I believe measuring quality helps healthcare systems improve. But I also believe that we can measure healthcare quality a lot better than we do.
If you really think about it, the way we measure hospital quality of care is pretty darn primitive. That's what I've concluded after a few days contemplating today's methods and practice of measurement.
The way we think we know how good we are at providing high value care is really flawed.
Please don't get me wrong. We need to measure the hell out of healthcare to help us compare one organization or system with others. Yes, it can be painful and stunning to realize competing hospitals have figured out how to do things better. I believe measuring quality helps healthcare systems improve.
But we have a ton of work to do to make our evaluations meaningful to leaders, to providers, to patients, and to the public. With the system as it is, too many organizations just use this imperfection as grounds for inattention.
Here's Part 1 of my list.
1. There are too many rating systems.
There are too many types, methods, and brands of measurement systems. They're confusing the public and the providers who get scored. Some rating systems have evolved to a masterful marketing strategy that hospitals pay dearly for, and fail to explain to patients what it all means. Exactly what gets measured is often complicated and/or opaque:
- Truven Health Analytics' Top 100 Hospitals
- The Joint Commission's Top Performers
- U.S. News & World Report'sBest Hospitals
- Leapfrog Group's Safety Score
- Leapfrog Group's Survey
- Healthgrades' America's Best Hospitals
- Consumer Reports' Hospital Safety Ratings
- Hospital Compare
I have my favorites, but then I know what I'm looking for based on my likelihood of needing acute care. Hey, you know what, I changed my mind: The more the merrier. But these rating systems should make it much more clear what it is that they are measuring.
2. Ratings are often inaccurate.
Administrative data based on diagnostic codes for billing purposes, often called "claims" data, is what many measures within Medicare's reporting system are based on. But it's awfully inaccurate.
In their article in this week's the Journal of the American Medical Association, Robert Panzer, MD, chief quality officer and associate vice president for the University of Rochester Medical Center in Rochester, NY, and colleagues noted that when auditors compared objective clinical findings in the record with the billing code data, "21% of those positive for the claims-based Patient Safety Indicator 'postoperative pulmonary embolus or deep venous thrombosis' were miscoded."
They wrote, "These flaws are expected because claims data are primarily intended to communicate sufficient information for fair payment, not to accurately reflect the nuances of the clinical condition of the patient."
3. The ratings are based on old data.
On the Centers for Medicare & Medicaid Services' Hospital Compare website and many of the above listed rating sites which rely on CMS data, performance periods in some cases began as long as five years ago and ended as long as three or two years ago.
That delay allows hospital officials and front line staff to make excuses for their poor scores, arguing that they're doing it much better now. Of course no one knows for sure, because today's data won't be out for another three or four years.
4. There's too much in the middle.
One thing bugs me about Hospital Compare and a few other rating systems that most folks don't realize. Only 2% or 5% of organizations are "better" or "worse" with everyone else being in the middle, okay. If 90% to 96% percent of all hospitals are the same, why bother measuring?
5. Only some diseases are evaluated.
Only a few types of patient conditions are evaluated for core process measures, such as whether patients got an aspirin or an antibiotic when they should or whether they were readmitted. Inpatients do get care for other diseases or procedures that bring them to the hospital besides heart failure, pneumonia, heart attack, joint replacement and certain types of surgeries.
6. Emphasis for most measures from process to outcome is still evolving.
Outcomes, including mortality, infection rates and readmissions, are now being measured with federal penalties for poor outcomes. Reporting on functional outcomes, meaning whether patients can function as they might reasonably be expected after their procedures, is coming. But it's not here yet. "Despite discussion of the challenges of a rapidly expanding number of quality measures, much of health care remains poorly measured or unmeasured," Panzer wrote.
I could go on and on. And I will in next week's column, where I'll share the second half of my list. In the meantime, if you have suggestions, leave them in the comments section below or email me directly.