Intelligence Unit Special Reports Special Events Subscribe Sponsored Departments Follow Us

Twitter Facebook LinkedIn RSS

What's Wrong With Healthcare Quality Measures? Part I



We need to measure the hell out of healthcare to help us compare one organization or system with others. I believe measuring quality helps healthcare systems improve. But I also believe that we can measure healthcare quality a lot better than we do.



6 comments on "What's Wrong With Healthcare Quality Measures? Part I"
Jim Reinertsen (11/19/2013 at 11:04 PM)

Fifteen years ago, Mosser, MacDonald and Solberg wrote a brilliant paper describing 3 very different purposes of measurement in healthcare: research, accountability, and improvement. (Jt Comm J Qual Improv. 1997 Mar;23(3):135-47.) Cheryl, your post focuses on what Mosser et al. would term measurement for accountability[INVALID]i.e. comparison to peers, or to a standard, in which reports are typically in the form of decile rankings, or "above or below the median." The problem with this type of measurement is that for all those who look bad, most of the energy in response goes into self defense, rather than improvement. When we look bad on the comparisons, we say "the data are wrong." And the fact is, we're usually right. The data are always wrong in one way or another (bad risk adjustment, errors in the claims database, etc.) The classic recent example of this is the NYC hospital leaders who explained their poor HCAPS scores by saying "Our patients are whinier." Give me a break. And there's another problem. The top two deciles for most comparative measures of process quality e.g. many Value-Based Purchasing measures are currently 99 or 100%. The bottom deciles are 92-95%. There is absolutely no evidence that there is any clinical outcome difference between 95 and 100% i.e. between the worst and the best deciles in these measures. It's largely a matter of better coding and documentation, not better clinical care. My view is that healthcare leaders waste far too much energy "How do we compare to others?" i.e what I call the "Healthcare BCS Rankings." Patients would be better served if we all focused on measurements that asked two questions: 1) Are we getting better? and 2)What's the gap between our current performance and the theoretical ideal? In Mosser's lexicon, that's called "Measurement for Improvement."
Naomi (11/15/2013 at 1:46 PM)

As a Kaiser patient, I have two complaints: [INVALID] Kaiser uses sampling to game the system and say it has too few admissions to report results. [INVALID] Or perhaps it's because Kaiser doesn't report on its Medicare Advantage patients [INVALID] the vast, vast majority [INVALID] skewing the data terribly. On that same point, without including MA, Kaiser's reporting is meaningless and yet the government uses the data to rate KP for Medicare "stars." This isn't trivial. KP covers 7 million people; I don't know the percentage >65.
Jacob Kuriyan (11/15/2013 at 9:57 AM)

Obama is ready to diss the healthcare IT vendors and Ms. Clark is challenging the likes of JCAHO - the "Emperors" that rule healthcare. These are the unintended consequences of bringing free market forces into healthcare. The first question I have is why "healthcare quality" is equated to "hospital quality"? Sure, that's part of it, but don't we need to get away from measuring "procedures" to measuring "health"? The second question is- why focus on hospital treatments? Why not discuss treatment protocols or even medications - especially since drug costs are much higher than hospital costs in many cases. As Dr. Berwick points out in his seminal paper in Health Affairs - we must measure healthcare performance using three measures of Triple Aim - patient experience, improvements in health of populations served and cost. So these rating services are of limited value. I look at them as a valuable way to spot the bad performers - the hospitals you may want to avoid. Of course, most people select a specialist - and inherit the hospital based on the attending privileges of the specialist. Usually good doctors practice only in good hospitals - no matter what the rating of the hospital may be! I cannot wait to read her next column!
Tony Heath (11/15/2013 at 9:52 AM)

I applaud you Cheryl for raising these issues. Having worked in healthcare for 20 years I have almost given up with the abysmal quality of healthcare data. I wonder if many others have also said "Oh well" about this problem. So let's join up and talk about this. It's a start.
E (11/14/2013 at 3:57 PM)

I look forward to hearing what you propose for alternative measures.
Loran Cook (11/14/2013 at 3:48 PM)

For all of the flaws obvious with quality measures at their face value, the problems are even more systemic from a data perspective. In my former role as Database Project Lead at Billian's HealthDATA, I had the privilege of importing the new quality data into our system every quarter. Most quarters, at least one new measure was introduced, many times duplicated with differing textual fields and poor documentation to explain differences between measures from quarter to quarter. One of the benefits of using our tool is that we researched these measures every quarter and clarify them if necessary, so within our tool you can see how scores change over time. This is valuable as more of a benchmarking tool than a live dashboard, but will prove to be more useful as Value Based Purchasing adds industry performance into the algorithm for rewards/penalties.