For his part, Kern says the methodology will have to be greatly improved for the CMS system to be considered a viable measure of quality and outcomes, upon which patients and doctors should make treatment choices.
"I don't pretend to be an expert on the methodology, but the truth is that it's an issue," he says.
Under the current CMS methodology, rural hospitals, and for that matter, specialty hospitals, can benefit in that they don't have the volume to generate the results that would drive them statistically off the five-star list.
They also benefit by being able to be highly selective of the patients they do treat, he says.
"They can be pretty selective and for high-risk patients, they refer elsewhere," Kern says. "Larger academic medical centers and general community hospitals take all comers and manage as best they can."
3. Patients' Perceptions Aren't a Proxy for Quality
In the Quantros study, CMS one-star hospitals made up a large percentage of the top 10%.
More telling, Mazza says, the Quantros study found no correlation at all between CMS star ratings and clinical outcomes, even when it broke down the data to look separately at mortality and complication rates.
And there was no correlation with an organization's CMS star rating except in patient safety.
"CMS is essentially saying you can use patient experience scores as a proxy for outcomes and clinical quality, but you really can't," says Mazza.
"If you were to use the star rating system as a proxy for this, not only might you not wind up at a high-quality institution, you might end up at substandard institution. Experience is in some respects a measure of quality but not a measure of health outcomes."
Ratings System Should be Refined
Kern is hopeful that CMS will, over time, broaden the data that generate the star ratings and improve upon it. Even in its current format, he says, Sentara pays attention to it and is working toward improving its hospitals that scored relatively poorly on patient experience.
"We give pats on the back to high performers and try to improve the ones that could do better. At the same time, it's not just about getting a score."
Philip Betbeze is the senior leadership editor at HealthLeaders.