Skip to main content

Analysis

Why U.S. News Rankings Matter—And Why They Don't

By John Commins  
   August 17, 2018

The magazine's renowned ranking system, now in its 29th year, is ostensibly designed to help patients choose care venues, but much of its importance lives in the eyes of industry insiders.

The U.S. News & World Report annual hospital rankings continue to be an industry-wide must-read for the hospital sector.

The most-read HealthLeaders story this week—and so far this month—was Tuesday's news that the Mayo Clinic held onto its top spot for a third consecutive year. Readers were also interested, of course, in knowing which hospitals climbed the rankings and which slipped down the list.

Related: Mayo Clinic Tops U.S. News Hospital Honor Roll for 3rd Straight Year

Related: Cedars-Sinai Rises in Ranks to U.S. News Top 10 Hospitals

Related: NewYork-Presbyterian Slips in Ranks, Clings to U.S. News Top 10

But what do the rankings really mean? Do they matter? That depends on who you are.

While the rankings are purportedly designed to help consumers pick care venues, one researcher says the annual list is far more important to hospital executives than potential patients.

Melissa Riba, director for Research and Evaluation at University of Michigan's Center for Healthcare Research & Transformation, has been studying the many annual hospital rankings issued from various sources throughout the year.

Riba spoke with HealthLeaders Media about the U.S. News rankings, and how to place it in a proper perspective. The following is a lightly edited transcript.

HLM: Why do these rankings continue to be so popular?

Riba: The short answer is the brand. They're known for their rankings, and that U.S. News & World Report seal. It's identifiable and people associate it with rankings, and not just in healthcare but in education, colleges. They've done a brilliant job positioning and branding themselves.

HLM: The rankings target consumers, but seem to get more attention from within the hospital industry.

Riba: The biggest "wows" actually come from within the industry. I work at the University of Michigan. We are ranked No. 5 on the list and U of M was very proud of it, and immediately circulated the news in our internal newsletters. It was up on the website within milliseconds of being released. It's a point of pride. It's a marketing tool for the industry.

HLM: So then, who is served with these rankings?

Riba: This is absolutely meant to be consumer-facing, but what we've learned from the research is that consumers don't really use these rankings to decide where they're going to get care.

Where does their doctor recommend they go because they have that trusted relationship? Family and friends have recommendations too. But overwhelmingly it's geography. People rarely have the ability to go to another part of the state or the country to get this care. U.S. News acknowledges that. It is not realistic to think consumers are going to go to U of M if you're living in Iowa.

HLM: What is the value of these rankings?

Riba: By and large, it is a marker of quality. U.S. News has a rigorous, systematic way that they develop these rankings. As we've also found, with all the different rankings out there, there are a hundred hospitals who can say "We're No. 1" in some area, and that can be confusing. Questions need to be raised about what they're measuring and indicating as "best," and those vary.  

HLM: It sounds like you're saying that the rankings' biggest value is as a marketing tool.

Riba: I don’t want to sound cynical, but yes and no. There is some method that indicates a level of quality but it is based on that particular ranking system and how they are defining quality, and that is where it gets confusing for consumers.

Most people don't get into metrics. Most people just want to know "Am I going to go in, and am I going to come out, and am I going to feel OK. Are they not going to kill me?"

In general, all of these rankings are an indicator of some kind of quality, but that varies depending upon who's doing the ranking. And it can give consumers some modicum of comfort knowing they're going to a great hospital. But, there are other things that weigh into the decisions of where to go, and there are lots of things that go into making high-quality care.

HLM: Do hospitals game the rankings?

Riba: I don’t know that I'd call it gamesmanship, but sometimes it is a little like teaching to the test. That's across the board. Anytime you get to performance and quality metrics that say "this is what you're going to be graded on," some organizations are going to put their resources toward achieving that.

Hospitals that teach to the test might put more resources and effort into the patient experience, for example. That's great! We want patients to be happy and have the great experience, but some would argue that's not the best indicator of quality.

HLM: Is there much of a difference between No. 1 and No. 50 hospitals?

Riba: It depends on what's being measured. For example, U of M ranks No. 5 on the U.S. News Honor Roll. IBM Watson also does a ranking. They have different goals and they use publicly available claims data so they are very hard data oriented. U of M doesn't appear in their Top 100. Different criteria, different outcomes.

Saying you're No. 1 is great. It makes people feel good. It provides a marker and indicator of quality. But the quality is based on that specific methodology that the ranking organization has set up. It's really like oranges to apples when you're trying to compare.

HLM: What caveats do you apply when looking at this ranking?

Riba: For a consumer, it's one of many pieces of information you want to consider. The fact that a hospital is ranked in these specialties maybe is a good starting point. But you also want to see what your doctor recommends, and talk to friends and family about their outcomes. Maybe not just rely on one ranking, but use multiple sources to see where you want to go for care.

For hospitals that didn't make it in, it helps them understand where they can improve and it is incumbent on them to determine where they want to put their resources to improve.

For hospitals that win, celebrate a win for your organization. But also it’s not the end all, be all.

John Commins is a senior editor at HealthLeaders.


KEY TAKEAWAYS

The rankings are as much a marketing tool as they are a quality metric.

Patients mostly rely on physicians, family, and friends—not rankings—when choosing care venues.


Get the latest on healthcare leadership in your inbox.