'We're Still Learning How to Do This'
Still, the Leapfrog survey is a common point of contention. The AHA's Foster says the survey has yet to be properly validated. A recent Penn State University study illustrates the issue. It found that Leapfrog's current methodology may lead to a lower score for the significant number of hospitals that don't complete the survey.
Binder says the Penn State study was underpowered. And the sour grapes issue persists. Neil Goldfarb, president of the Greater Philadelphia Business Coalition, wrote to the Journal of Hospital Medicine to point out that the researchers' home hospital, Penn State Milton S. Hershey Medical Center, received a C rating in the last two Leapfrog surveys.
Yet, look at the home page of Penn State Milton S. Hershey Medical Center, and you'll see the US News & World Report badge noting its ranking as a top children's hospital for five specialties.
What about Leapfrog? It doesn't rate children's hospitals.
It's a messy world, acknowledges Wachter.
"The fact is that trying to judge a hospital and decide whether you want to go there, or bring your family there, or send your mother there is an extraordinarily hard thing to do," he said. "The way we measure quality is still in its infancy. We're still learning how to do this."