Beware Flaws in Online Physician Rating Sites

This article appears in the March 2013 issue of Medicine on the Net.

Since they debuted in the mid-2000s, websites that provide user-generated ratings of physicians have become a popular destination for consumers looking for a doctor. But several recent studies suggest the websites raise more questions than answers due to a number of flaws, including questionable rating methods and small sample pools.

While researchers acknowledge that physician-rating websites serve a purpose in providing consumers with useful information, they advise would-be patients to be cautious when shopping online for a doctor, since the ratings don't necessarily reflect the clinical competency of the physicians they rate.

A grain of salt
A recent study conducted by Loyola University Medical Center and published in the Journal of Urology reviewed the ratings of 500 urologists from more than a half-dozen websites including Healthgrades.com and Vitals.com. The study found that, on average, physician scores were based on ratings from an average of just 2.4 patients, a sample size so small that it made the ratings almost irrelevant.

"Our findings suggest that consumers should take these ratings with a grain of salt," said Chandy Ellimoottil, MD, one of the authors of the study.  "These sites have the potential to help inform consumers, but the sites need more reviews to make them more reliable."

The study included a qualitative analysis of comments written about physicians on the Vitals.com website. It found that 53% of the comments were positive or very positive and that 25% were negative, with the rest of the comments rated as neutral. The comments included one observation that an elderly physician "needs to retire because he can barely walk" and another that described a recent visit as "one of the best check-ups in a long time."

The fact that many physician rating sites don't generate enough reviews is reflected in a Pew Research Center survey that found only 16% of U.S. adults look online for physician ratings and that few of those users ever post a rating on a website.