Adams says he doesn't think revising the evaluation process has to start from square one. "There are a lot of pieces here that could be useful," Adams said. "There has been a failure on a number of levels. The people that use these tools should have done more physician engagement for two reasons: One, the physicians could have made the tool better; and two, why pick a gratuitous fight?"
The Massachusetts Medical Society (MMS) is involved in litigation against the Massachusetts Group Insurance Commission (GIC), which purchases health insurance for state employees and other public sector employees, because of its requirement that health plans tier physicians. That litigation involves GIC and two participating health plans, according to MMS.
Mario E. Motta, MD, president of MMS, said his organization is not against public reporting as long as it is accurate and fair. "It's critically important that patients and physicians get clear, accurate information about the cost and quality of healthcare. But this report, produced by an independent, renowned research firm, clearly demonstrates that these profiling programs fail to accomplish those goals."
Ranking systems not reliable
The RAND researchers analyzed information from insurance claims for 2004 and 2005 from four health plans in Massachusetts that provide coverage to about 80% of non-elderly adults with private insurance. The study examined the costs of treating common illnesses, such as diabetes and heart attack, assigning each care episode to a physician, and creating a cost profile for each physician based on all similar episodes of care.
The researchers evaluated the reliability of physician cost scores by considering factors, such as the number and types of patients physicians treated. The results show that the reliability of cost-profiling scores was unacceptably low for physicians in most of the specialty groups.
Researchers also examined how reliability scores might change under different scenarios, such as requiring at least 30 episodes of treatment to create a profile and different methods for assigning episodes to physicians. While some scenarios modestly increased reliability, the results still fell short.
"These ranking systems may be useful for some purposes, but they are not reliable enough at this point to make decisions about encouraging patients to see certain providers or excluding some doctors from insurance networks," Adams said. "Much work remains to be done to improve these systems before they are used for high-stakes activities."
He adds that the current systems may be useful for warning physicians that their treatment methods may cost more than those of their peers and urging them to reexamine their practice styles.
Adams says cost profiling "can be made better," but it can't be successful until better tools are developed to use claims data and other information to create reliable cost profiles for physicians.
"One of the things you need to do is get a handle on how difficult the provider's patient mix is. It's an illusion that you have it now," he says. "I have hopes that some of the noise can be taken out of this with better data and particularly the stuff we are going to get out of improved information systems. Even the simplest EHR is going to give us blood pressures and whether or not the person is overweight. That is going to be a big help."
"It will be an evolutionary process. I can see these things on the multiyear scale getting up to an adequate standard," Adams says.