When 310 doctors at 22 U.S. healthcare facilities were asked to anonymously confess the diagnostic errors they made or witnessed, the two most frequently listed conditions involved pulmonary embolism and adverse drug reactions, including overdoses and poisoning. Lung cancer diagnostic mistakes ranked a close third, followed by colorectal cancer, acute coronary syndrome, breast cancer, and stroke.

The doctors failed to: order tests, report the results to their patients or follow-up when testing revealed abnormal findings.

As it turns out, lab and radiology testing errors, including test ordering, test performance, and clinician processing, accounted for 44% of the missed diagnoses, which was the greatest share. Those surveyed included descriptions of 583 diagnostic errors by primary care and specialist physicians.

Those are some conclusions from a report by Gordon Schiff, MD, associate director at the Center for Patient Safety Research and Practice at Brigham and Women's Hospital in Boston, and colleagues at five other institutions. The authors describe the survey as one of "the largest reported case series of diagnostic errors to date and affords valuable insights into the types of errors that physicians are committing and witnessing." It was published in the Nov. 9 Archives of Internal Medicine, and received funding from the Agency for Healthcare Research and Quality.

Additionally, according to a May 2008 American Journal of Medicine article titled Overconfidence as a Cause of Diagnostic Error in Medicine, errors in diagnosis constitute as many as 5% of errors in perceptual specialties and 10% to 15% of errors in other fields. Often these errors are made because of a thinking failure, said Pat Croskerry, MD, PhD. Croskerry and Schiff spoke on an Institute for Healthcare Improvement’s WIHI program last week.

"This doesn't ever seem to be a feature of someone not trying hard enough," said Croskerry, professor in emergency medicine at Dalhousie University in Halifax, Nova Scotia. "Historically, there has been a lot of confidence placed in physicians and their thinking abilities."

In the field of cognitive psychology there's been a large focus on evaluating thinking failures. It's a short step to apply that to medicine, which is in large part a thinking business, said Croskerry.

However, diagnosis errors are both cognitive- and systems-based. "These two worlds needed to come together, rather than being separate silos," said Schiff of trying to classify a diagnosis error as either a cognitive or a system error.

Causes of diagnosis error

While the notion of overconfident, arrogant physicians being the cause of diagnosis error is not wholly wrong, those qualities alone are certainly not the reason that misdiagnoses are made, said Schiff. Other factors include spotty follow-up, time pressure, failure of physicians to share their uncertainties, malpractice fears, defensiveness, and inadequate feedback.

Schiff likened this last factor to the lawn sprinkler system that goes on automatically, regardless of whether it has rained that day. Instead of acting in a closed-loop system that provides feedback about whether diagnoses were right or wrong, often physicians work in an environment that does not allow for this follow-up or does not attempt to capture this feedback.

"What comes across as arrogance and carelessness is often a lot of constraints that physicians are working under," said Schiff.

Additionally, understanding why people think in certain ways is key to understanding diagnosis error, said Croskerry. By undoing certain biases in thinking, one might be able to think more clearly and perhaps come up with a more accurate diagnosis. But this task is easier said than done.

"The problem is it's extremely difficult to de-bias people," said Croskerry. "Generally speaking, trying to change the way that people think is a very challenging task." He recommended addressing physician thinking biases in medical school.

"Not Yet Diagnosed" may be better for patient

Another way of thinking that has become ingrained in most physicians is the idea that an overconfident, perhaps wrong diagnosis is better than not diagnosing a patient at all. Most patients don't appreciate when their physicians don't seem decisive about a diagnosis, and historically, physicians have been more successful when they confidently make a diagnosis, said Croskerry. Additionally, physicians validate their ability to make diagnoses when they are confident.

"Physicians tend to place a lot of faith in their own diagnoses—most physicians think it's the most important skill that they have," said Croskerry. "It's a lot easier, if you give the patient wrong medication, to admit to something like that than to actually admit to your thinking processes having gone astray. People take that far more personally."

Once a diagnosis is made, many other decisions concerning medical care are then made based off of it, said Croskerry. Further thinking about what else the diagnosis could possibly be tends to stop. Assigning a "Not Yet Diagnosed" or "NYD" label to a patient whose symptoms don't lead to a specific, certain diagnosis may ultimately help the physician ascertain the true cause of an illness—by thinking further on what the patient might have. Croskerry has seen success in Canada with this technique.

Both Schiff and Croskerry agreed that more emphasis should be placed on physicians admitting to their patients uncertainty about diagnosis and that more work needs to be done at an earlier stage, perhaps in medical school, to introduce the idea of feeling comfortable with uncertainty to physicians. Additionally, Schiff said the question of physicians admitting a diagnosis error is first and foremost about a patient safety culture.

"This idea about patient safety culture—creating a system where people can honestly look at errors in a blame-free way, learning from mistakes and improving from those, rather than covering them up or having to defend them—is so central for us learning," said Schiff.

Technology: a double edged sword

Technology has allowed the medical profession to make better decisions as far as diagnoses go. Clinical decision support and electronic medical records offer places where some progress has been made, and more will likely be seen in the future, although one disadvantage of computerized decision-making is its inability to read the context of a doctor/patient interaction, said Croskerry.

Medical tests, however, are an area where the "ball" is often dropped with diagnoses, said Schiff. Instead of physicians making a diagnosis and ordering tests to confirm, the reverse often happens.

"We are now short-circuiting the diagnostic process and going right to diagnostic tests," said Schiff. "The questions is, 'Is this a step forward or a step backward?' These are powerful new modalities to hopefully make us do better with diagnosis, but they also introduce all sorts of problems from ordering the right tests, to harming people with these modalities (the radiation from repeated x-rays)."

Schiff also said it's important for physicians to ensure they are using the tests properly, interpreting the results correctly, and following up with patients to measure if the course of treatment based on a test was the best one.

Heather Comak is a Managing Editor at HCPro, Inc., where she is the editor of the monthly publication Briefings on Patient Safety, as well as patient safety-related books and audio conferences. She is also is the Assistant Director of the Association for Healthcare Accreditation Professionals. Contact Heather by e-mailing hcomak@hcpro.com.

Cheryl Clark is a senior editor and California correspondent for HealthLeaders Media Online. She can be reached at cclark@healthleadersmedia.com.