Peer Review of Surgeons' Skills Carries 'Threatening Undertones'
HLM: I want to come back to the idea that in the view of these participating surgeons, this was their best work they submitted, right?
JB: The advantage of having surgeons select videos of their best work makes this even more reproducible. If you didn't control for case difficulty, you'd introduce bias and measurement noise.
HLM: Were you surprised that fellowship completion or teaching hospital practice weren't linked to better outcomes?
JB: No. Keep in mind that we were focusing on surgeons who had been practicing for a mean of a decade. They had done thousands of bariatric operations.
HLM: How can you protect against bias among the peer reviewers? Perhaps some might favor a technique or sequence similar to their own.
JB. We found very little evidence of that. I worried we'd have hard graders and easy graders that would contaminate findings. We re-rated all these videos with a second expert panel, with no attachment to the Michigan collaborative, and they were slightly harsher. And all the best surgeons in the original rating were still the best, and the worst surgeons still the worst.
And we had the best and worst surgeons submit a second video, just to make sure there wasn't something weird about what they submitted on the first pass. And again, when we re-rated those, the best and worst got put in exactly the same place.
And when I showed these videos to non-surgeons, including lay people, with no labels or prompts, every single lay person said, "Oh my God, I definitely would want the first guy," and they were exactly right, of course. Everything wasn't quite as good.
Cheryl Clark is senior quality editor and California correspondent for HealthLeaders Media. She is a member of the Association of Health Care Journalists.
- Two-Midnight Rule Must be Fixed or Replaced, Say Providers
- Don't Underestimate Emotional Intelligence
- The Secret to Physician Engagement? It's Not Better Pay
- Care Coordination Tough to Define, Measure
- Yale New Haven Health Partners with Tenet Healthcare in CT
- Physicians Take SGR Repeal Message to Washington
- Size Matters in Antibiotic Overuse
- CDC Warns of Antibiotic Overuse in Hospitals
- SCOTUS Review of NC Board Case 'A Very Big Deal' to Providers
- 4 Reasons PCMH Principles Aren't Going Away