Quality e-Newsletter
Intelligence Unit Special Reports Special Events Subscribe Sponsored Departments Follow Us

Twitter Facebook LinkedIn RSS

Peer Review of Surgeons' Skills Carries 'Threatening Undertones'

Cheryl Clark, for HealthLeaders Media, October 17, 2013

HLM: I want to come back to the idea that in the view of these participating surgeons, this was their best work they submitted, right?

JB: The advantage of having surgeons select videos of their best work makes this even more reproducible. If you didn't control for case difficulty, you'd introduce bias and measurement noise.

HLM: Were you surprised that fellowship completion or teaching hospital practice weren't linked to better outcomes?

JB: No. Keep in mind that we were focusing on surgeons who had been practicing for a mean of a decade. They had done thousands of bariatric operations.

HLM: How can you protect against bias among the peer reviewers? Perhaps some might favor a technique or sequence similar to their own.

JB. We found very little evidence of that. I worried we'd have hard graders and easy graders that would contaminate findings. We re-rated all these videos with a second expert panel, with no attachment to the Michigan collaborative, and they were slightly harsher. And all the best surgeons in the original rating were still the best, and the worst surgeons still the worst.

And we had the best and worst surgeons submit a second video, just to make sure there wasn't something weird about what they submitted on the first pass. And again, when we re-rated those, the best and worst got put in exactly the same place.

And when I showed these videos to non-surgeons, including lay people, with no labels or prompts, every single lay person said, "Oh my God, I definitely would want the first guy," and they were exactly right, of course. Everything wasn't quite as good.


Cheryl Clark is senior quality editor and California correspondent for HealthLeaders Media. She is a member of the Association of Health Care Journalists.
Twitter
1 | 2 | 3 | 4 | 5

Comments are moderated. Please be patient.

2 comments on "Peer Review of Surgeons' Skills Carries 'Threatening Undertones'"


Rosemary Gibson (10/21/2013 at 7:53 AM)
Too bad that the title chosen for this article suggests the author and editors are concerned more about the surgeons than the patients. Patients want a competent surgeon. How is that threatening? Rosemary Gibson Senior Advisor, The Hastings Center

S.Noel,M.D. (10/18/2013 at 9:46 PM)
This study provides objective documentation of facts well known to those of us who have done agonizingly difficult surgical peer review for decades. Unfortunately, Medical Staff peer review bodies have little ability or authority to assist or force poor performers to improve their skills, if their outcomes are "acceptable", but not "optimal". Who is to define what is "acceptable"?