Program directors and coordinators can spend countless hours chasing after residents to get them to complete administrative tasks, such as evaluations, procedure logs, and duty hour reporting.
Between restricted work hours, educational commitments, and patient care responsibilities, residents often put paperwork on the backburner.
“Individuals in residency programs are overwhelmed by the large amount of work they have to do for ACGME [Accreditation Council for Graduate Medical Education] documentation purposes,” says John R. Barbour, MD, fellow in the Division of Plastic and Reconstructive Surgery at the Medical University of South Carolina (MUSC) in Charleston.
However, submission of documentation showing how the program meets ACGME standards is critical to maintaining accreditation. Site visitors want to see completed evaluations, duty hour reports, and procedure logs.
Additionally, without data provided on evaluations and procedure logs, the program misses out on feedback regarding the quality of education received on a rotation or from an attending physician. Without that feedback, it becomes difficult for educational leaders to identify areas for improvement.
Although residency management systems have streamlined the collection of many administrative items, compliance with documentation requirements can still be a problem for programs.
One way to get residents motivated about completing required documentation is to publicly display an administrative grade. Barbour and program leaders took this route. They graded surgery residents on their ability to turn in paperwork and logs and saw compliance rates increase dramatically. “Everyone averaged 30%–40% compliance until we displayed a grade publicly, and then everyone went to far greater than 80%, with most people above 95%,” says Barbour.
Enhance your residents’ documentation compliance rates by implementing a similar administrative grade in your program.
What is an administrative grade?
An administrative grade is a composite score based on the completion and submission of logs and evaluations. The grade should include the items you have the most trouble getting residents to submit, Barbour suggests.
The administrative grade in Barbour’s program comprises five categories, and a percentage value for each is calculated based on data at the end of each month. The categories are:
- Duty hour logging (how often the resident logged his or her hours on time)
- Duty hour compliance (the percentage of times the resident complied with the ACGME work hour regulations)
- Lecture attendance (how often the resident attended the weekly basic science lecture)
- Evaluation completion (the percentage of evaluations completed, including peer, rotation, and faculty evaluations)
- Case log submission (how often the resident was on time logging his or her operative case log and providing a hard copy to the program director)
The composite administrative score is calculated by averaging the scores for each group. The letter grades are based on a percentage scale:
- A: 90%–100%
- B: 80%–89%
- C: 70%–79%
- D: 60%–69%
- F: 59% and lower
Barbour and MUSC surgery program director Thomas E. Brothers, MD, post a report monthly in the resident lounge (see Sample administrative report on p. 3). The report lists:
- Residents’ names
- Composite grades
- Residents’ rank compared to peers
- Grades for each documentation requirement category factored into the composite score
Methods for collecting the data do not need to change when you implement the administrative grade, says Susan F. Wetherholt, surgery program coordinator at MUSC.
“I worked closely with the program director, Dr. Brothers, to provide him the data needed to calculate the scores for each category,” Wetherholt says.
Duty hour and evaluation data were gathered via the program’s electronic residency management system. Residents entered their case log information at the end of each rotation on the ACGME’s website. Attending surgeons validated printed copies; the program director also received a copy. Weekly conference attendance was measured using a badge scanning system: Residents simply scan their badge and the system generates a report, Wetherholt explains.
Brothers built a spreadsheet that calculates the administrative score and generates the monthly report. Calculating the composite score only takes about one hour per month because the formulas in the spreadsheet are constants—all that’s needed is to enter the data.
Why the administrative grade works
Posting the grades in a public place for everyone to see brings out the competitive nature in residents.
“People who are competitive want to do well. They are disappointed at poor performance, and they’re going to do better when evaluated in a public setting,” he explains.
Barbour says he and Brothers tracked documentation compliance for one year prior to implementing the administrative grade and then tracked it for a year after the intervention. They found that completion of all evaluations improved, as follows:
- Peer evaluations: 90% vs. 62%
- Rotation evaluations: 87% vs. 60%
- Faculty evaluations: 86% vs. 52%
- Work hour logging: 87% vs. 73%
- Work hour compliance: 99% vs. 61%
- Case log completion: 80% vs. 73%
At the end of the year, the resident with the best score received a prize. “This is positive reinforcement for complying with requirements rather than a negative or punitive approach,” Barbour says.
Publicly displaying residents’ grades prepares them for a future in which physicians will be more heavily scrutinized by the public.
“There is an impending public display of all aspects of physicians’ or healthcare providers’ outcomes in the future,” Barbour explains. “Residents are not going to be well prepared for having their individual outcomes or morbidity/mortality displayed.”
The administrative grade also helps residents learn how to self-assess. Once trainees see their grade, they self-assess and adjust their behaviors in order to improve. This hits on the practice-based learning and improvement competency, says Barbour.