The Off-Service Letter of Evaluation....the Over-Ranked Service Letter of Evaluation?
Abstract
Learning Objectives: To review the results of the Off-Service Letter of Evaluations (OSLOEs) in the 2020-21 academic year and analyze their utility and value in the emergency medicine residency application process.
Background: Standardized Letters of Evaluation (SLOE)s are designed to objectively compare medical students to their peers for completed emergency medicine (EM) rotations. Coronavirus disease of 2019 (COVID-19) mitigation efforts decreased medical students’ ability to obtain multiple SLOEs for their application to the EM match. To compensate, the Council of Residency Directors in Emergency Medicine (CORD) implemented “off-service” SLOEs (OSLOEs). The purpose of our study is to summarize the OSLOEs submitted during the 2020-21 academic year and assess for grade inflation and overall utility of the letters for applicant selection.
Methods: A retrospective review of OSLOEs submitted during the 2020-21 academic year to a single EM residency program was performed. Summary statistics for global rank (top 10%, top ⅓, middle ⅓, and lower ⅓), grade (honors, high pass, pass, low pass, fail) and specific category (knowledge, work ethic, communication, teachability, respectfulness, admits mistakes, accountable, and reliability) ranks were calculated.
Results: A total of 270 OSLOEs were reviewed and summarized. Global assessments revealed 61.9% were ranked at the top 10% of their class, with 95% being ranked in the top 10% and top ⅓. No student was ranked in the bottom ⅓ of their class. Over 90% of students were graded as honors or high pass; no students received low pass or failing grades. Over 75% of students were ranked in the top ⅓ for each specific OSLOE category.
Conclusion: In an attempt to adapt quickly to the lack of availability of in-person EM rotations due to COVID-19, the OSLOE was a logical alternative. However, our findings reveal signs of grade inflation providing evidence that the ranking distribution of the OSLOE may have little value in the evaluation of student performance. Given our findings, the OSLOE may not carry the same weight as a SLOE when objectively evaluating prospective students for a match into EM.