LO080: Performance and proximity: exploring resident factors that impact the quality of work-based assessments

Abstract

Introduction: Much of the literature investigating the challenges associated with completing high quality work-based assessments (WBAs) have raised specific concerns over the appropriate documentation of assessments of underperforming trainees or trainees in difficulty. The purpose of this study was to examine the relationship between resident performance and the quality of assessments documented by supervisors on Daily Encounter Cards (DECs). The effect of trainee proximity (i.e. on-service versus off-service status) on this relationship was also examined. Methods: A series of DECs from the Department of Emergency Medicine at the University of Ottawa was scored by two raters using the Completed Clinical Evaluation Report Rating (CCERR). The CCERR is a 9-item instrument that has previously demonstrated reliable ratings and the ability to discriminate the quality of completed DECs. A proxy measure of resident performance was calculated by averaging the scores across performance items on the DEC to produce a “mean DEC rating”. Linear regression analysis was conducted with “mean DEC rating” as the independent measure and CCERR score as the dependent measure. Separate linear regression analyses were repeated for DECs completed for on-service versus off-service residents. Results: Linear regression analysis demonstrated a small but significant inverse relationship between mean DEC rating and CCERR score (p<0.001, r=-0.184), suggesting that when residents performed poorly, their supervisors tended to document higher quality assessments, and conversely, when residents performed well, their supervisors provided lower quality assessments. Further analysis demonstrated that this relationship was present for the on-service group (p<0.001, r=-0.24). However, no relationship was observed in the off-service group (p=0.62, r=-0.05). Conclusion: Resident performance and trainee proximity are important factors impacting the quality of documented clinical performance assessments. Greater attention needs to be given to determining ways of improving the quality of assessments reported for residents who are appropriately progressing in their clinical competence as well as for off-service trainees.