P061: Implementing CBME in emergency medicine: lessons learned from the first 6 months of transition at Queens University

Abstract

Introduction: The specialist Emergency Medicine (EM) postgraduate training program at Queens University implemented a new Competency-Based Medical Education (CBME) model on July 1 2017. This occurred one year ahead of the national EM cohort, in the model of Competence By Design (CBD) as outlined by the Royal College of Physicians and Surgeons of Canada (RCPSC). This presents an opportunity to identify critical steps, successes, and challenges in the implementation process to inform ongoing national CBME implementation efforts. Methods: A case-study methodology with Rapid Cycle Evaluation was used to explore the lived experience of implementing CBME in EM at Queens, and capture evidence of behavioural change. Data was collected at 3- and 6- months post-implementation via multiple sources and methods, including: field observations, document analysis, and interviews with key stakeholders: residents, faculty, program director, CBME lead, academic advisors, and competence committee members. Qualitative findings have been triangulated with available quantitative electronic assessment data. Results: The critical processes of implementation have been outlined in 3 domain categories: administrative transition, resident transition, and faculty transition. Multiple themes emerged from stakeholder interviews including: need for holistic assessment beyond Entrustable Professional Activity (EPA) assessments, concerns about the utility of milestones in workplace based assessment by front-line faculty, trepidation that CBME is adding to, rather than replacing, old processes, and a need for effective data visualisation and filtering for assessment decisions by competency committees. We identified a need for administrative direction and faculty development related to: new roles and responsibilities, shared mental models of EPAs and entrustment scoring. Quantitative data indicates that the targeted number of assessments per EPA and stage of training may be too high. Conclusion: Exploring the lived experience of implementing CBME from the perspectives of all stakeholders has provided early insights regarding the successes and challenges of operationalizing CBME on the ground. Our findings will inform ongoing local implementation and higher-level national planning by the Canadian EM Specialty Committee and other programs who will be implementing CBME in the near future.