LO85: Development of a competency based assessment tool for emergency department point of care ultrasound

Abstract

Innovation Concept: Assessment of residents' Point of Care Ultrasound (PoCUS) competency currently relies on heterogenous and unvalidated methods, such as the completion of a number of proctored studies. Although number of performed studies may be associated with ability, it is not necessarily a surrogate for competence. Our goal was to create a single Ultrasound Competency Assessment Tool (UCAT) using domain-anchored entrustment scoring. Methods: The UCAT was developed as an anchored global assessment score, building on a previously validated simulation-based assessment tool. It was designed to measure performance across the domains of Preparation, Image Acquisition, Image Optimization, and Clinical Integration, in addition to providing a final entrustment score (i.e., OSCORE). A modified Delphi method was used to establish national expert consensus on anchors for each domain. Three surveys were distributed to the CAEP Ultrasound Committee between July-November 2018. The first survey asked members to appraise and modify a list of anchor options created by the authors. Next, collated responses from the first survey were redistributed for a re-appraisal. Finally, anchors obtaining >65% approval from the second survey were condensed and redistributed for final consensus. Curriculum, Tool or Material: Twenty-two, 26, and 22 members responded to the surveys, respectively. Each anchor achieved >90% final agreement. The final anchors for the domains were: Preparation – positioning, initial settings, ensures clean transducer, probe selection, appropriate clinical indication; Image Acquisition – appropriate measurements, hand position, identifies landmarks, visualization of target, efficiency of probe motion, troubleshoots technical limitations; Image Optimization – centers area of interest, overall image quality, troubleshoots patient obstacles, optimizes settings; Clinical Integration – appropriate interpretation, understands limitations, utilizes information appropriately, performs multiple scans if needed, communicates findings, considers false positive and negative causes of findings. Conclusion: The UCAT is a novel assessment tool that has the potential to play a central role in the training and evaluation of residents. Our use of a modified Delphi method, involving key stakeholders in PoCUS education, ensures that the UCAT has a high degree of process and content validity. An important next step in determining its construct validity is to evaluate the use of the UCAT in a multi-centered examination setting.