LO44: Optimizing skill retention in radiograph interpretation: a multicentre randomized control trial

Abstract

Introduction: Simulation-based education systems have increased physician skill in radiograph interpretation. However, the degree of skill retention and the factors that influence it are relatively unknown. The main objective of this research was to determine the rate and quantity of skill decay in post-graduate trainee physicians who completed a simulation-based learning intervention of radiograph interpretation. The impact of testing and refresher education on skill decay was also examined. Methods: This was a prospective, multicenter, analysis-blinded, four arm randomized control trial conducted from November 2014 to June 2016. Study interventions were administered using an on-line learning and measurement platform. Pediatric and emergency medicine residents in the United States and Canada were eligible for study participation. Participants were randomized to one of four groups. All participants completed an 80-case deliberately practiced learning set of pediatric elbow radiographs followed by an immediate 20-case post-test. Following this, Group 1 had no testing until 12 months; Groups 2, 3, and 4 had testing (20 cases without feedback) every 2 months until 12 months, but Group 3 also had refresher education (20 cases with feedback) at six months while Group 4 had refresher education at two, six, and ten months. The main outcome measure was accuracy at 12 months, adjusted for immediate post-test score, days to completion of 12 month test, and time on case. Based on prior data, we assumed the smallest important difference between groups in learning decay is 10%, a between-participant/within-group standard deviation of 17%, a type I error probability of 5%, a power of 80% and adjusted for three tests with a Bonferroni correction. For the primary analysis of Group 1 versus 2, 3, 4, this resulted in a minimal total sample size of 56, with 14 participants per group. Results: We enrolled 106 participants that completed all study interventions. The sample sizes in Groups 1, 2, 3, and 4 were 42, 22, 22, and 20 respectively. Overall, accuracy increased by 11.8% (95% CI 9.8, 13.8) with the 80-case learning set and then decreased by 5.5% (95% CI 2.5, 8.5) at 12 months. The difference in learning decay in Group 1 vs. Groups 2, 3, 4 was -8.1% (95% CI 2.5, 13.5), p=0.005. For Group 2 vs. Group 3 and 4, it was +0.8% (95% CI -7.2, 7.3), p=0.8, and between Group 3 vs. Group 4 it was +0.8% (95% CI -7.3, 10.1), p=0.8. Conclusion: Skill decay was significantly reduced by testing with 20 cases every two months. Refresher education had no additional effect to testing on reducing learning decay.