LO81: Interrater agreement and time it takes to assign a Canadian Triage and Acuity Scale score pre and post implementation of eCTAS

Abstract

Introduction: In addition to its clinical utility, the Canadian Triage and Acuity Scale (CTAS) has become an administrative metric used by governments to estimate patient care requirements, emergency department (ED) funding and workload models. The electronic Canadian Triage and Acuity Scale (eCTAS) initiative aims to improve patient safety and quality of care by establishing an electronic triage decision support tool that standardizes that application of national triage guidelines across Ontario. The objective of this study was to evaluate triage times and score agreement in ED settings where eCTAS has been implemented. Methods: This was a prospective, observational study conducted in 7 hospital EDs, selected to represent a mix of triage processes (electronic vs. manual), documentation practices (electronic vs. paper), hospital types (rural, community and teaching) and patient volumes (annual ED census ranged from 38,000 to 136,000). An expert CTAS auditor observed on-duty triage nurses in the ED and assigned independent CTAS in real time. Research assistants not involved in the triage process independently recorded triage time. Interrater agreement was estimated using unweighted and quadratic-weighted kappa statistics with 95% confidence intervals (CIs). Results: 1491 (752 pre-eCTAS, 739 post-implementation) individual patient CTAS assessments were audited over 42 (21 pre-eCTAS, 21 post-implementation) seven-hour triage shifts. Exact modal agreement was achieved for 567 (75.4%) patients pre-eCTAS, compared to 685 (92.7%) patients triaged with eCTAS. Using the auditor's CTAS score as the reference standard, eCTAS significantly reduced the number of patients over-triaged (12.0% vs. 5.1%; Δ 6.9, 95% CI: 4.0, 9.7) and under-triaged (12.6% vs. 2.2%; Δ 10.4, 95% CI: 7.9, 13.2). Interrater agreement was higher with eCTAS (unweighted kappa 0.89 vs 0.63; quadratic-weighted kappa 0.91 vs. 0.71). Research assistants captured triage time for 3808 patients pre-eCTAS and 3489 post implementation of eCTAS. Median triage time was 312 seconds pre-eCTAS and 347 seconds with eCTAS (Δ 35 seconds, 95% CI: 29, 40 seconds). Conclusion: A standardized, electronic approach to performing CTAS assessments improves both clinical decision making and administrative data accuracy without substantially increasing triage time.