LO70: Interrater agreement and time it takes to assign a Canadian Triage and Acuity Scale score pre and post implementation of eCTAS

Abstract

Introduction: In addition to its clinical utility, the Canadian Triage and Acuity Scale (CTAS) has become an administrative metric used by governments to estimate patient care requirements, ED funding and workload models. The Electronic Canadian Triage and Acuity Scale (eCTAS) initiative aims to improve patient safety and quality of care by establishing an electronic triage decision support tool that standardizes the application of national triage guidelines (CTAS) across Ontario. The objective of this study was to evaluate the implementation of eCTAS in a variety of ED settings. Methods: This was a prospective, observational study conducted in 7 hospital EDs, selected to represent a mix of triage processes (electronic vs. manual), documentation practices (electronic vs. paper), hospital types (rural, community and teaching) and patient volumes (annual ED census ranged from 38,000 to 136,000). An expert CTAS auditor observed on-duty triage nurses in the ED and assigned independent CTAS in real time. Research assistants not involved in the triage process independently recorded the triage time. Interrater agreement was estimated using unweighted and quadratic-weighted kappa statistics with 95% confidence intervals (CIs). Results: 1200 (738 pre-eCTAS, 462 post-implementation) individual patient CTAS assessments were audited over 33 (21 pre-eCTAS, 11 post-implementation) seven-hour triage shifts. Exact modal agreement was achieved for 554 (75.0%) patients pre-eCTAS, compared to 429 (93.0%) patients triaged with eCTAS. Using the auditors CTAS score as the reference standard, eCTAS significantly reduced the number of patients over-triaged (12.1% vs. 3.2%; 8.9, 95% CI: 5.7, 11.7) and under-triaged (12.9% vs. 3.9%; 9.0, 95% CI: 5.9, 12.0). Interrater agreement was higher with eCTAS (unweighted kappa 0.90 vs 0.63; quadratic-weighted kappa 0.79 vs. 0.94). Research assistants captured triage time for 4403 patients pre-eCTAS and 1849 post implementation of eCTAS. Median triage time was 304 seconds pre-eCTAS and 329 seconds with eCTAS ( 25 seconds, 95% CI: 18, 32 seconds). Conclusion: A standardized, electronic approach to performing CTAS assessments improves both clinical decision making and administrative data accuracy without substantially increasing triage time.