Introduction: Emergency department (ED) crowding is a major problem across Canada. We studied the ability of artificial intelligence methods to improve patient flow through the ED by predicting patient disposition using information available at triage and shortly after patients’ arrival in the ED. Methods: This retrospective study included all visits to an urban, academic, adult ED between May 2012 and June 2019. For each visit, 489 variables were extracted including triage data that had been collected for use in the Canadian Triage Assessment Scale (CTAS) and information regarding laboratory tests, radiological tests, consultations and admissions. A training set consisting of all visits from April 2012 up to December 2018 was used to train 5 classes of machine learning models to predict admission to the hospital from the ED. The models were trained to predict admission at the time of the patient's arrival in the ED and every 30 minutes after arrival until 6 hours into their ED stay. The performance of models was compared using the area under the ROC curve (AUC) on a test set consisting of all visits from January 2019 to June 2019. Results: The study included 536,332 visits and the admission rate was 15.0%. Gradient boosting models generally outperformed other machine learning models. A gradient boosting model using all available data at 2 hours after patient arrival in the ED yielded a test set AUC 0.92 [95% CI 0.91-0.93], while a model using only data available at triage yielded an AUC 0.90 [95% CI 0.89-0.91]. The quality of predictions generally improved as predictions were made later in the patient's ED stay leading to an AUC 0.95 [95% CI 0.93-0.96] at 6 hours after arrival. A gradient boosting model with 20 variables available at 2 hours after patient arrival in the ED yielded an AUC 0.91 [95% CI 0.89-0.93]. A gradient boosting model that makes predictions at 2 hours after arrival in ED using only variables that are available at all EDs in the province of Quebec yielded an AUC 0.91 [95% 0.89-0.92]. Conclusion: Machine learning can predict admission to a hospital from the ED using variables that area collected as part of routine ED care. Machine learning tools may potentially be used to help ED physicians to make faster and more appropriate disposition decisions, to decrease unnecessary testing and alleviate ED crowding.

Introduction: Emergency Department (ED) crowding is a pervasive problem and is associated with adverse patient outcomes. Yet, there are no widely accepted, universal ED crowding metrics. The objective of this study is to identify ED crowding metrics with the strongest association to the risk of ED revisits within 72 hours, which is a patient-oriented adverse outcome. Methods: Crowding metrics, patient characteristics and outcomes were obtained from administrative data for all ED encounters from 2011-2014 for three adult EDs in Calgary, AB. The data were randomly divided into three partitions for cross-validation, and further divided by CTAS category 1, 2/3 and 4/5. Twenty unique ED crowding metrics were calculated and assigned to each patient seen on each calendar day or shift, to standardize the exposure. Logistic regression models were fitted with 72h ED revisit as the dependent variable, and an individual crowding metric along with a common list of confounders as independent variables. Adjusted odds ratios (OR) for the 72h return visits were obtained for each crowding metric. The strength of associations between 72h revisits and crowding metrics were compared using Akaike's Information Criterion and Akaike weights. Results: This analysis is based on 1,149,939 ED encounters. Across all CTAS groups, INPUT metrics (ED census, ED occupancy, waiting time, EMS offload delay, LWBS%) were only weakly associated with the risk of 72h re-visit. Among THROUGHPUT metrics, ED Length of Stay and MD Care Time had similar adjusted ORs for 72h ED re-visit (range 0.99-1.15). Akaike weights ranging from 0.3/1.00 to 0.4/1.00 indicate that both THROUGHPUT metrics are reasonable predictors of 72h ED re-visits. All OUTPUT metrics (boarding time, # of boarded patients, % of beds occupied by boarded patients, hospital occupancy) had statistically significant ORs for 72h ED re-visits. The median boarding time had the highest adjusted OR for 72h ED re-visit (adjusted OR 1.40, 95% CI 1.33-1.47) and highest Akaike weight (0.97/1.00) compared to all other OUTPUT metrics, indicating that median boarding time had the strongest association with 72h re-visits. Conclusion: ED THROUGHPUT and OUTPUT metrics had consistent associations with 72h ED re-visits, while INPUT metrics had little to no association with 72h re-visits. Median boarding time is the strongest predictor of 72h re-visits, indicating that this may be the most meaningful measure of ED crowding.

Introduction: In 2008 British Columbia created a nurse (RN) staffed telephone triage service, (TTS) to provide timely advice to non-911 callers (811). A perception exists that some callers are inappropriately directed to emergency departments (EDs) thereby worsening crowding. We sought to determine whether supplementary emergency physician (EP) triage would decrease ED visits while preserving caller safety and satisfaction. Methods: TTS RNs use computer algorithms and judgment to triage callers. Potentially sick callers are directed to “seek care now” (red calls). Often this is to an ED depending on acuity and time of day. In the Vancouver Health Region from April-September 2016 between 8:00-24:00 hours, a co-located EP also spoke with “red” callers to provide further guidance. Callers were followed up with 1 week and satisfaction was evaluated on a 5-point Likert scale. The TTS data was linked to the regional ED database to assess ED attendance within 7 days, and the provincial vital statistics database for 30-day mortality. Our primary outcome was the proportion of unique “red” callers who did not attend the ED compared with a historical cohort one year earlier without EP triage in place. Secondary outcomes were the proportion of “red” callers advised not to attend the ED but (a) attended, (b) admitted, or (c) died. Results: In the study period there were 5105 “red” calls of which 3440 were transferred to the EP (67.4%), 2958 of EP assessed callers (86.0%) had a family doctor, but only one-quarter of such patients could contact their family doctor. Overall, 2301/3440 “red” callers did not attend an ED (67.0%) compared to 2508/4770 in the control period (52.6%), for an absolute reduction of 14.4% (95% CI 12.2 to 16.4%, p<0.0001). In callers for those <17 years old there was a 20.3% (95% CI 16.5 to 24.1%) reduction in ED visits compared to the control group: 771/1520 (50.7%) vs 364/1067 (30.4%). 40% of callers attending an ED (458/1139) were advised to try non-ED follow up by the MD and 108 (9.5%) were admitted, with no difference in 30-day mortality between groups. Age and CTAS distribution were similar between the two groups and the non MD-transferred cohort. Mean caller satisfaction was excellent (4.7/5.0). Conclusion: EP supplementation of a RN advice service has the potential to reduce ED visits by almost 15% while providing excellent safety and satisfaction.

Introduction: Highly frequent users (HFU) of the emergency department (ED) remain a poorly defined and complex population. This study describes patient and visit characteristics for HFU of the ED, and analyzes subgroups of patients with mental illness, substance abuse, and/or ≥30 yearly ED visits. Methods: We performed a health records review of 250 randomly selected adults with >99th percentile of ED visit frequency (≥7 visits) at a tertiary care academic hospital with two EDs in 2014. Two reviewers collected demographic variables (age, sex, and comorbidities) and visit data (ED diagnosis, ED length of stay (LOS), ED presentation time (daytime 0800-1559 h, evening 1600-2359 h, overnight 2400-0759 h), consultation services, and final disposition). Data were analyzed using descriptive and univariate analyses, student t and Mann Whitney U tests. Results: Of 897 eligible patients who experienced 9,376 ED visits we included 250 patients (2,670 visits) in our main analyses, and an additional 11 patients (494 visits) outside of the random selection with ≥30 ED visits. Mean age was 53.4±1.3 (SEM), and 55.6% were female. Most patients had a fixed address (88.9%), and a family physician (87.2%). Top comorbidities included gastrointestinal (61.6%), cardiovascular (52%), and chronic pain issues (47.2%). Top ED diagnoses included musculoskeletal pain (9.6%), abdominal pain (8.4%) and alcohol-related presentations (8.5%). Hospital admission was required for 15.6% of visits. From all possible visits (3164 visits), consultations for social workers, geriatric emergency medicine nurses, or Community Care Access Centres were made for 5.9% of visits, with 47.3% of these patients presenting during daytime hours. Among visits requiring these consultations, median ED LOS was greatest in the evening (12.7 hours, range 1.4-45.2 hours), compared to daytime (5.4, 1.2-33.6; p=0.0002) or overnight (7.9, 1.0-38.3, p=0.02). Inter-rater review of 4.5% of abstracted health records revealed a kappa score of 0.8. Conclusion: This study highlights that a remarkably low proportion of HFUs received allied health consultations at the study sites, likely corresponding to a lack of available consultants outside of daytime work hours. Our findings suggest the need to address significant gaps in order to balance the clinical needs of patients who frequent the ED with currently available resources.

Introduction: Over 700 different input, throughput and output metrics have been used to quantify ED crowding. Of these, only ED length-of-stay (ED LOS) has been shown to be associated with mortality. No comparative evaluation of ED crowding metrics has been performed to determine which ones have the strongest association with patient mortality. The objective of this study was to compare the strength of association of common ED input, throughput and output metrics to patient mortality. Methods: Administrative data from five years of ED visits (2011-2014) at three urban EDs were linked to develop a database of over 900,000 ED visits with patient demographics, electronic time stamps for care processes, dispositions and outcomes. The data were randomly divided into three partitions of equal size. Here we report the findings from one partition of 253,938 ED visits. The remaining two data partitions will be used to validate these findings. Commonly-used crowding metrics were quantified and aggregated by day or by shift (0800-1600, 1600-2400, 2400-0800), and the shift-specific metrics assigned to each patient. The primary outcome was 7-day all-cause mortality. Multilevel logistic regression models were developed for 7-day mortality, with selected ED crowding metrics and a common set of confounders as predictors. The strength of association between the crowding metrics and mortality was compared using Akaike’s Information Criterion (AIC) and the Bayesian Information Criterion (BIC): ED crowding metrics with lower AIC and BIC have stronger associations with 7-day mortality. Results: Of 909,000 ED encounters, 124,679 (16.5%) arrived by EMS, 149,233(19.7%) were admitted, and 3,808 patients (0.5%) died within 7 days of ED arrival. Of input metrics, the model with ED wait-time was better (i.e. had a smaller AIC and BIC) than models for daily census, ED occupancy or LWBS proportion for predicting 7-day mortality. Of throughput metrics, the model with mean ED LOS was better than the model for mean MD care time. Of output metrics, the model with daily inpatient hospital occupancy was better than the model with mean boarding time. Conclusion: Based on one data partition, regression models based on the average wait-time, ED LOS and inpatient occupancy best predicted 7-day mortality. These results will be validated in the two other data partitions to confirm the best-performing ED input, throughput and output metrics.