Comparison of the Sacco Triage Method Versus START Triage Using a Virtual Reality Scenario in Advance Care Paramedic Students

Introduction

Since the time of the Napoleonic wars, health care providers have discussed, debated and invented multiple triage systems designed to rapidly triage victims in order to save the most lives. ¹ A key medical procedure in a mass casualty incident (MCI) is the rapid and accurate sorting of victims with life, limb, or serious injuries from those without. ²

There are different triage methods proposed for use during a MCI, but few have been validated scientifically. ³ Validation of field triage methods for MCI is difficult because of issues of rarity, reproducibility, study groups, logistics, and different processes using different methods. Most of the evidence to date has been based on simulation and post-event evaluations, rather than prospective trials. ² START (Simple Triage and Rapid Treatment) is now the most widely used method in the world, but there has been little scientific evaluation of this triage method. ^{4 - 6} This method divides patients into ambulatory and non-ambulatory, and then, using a set of basic physiologic parameters, sorts remaining patients into immediate, delayed, minor, or expectant categories, with each victim receiving a color code of red, yellow, green, or black.

After the attacks on September 11th, 2001, in New York City, ThinkSharp Inc. began developing a commercial, evidence-based and outcome-driven approach called the Sacco Triage Method (STM). ^{4 , 6} It is the only method evaluated for National Incident Management System compatibility in the United States (US). ^{4 , 6 , 7} STM uses a simple physiologic score that predicts survival and deterioration, where patients are triaged to maximize expected survivors in consideration of the timing and availability of transport and resources. ⁸ In addition to the START and STM triage systems having been derived using markedly different methods, an important difference between the two triage methods is that the START system is open-source and in the public domain, while the STM system is proprietary and licensed for commercial use only.

The aim of this study was to use virtual reality (VR) simulation of actual historical train crash MCI victims to compare advance care paramedic students’ use of START and STM triage methods in terms of time to triage and assignment of severity order of MCI victims.

Methods

Study design

This was a prospective observational cohort study using VR simulation technology based on actual MCI cases that resulted from a train accident in Chatsworth, Los Angeles, California (US) on September 12, 2008. ⁹ The ethics committee for human research at Holland College, Charlottetown, Prince Edward Island, Canada approved the study.

Study setting and population

Twenty-six students in their final year of the Advance Care Paramedic program at Holland College were invited to participate in the study. Inclusion criteria included male and female students who were 18 years or older and in good academic standing within the Advanced Care Paramedic program. Potential participants were excluded if they opted not to voluntarily participate in the study or were unable to complete the study. All study participants consented to be included in the study. The study database contained no participant personal identifiers.

Study protocol

The XVR training software (E-Semble, Delft, the Netherlands) was used for developing the VR study scenario. ¹⁰ XVR is currently used by different institutions and organizations in more than 16 countries. ¹⁰ It has been validated as an effective simulation tool for disaster management training. ¹⁰ XVR has also been demonstrated to be a reliable and effective tool for training and teaching different mass casualty triage methods. ¹¹ The XVR library contains dozens of 3D environments with multiple open objects from casualties, vehicles, buildings, sounds, and equipment. Each casualty has physiologic parameters that can be accessed by the user. By using a joystick, XVR allows the user to walk around in the simulated reality of an incident. By clicking on the virtual victims, the Triage Card (Figure 1) appears on the screen. This card displays the victim’s clinical condition with parameters, and allows the assignment of a triage color using the joystick.

Figure 1 A screenshot of the virtual reality scenario environment with an example of Triage Card visualized during the simulation.

To create the study scenario, ten real patient scenarios with known outcomes were taken from the database of over 100 case descriptions from the 2008 train accident in Chatsworth, Los Angeles, and the triage cards of the virtual victims were created (Table 1). For each victim, real clinical conditions, including general victim data, such as sex, age, nature of injury, and physiologic parameters, were imported into the software. These parameters included respiratory rate, capillary refill time, radial pulse, level of consciousness in terms of ability to follow simple commands, and the ability to walk. The patients were distributed in a realistic fashion based on real life locations from the Chatsworth train crash.

Table 1 Patient parameters with known outcomes from a train accident in Chartsworth, Los Angeles, California, on September 12, 2008

Abbreviations: Amb=ambulatory; CRT=capillary refill time (seconds); HR=heart rate (beats/minute); RR=respiratory rate (breaths/minute); N=no; Y=yes.

Study flow

The study was carried out over a 1-day period. Following an explanation of the study, informed consent was provided by the 26 advance care paramedic students. No volunteers out of the 26 withdrew from the study. The volunteers were randomized into two groups; 50% were randomized to use the STM triage method and 50% were randomized to use the START triage method.

The groups were provided a 30-minute lecture on their assigned triage method. Four computer stations, each loaded with the XVR training software, were set up in one large room. Study participants were given a 30-minute orientation session on how to interact with the XVR virtual reality environment. This consisted of a three-person scenario allowing the participant to become familiar with the computer simulation software, and to practice how to use the mechanics of the program. Included were how to: assign triage scores using the joystick, open car doors, and enter a bus in the 3D environment. The orientation scenario was the same for both groups. After fielding any questions from the study participants with regards to the mechanics and use of the program, the study was started. The volunteers were told prior to the start of the simulation that there were 10 casualties. The order of triage and the approach to the accident scene were left up to the study participants. All patient triage maneuvers were done in real time.

Study participants were able to request assistance from an evaluator who was co-located in the workstation with them. If there was an issue with the program or the participant had questions about the mechanics of the simulation, an expert on XVR was available in the room.

Each participant entered the VR environment and was directed to an artificially intelligent, computer-generated firefighter where the first casualty was located. Having already been informed that there were 10 causalities, and that the scene was safe, the participant user approached the scene and triaged each patient.

Measurements

The primary outcome measures were the differences in time to triage and the differences in triage order.

Using a data collection tool, the participant was timed using a stopwatch from the start of the scenario until the last patient was triaged. This time was recorded. The participant then communicated which order they wanted the patient evacuated from the scene.

Data analysis

Statistical analysis used a two sample t test to compare the time to triage. Differences in victim rank order between the two groups were assessed statistically by a nonparametric permutation test based on the sum of squared deviations between average rank order across victims. A rank-ordered logistic regression model analysis was performed in Stata software (Statistical Software, College Station, TX: StataCorp LP, 2013) to explore whether specific victim characteristics were related to differences between the two groups. Generally statistical tests had a two-tailed alternative and were interpreted at a significance level of p=0.05.

Results

Twenty-six students agreed to participate in the study (2 groups of 13 students each). All 26 were present for their respective triage method lecture, the orientation session, and the study scenario.

The mean total triage times per individuals in groups STM and START were 709 seconds (11 min, 49 sec) and 609 seconds (10 min, 9 sec), respectively, corresponding to a mean difference of 100 seconds (95% CI: −11 sec, 211 sec). The difference was not statistically significant (p=0.07).

Statistical analysis by a nonparametric permutation test based on 9,999 simulations shows a significant (p=0.008) difference between victim orders in the two groups. The statistical significance is essentially driven by victim 538, who was given higher priority by the START method; the difference disappeared when this victim was removed from the ordering (Figure 2). This victim was located in the train and had a left ankle fracture. The difference between the two groups was not significantly linked to any physiologic victim characteristics (airway, respiratory rate, heart rate, ambulatory) when assessed by a rank-ordered logistic regression model.

Figure 2 Boxplot of the victim ordering in the two groups, STM and START. This figure also shows the distribution of order scores for each victim within each of the two groups. The statistical significance is essentially driven by Victim ID 538, and disappears when this victim is removed from the ordering. The difference between the two groups is not significantly linked to any physiological characteristics. The asterisks indicate outliers.

Discussion

Although STM has been marketed as a mathematical model that orders the treatment of patients based on their probability of survival, potential for deterioration, and available resources, ^{12 , 13} we wanted to look at time to triage and triage order to compare the two methods.

This study showed no statistically significant difference in the time to triage between the STM and START methods. The average time to triage was 100 seconds longer by the STM method. In this study, we believe that 100 seconds is not a clinically significant time difference between these two triage methods, based on this type of MCI.

According to our results, there was a difference in the order of triage. This was statistically driven by victim number 538, who was located in the train with a left ankle fracture. The statistical difference disappeared when the victim was removed from the ordering. It is of interest to note that the triage order was not significantly linked to any of the provided physiologic characteristics. START triage takes into account the mechanical act of ambulation, whereas STM does not. This may have had an impact with regards to the ordering involving victim 538.

Previous studies have looked at the operational viability of the STM method compared with the START method. ^{4 , 6 , 8} According to Navin and co-authors, emergency responders’ STM assessments were more accurate, and the STM time to complete the triage assessments and to clear the scene was less. ⁴ According to their results, emergency responders did not implement START successfully. Navin et al. also stated that STM out-performed START in all objectives. ⁴ The study arms examined were not completed during the same time of day, with START being done first and STM second. The study’s physiologic parameters were also artificial, as they were read from patient profile cards. There was also the question of conflict of interest, in that at least one of the authors was involved with the company that sells the STM method. The present study looked at the time to triage and triage ranking order. No significant difference was found for time to triage, while a difference was found in triage ranking order. This difference was due to one patient scenario and not related to patient physiologic characteristics.

STM relies on resource allocation as part of outcome measures and the order of triage, and therefore the order of transport to definitive care. If one considers the medical assistance chain as having three parts, with the first being triage capacity, the second being transport capacity, and the third being hospital treatment capacity, it can be argued that triage capacity is the more important of the three. With no difference demonstrated between both methods with regards to triage outcome measures used for this study, the relative cost of the STM may justify using START for emergency medical service systems with limited funding.

Unlike other aspects of paramedic training in which exposure to real situations during clinical rotations is the primary method of teaching and a way of comparing processes, MCI training does not lend itself to such methods. This makes it difficult to study objective differences in triage methods. Simulation presents a practical way to study and evaluate different triage methods in a MCI setting, allowing for comparisons of simple outcome measures.

Limitations

There are design limitations that should be considered when interpreting the results of this study. The STM triage method uses physiologic parameters, as well as resources available, to look at the probability of survival. This study looked at the “front end” of the MCI event, focusing more on the time it takes to triage and the initial ranking order of evacuation. Although 100 seconds may not be of clinical significance for a difference in time to triage ten victims, a larger study with a greater number of casualties may demonstrate a difference in time that would be considered clinically significant in a MCI event. An important limiting factor for this study was the small number of study participants. We employed a convenience sample of paramedics without a prior sample size calculation based on a clinical difference in the primary outcome measure. A repeat of this study with more participants may have revealed differences that went undetected in this study. Finally, simulation was done in an environment that is different from the actual clinical setting, which may have distractors and other factors that are not represented by simulation formats.

Conclusion

This study demonstrated that there was no statistically significant difference in time to triage by advance care paramedic students using the STM or the START triage methods in a simulated MCI scenario based on a train crash. While a difference was found for ranking order for priority to triage, this difference was due to one of 10 victims, and not based on the victims’ physiologic parameters. This study was significantly limited by a sample of convenience and the research question merits further evaluation by employing a larger sample size to verify the results.