Mass casualty (no-notice) events (MCE) continue to increase in magnitude and severity both nationally and worldwide. Reference Waxman, Chan and Pillemer1 Data from the Federal Bureau of Investigation (FBI) between 2000 and 2013 showed an increasing frequency of active shooter incidents annually. In 2019 alone, the FBI identified a total of 28 active shooter incidents with a total of 247 casualties. 2 Specific to pediatrics, the 2018 Parkland School Shooting is considered the deadliest school shooting in US history followed by the 2012 Sandy Hook School Shooting. 3 Given the increasing frequency and unexpectedness of such no-notice events, health-care systems and the communities they serve need to be prepared to respond with efficiency and effectiveness.
Lessons learned from previous events show that prehospital systems may not be able to assess, triage, and treat all casualties. 4 Additionally, given the uncertain safety of the affected area, survivors in unstable clinical conditions may use any means possible to get to the nearest hospital. During these “high stake, low frequency events,” emergency department (ED) staff will need to shift from routine triage to disaster triage in which they must provide the greatest good for the greatest number of victims. This requires simultaneous communication, within and between departments and teams, while managing large numbers of trauma patients, some of which may be critically ill. Reference Tenenbein, Macias and Sharieff5,Reference Walsh, Subbarao and Gebbie6
Joint commission requires hospitals to perform 2 disaster drills annually, 1 full-scale community-based exercise or facility-based functional exercise, and another that can be a tabletop exercise or workshop that requires the activation of the hospital’s emergency plan. 7 With a variety of disaster drills being conducted, it is unknown which activities provide the richest educational experience for the providers involved. While there is consensus on health-care provider core competencies for disaster training, Reference Walsh, Subbarao and Gebbie6 there is limited literature regarding optimal training methodologies for pediatric disasters. Reference Bank and Khalil8–Reference Behar, Upperman and Ramirez10 Moreover, health-care providers have a constant educational need to maintain proficiency in their disciplines in general, leaving limited availability and restricted prioritization for disaster educational opportunities.
The purpose of this project is to describe a pediatric mass casualty disaster simulation curriculum for ED staff and its effectiveness for participants. Prior MCE training at the study institution consisted of biannual MCE tabletop drills as well as monthly, brief, no-notice 10-min in situ ED exercises focusing on initial steps during an MCE. Despite best attempts at scheduling, not all ED staff experience the no-notice exercises; warranting a more formal curriculum for staff. Key learning concepts of this curriculum were to teach initial triage, prioritization within individual critical patient care, prioritization of multiple critical care patients, and ideal interdepartmental communication in an MCE. The goal was to design a curriculum that was feasible in both duration and resource demand, so as to be repeatable on a semi-annual basis.
Methods
Study Design
This was a prospective cohort study of an MCE in situ simulation program conducted from June 2019 to October 2019. Participants completed pretest and post-test evaluations that contained knowledge-based questions and a self-assessment of preparedness for a mass casualty event. This study was deemed exempt by the Institutional Review Board of Boston Children’s Hospital.
Study Setting and Participants
The MCE simulation program included 1 pilot testing session and 4 separate 2-h sessions at an urban pediatric tertiary care Level 1 pediatric trauma hospital with an annual volume of 60,000 visits. All ED staff physicians (MDs) were required to attend. ED nursing, administrative staff, pharmacists, physician’s assistants (PAs), and clinical assistants were invited to participate. In addition, representatives from anesthesia, trauma, and intensive care units as well as hospital incident command system leaders were also invited to participate.
Development of the Mass Casualty Simulation
Educational Goals
The primary goal was to provide an experiential educational MCE experience where participants learn, reinforce, and practice MCE principles and MCE department specific protocols. The learning objectives for participants were to:
-
1. Demonstrate the ability to function in the triage role and classify patients quickly into the following categories: black (expectant), red (critically ill), yellow (urgent), and green (well).
-
2. Describe mass casualty principles in an MCE simulated setting, including: logistics of patient movement, aligning resources, anticipating resources, and communication with multiple teams
-
3. Recognize critically ill patient care priorities and practice implementation of these priorities in a resource limited MCE setting
-
4. Identify the needs of less acute patients and practice implementation with limited resources
-
5. Discuss communication/coordination challenges inherent to mass casualty situations between teams and disaster team leaders and develop solutions to optimize interactions and expedite quality care.
The secondary goal was to gain feedback from participants to further develop and enhance MCE department specific protocols and procedures.
Simulation Description
The MCE simulation scenario was an explosion at a school with a total of 25 pediatric patients (9 green patients, 10 yellow patients, 4 red patients, 2 expectant patients) coming to the ED. Before the simulation, a 10-min didactic presentation was given to review basic disaster triage and MCE principles. The simulation program was performed in the ED during normal operations to better emulate a real event where continued care for non-MCE patients would need to continue while simultaneously caring for MCE patients. The simulation was designed so that it could be easily repeated multiple times to ensure all ED staff could participate. As such, the goal was an exercise that would occupy the ED space for less than 2 h. Participants rotated through 3 modules during the simulation: (1) the triage exercise; (2) caring for a critical (red) patient during the 15-min MCE exercise; and (3) being in a disaster leadership role during the 15-min MCE (Figure 1). For the triage exercise, participants gathered in a separate conference room and were given a set of patient cards containing basic patient information (vital signs and physical exam findings). Participants were then assigned to ED MD/PA provider-nurse pairs as per the study site’s MCE triage protocol. The pair were asked to triage the patients as expectant, red, yellow, or green. For the 15-min MCE exercise, participants were told that the exercise starts after patients had been triaged and placed into patient care areas and to proceed by providing patient care. Four red patients were simulated with high fidelity simulation manikins in 4 separate patient rooms that were operated by medical simulation technicians allowing for real-time adjustment and feedback. Yellow and green patients were simulated using low fidelity Thomas MCE PedTM manikins placed in improvised patient care areas of the ED, as would occur per MCE protocols if limited patient rooms were available. The MCE exercise itself runs for 15 min and was repeated 3 times. Upon conclusion, all staff participated in a 25-min debrief. Total simulation time was 2 h.
Figure 1. Flow sheet of MCI simulation day.
Curriculum Development
A multidisciplinary team consisting of emergency medicine MDs, nurses, administrative staff, and pharmacists as well as hospital emergency management personnel designed the curriculum over a 6-mo period. Patient scenarios were adapted from the Baylor MCE program, which was developed by means of consensus among 3 pediatric emergency medicine MDs with disaster medicine training and experience. Curriculum was developed with a focus on emphasizing the MCE specific challenges for a disaster team leader including resource prioritization in a relatively resource limited situation and rapid communication with multiple teams. For MDs and nurses, the role of the disaster team leader is unique to MCE activations entailing disaster triage skills and communication that differs from routine ED care.
Simulation facilitators were solicited among experienced simulation faculty. There was 1 facilitator assigned to observe each of the following: all 4 red team patient care rooms, green team, yellow team, the disaster team leader, and to run and observe the triage exercise. A training session was performed for all facilitators to review learning objectives and logistics.
The simulation program was initially piloted with pediatric emergency medicine fellows. The initial layout of the sessions had been to have a brief debrief after each cycle but based on participant feedback from the pilot group that it was found to be difficult to debrief without having participated in all roles. Thus, the session layout was adjusted to have 1 large overall debrief at the end of the session to allow participants to rotate through all roles before debrief.
Measurements
Triage accuracy, knowledge, and self-evaluation of preparedness and MCE skills were measured. Participant pairs’ triaging assessments of 30 MCE patients were recorded on paper by participants and scored by 1 of the study authors (D.D.) for total correct answers, number of under triaged patients, and number of over triaged patients. ED MD/PA providers and ED nursing participants were given a knowledge test before the didactic (ie, pretest) and after the debrief (ie, post-test). The pre- and post-knowledge tests contained the same 14 multiple choice questions that were developed by 2 of the study authors (J.L., S.C.) who are board certified pediatric emergency medicine MDs with a background in disaster medicine. They were piloted for readability, clarity, and learner comprehension by emergency management staff and 2 of the study authors (D.D., C.P.). After the debrief, participants were asked, in the setting of a future MCE, to rank 2 statements regarding their self-assessment of their disaster preparedness to efficiently triage and prioritize resources before and after the simulation session. The participants were asked to rank these statements on a 5-point Likert scale (5 = very prepared, 4 = moderately prepared, 3 = somewhat prepared, 2 = minimally prepared, 1 = not at all prepared). In the post-test, participants were asked open-ended questions about their successes and challenges from the session and what could be done to further enhance their skills and abilities to prepare for future MCE.
Data Analysis
Participant demographic data are reported in terms of frequencies and proportions. Median (with interquartile range [IQR]) number of correctly triaged patients, under-triaged patients, and over-triaged patients is reported. Over-triaged is defined as a participant assigning a higher acuity level than the team had assigned. Under-triaged is defined as a participant assigning a lower acuity level than the team had assigned. Knowledge scores are presented as median scores with IQRs. Wilcoxon matched pairs signed rank test were performed to compare pre- and post-test knowledge scores, and the change in knowledge score between those with and without real-life prior disaster experience. Each individual item from the knowledge test between pre- and post-test assessments and self-assessment of preparedness in MCE triage and resource use were compared (comparing not at all/minimally prepared to somewhat/moderately/very prepared) using McNemar’s chi-squared test. Comments from the open-ended questions in post-test evaluations were reviewed and grouped into themes by 22 of the study authors (J.L., S.C.) and reviewed by the remaining authors.
Results
A total of 46 ED MDs, 1 PA, and 22 ED nurses participated over the 4 separate simulation days. As shown in Table 1, 65% of ED MD/PA providers and 73% of ED nurses have over 5 y of ED experience, with the majority having no prior mass casualty experience (61% of ED MD/PA providers, 77% of ED nurses). The majority of ED MDs have had some form of prior mass casualty training (72%) but only 32% of ED nurses had prior mass casualty training. All ED MDs and PAs completed the pretest and 98% completed the post-test. All ED nurses completed both the pretest and post-test.
Table 1. Demographics
Among ED MDs and PAs, there was a statistically significant increase in knowledge test scores (median difference, 0.07; (P < 0.001) as illustrated in Table 2 and Figure 2. ED nurses did not show a statistically significant difference in their knowledge after the simulation (P = 0.054; Table 2 and Figure 3), and no change in pre-/post-test scores was found among nurses with and without prior MCE experience (Table 3). There was a statistically significant improvement in 2 questions about triage and 2 questions about communication principles in MCE in the post-test compared with the pretest in the ED MD/PA provider group, while only 1 triage question showed improvement in the ED nursing group (Supplementray Materials Table 1). With regard to the triage exercise, the ED MD/PA provider-nursing pairs had a median correct score of 73% (IQR, 70%, 80%) with a median of only 2 patients being over-triaged (IQR, 1, 3) and median of 0 patients being under-triaged (IQR, 0, 1). There was a statistically significant increase in MCE triage and resource use preparedness in ED MD/PA providers and ED nurses (P < 0.01; Table 4).
Figure 2. Median pre-test and post-test overall scores of ED provider knowledge.
Figure 3. Median pre-test and post-test overall scores of ED nursing knowledge.
Table 2. Overall knowledge scores
Table 3. Comparative knowledge scores based on prior disaster experience
* P-Values from Wilcoxon signed rank test
Table 4. Self-assessment in preparedness before and after exercise
Upon review of comments to open ended questions for the ED MDs and PAs, the most common themes with regard to how the simulation was a success for them were: (1) improved knowledge of hospital incident command structure and the chance to practice communication in an MCE, (2) the opportunity to practice all different roles in an MCE, and (3) the opportunity to practice triage skills. The most common themes regarding potential MCE challenges were (1) struggling with limited resources both in terms of staff and actual equipment, (2) difficulty with defined communication roles in MCE that differ from typical routes of communication, (3) overall struggles with logistic organization. ED MDs and PAs suggested that more practice/educational opportunities and better clarification of communication roles of personnel outside of the ED would best improve MCE capabilities. ED nursing reported similar success themes in successes including practicing teamwork communication, practicing triage, and practicing skills not normally practiced by nurses in typical care (eg, needle decompression). ED nursing reported struggling with being flexible during chaotic situations, understanding the disaster nursing leader role, and struggling with communication in a chaotic situation. Last, ED nursing also echoed that more practice/educational opportunities would be important for improving MCE capabilities as well as the need for more organizational tools.
Discussion
This MCE curriculum represents a feasible and scalable method to implementing a multidisciplinary, in situ ED simulation response to a large-scale MCE during which participants practice MCE triage, management, and interdisciplinary communication. Insightful information about competency, comfort, perceived challenges, and future needs related to MCE response and training was gained. Unlike previously described simulations, Reference Bank and Khalil8 the entire care team of ED MD/PA providers, nurses, pharmacists, and clinical assistants were included and able to practice in the actual ED setting with actual equipment. While most MCE simulations focus on the initial surge of patients and medical management, this simulation was designed to allow participants to practice communication within the medical teams and with hospital incident command leadership while also prioritizing medical care for the critically ill patients. The simulation ran over 2 h and exposed each participant to multiple roles including triage, disaster team leader roles, and resource-limited acute care roles. There was a statistically significant increase in ED MD/PA provider knowledge and was universally well received by all participants who requested repeat similar trainings.
Scope, Participants, and Design
Prior reports have reinforced the value of simulation exercises as an effective means to both train for and assess local MCE response. Reference Waxman, Chan and Pillemer1,Reference Hsu, Jenckes and Catlett11–Reference Cocco and Thomas-Boaz14 High fidelity mannequins versus actors were chosen based on prior reports showing no appreciable difference in realism effect and notable reduced cost and simplified planning. Reference Gillett, Peckler and Sinert15 Inclusion of multidisciplinary staff in a pediatric hospital based simulation expanded on the work of Bank and Khalil who limited participants to MD staff and was not able to assess interdepartmental communication. Reference Bank and Khalil8 Similarly, McElroy et al. Reference McElroy, Steinberg and Keller16 and Chuang et al. Reference Chuang, Woods and Ting17 reported on the challenges of interdepartmental coordination and communication amidst the limited resources of an MCE event. Learning from local experiences of the Boston Marathon bombing MCE, pharmacy and administrative (registration) staff were specifically included in the exercise to both educate and assess their function during an MCE response. Reference Landman, Teich and Pruitt18 Finally, the modular drill and evaluation design adopted lends itself to continuous quality improvement proposed by Hsu et al. Reference Hsu, Jenckes and Catlett11 This enabled both operationalized exposure and debriefing the participants experience in triage, team leader, and acute care roles in 1 efficient exercise. Individual learning and comfort with disaster response, as well as, understanding the skills competency and system challenges would contribute to future improvement.
Lessons Learned
The study ascertained the staff’s strengths and challenges in an MCE response by means of assessments of triage accuracy, knowledge testing, and self-evaluation of preparedness. Most of the participants in this exercise had greater than 5 y of experience in the ED with an anticipated lack of exposure to an actual MCE experience. This underscores both the need for continuous offerings and the need to include the various disciplines needed for an effective response. Multidisciplinary drills are reported to have led to effective real event responses in Colorado, Orlando, and Toronto. Reference Cocco and Thomas-Boaz14,Reference Authors19,Reference Albert and Bullard20 The goal was to create an exercise that was more feasible to enable all ED MDs to be able to attend, include multiple disciplines, and would be able to be repeated on a more regular basis to expand education and inform improvement.
The triage MD/PA provider-nurse pairs performed well with a high rate of correct classification (73%), and low rates of over-triage (3%) and under-triage (1%). Vargas et al. Reference Vargas, Hubloue and Pinzon21 reported similar triage accuracy among ED providers with greater than 5 y of ED experience after a brief training in MCE triage at 67.66%, but higher rates at over-triage (15.19%) and under-triage (17.14%). Additionally, other studies have also found increased rates of both over and under triage of pediatric MCE patients. Reference McElroy, Steinberg and Keller16,Reference Heffernan, Lerner and McKee22 The high rate of accurate triage is likely due to extensive experience specifically triaging pediatric ED patients combined with the participant’s exposure to regularly scheduled tabletop and no-notice in situ triage drills and the pre-exercise education on triage and MCE skills.
The objective knowledge assessment for both triage and MCE communication skills was assessed by a pre- and post-knowledge test. ED MD/PA providers had a statistically significant increase in knowledge regardless of prior MCE experience, whereas, ED nurses did not show an increase in knowledge score irrespective of prior MCE experience. However, nurses demonstrated a more uniform and higher baseline overall knowledge score. The ED MD/PA provider knowledge improvement primarily were based on improved scores in triage and Disaster Team Leader communication skills, which reached statistical significance and reflects a learning curve associated with the limited prior triage experience of MDs and the value of deliberate practice in both the triage and team leader role for MDs. The study did not demonstrate a similar change in nursing knowledge. This is likely due to the fact that nurses have more regular exposure and practice in triage skills, which composed a large part of the knowledge assessment.
Initial plans were to attempt to assess specific MCE skills completion by the use of dedicated evaluators and a checklist. Unfortunately, there was poor completion rate of the evaluation tool prohibiting analysis. The low rate of completion is likely due to the fact that there were only single drill facilitators to run each simulation in the red patient rooms and with the disaster team leaders. While there were attempts to apply Jencke’s principles of evaluation tool development, Reference Jenckes, Catlett and Hsu23 the balance of documenting time stamps and task completion may have reflected the difficulty of the dual role. A deficiency of evaluator training or the lack of a user-friendly tool may also have contributed to the low completion rate. However, the group debrief around MCE communication skills involving participant and evaluator observations was helpful to identify successes, challenges, and areas for improvement. Evaluators provided insightful observations to the overall debrief. Splitting the role of the simulator/evaluator, further training of evaluators and simplification of the tool have been identified as ways to improve the exercise and its evaluation. This essential goal of a uniform method of evaluation of the exercise for continuous improvement of MCE response has been highlighted as an ongoing challenge and requirement if disaster drills are to be used to assess capabilities. Reference Hsu, Jenckes and Catlett11
Similar to most reports from other disaster drill curriculums, the participants self-reported improved preparedness for MCE response from somewhat to moderately prepared. With goals to emphasize triage, interdisciplinary communication skills, and provision and coordination of resource limited acute care, the open-ended assessment responses highlighted the simulation’s value to staff in exposure to multiple roles in an MCE, understanding incident command MCE role, practicing interdisciplinary communication, and practicing triage skills. Challenges identified by participants reflected previously described needs to reframe care and communication when resources of staff, equipment, and services are limited or overwhelmed. Reference Hsu, Jenckes and Catlett11,Reference Chuang, Woods and Ting17 Generally, the study site staffing for critical care patients is abundant. However, in an MCE, it is far more likely that individual critical care patients will each have limited staff. This exercise allowed ED MD/PA providers to experience this limited staffing, which participants described as stressful and unanticipated, in a safe environment. Staff uniformly welcomed more practice and educational opportunities and endorsed this simulation as an effective educational opportunity. Other disciplines including pharmacy and administration were also grateful for the opportunity to participate. For example, the ED administration staff used a previously developed MCE registration system where the unique disaster patient identification system for MCE patients were preassigned numbers. The drill elucidated that these numbers for patient identification in MCE were often confused with patient bed spaces as the number systems overlapped (eg, patient 2 in room 1). This created communication challenges, which led to a revision of the MCE registration system to use numbers that did not overlap with patient bed spaces (eg, the highest room number in the ED is 52 so MCE patient identifiers now start at 60).
Limitations
There are some limitations in this study. First, the exercise was conducted in a single center, which limits its generalizability. However, the goal in design was to create an exercise that focuses on using in-department personnel and space with a limited time course to maximize the efficiency of these exercises. Additionally, the simulation also ran over several dates, which demonstrates its replicability. Second, while other departments (anesthesia, operating room, trauma surgery) were invited, they did not routinely participate. When hospital incident command leaders participated, they were able to observe the exercise to understand their roles in both disaster response and training needs. Future iterations of this curriculum will expand to include these other departments and administrators more routinely in these drills. Third, as this was an educational pilot, we did not have a formal control group but our institution has held other tabletop and short drills prior with participants commenting that this was a preferred exercise. Last, there are no universal validated tools for evaluating MCE exercises given the heterogeneity of exercises available. However, tool creation was based on the current literature available regarding MCE training exercises.
Conclusions
This study illustrates the development of an efficient, feasible model for a multidisciplinary ED disaster drill that provides a multi-modular, deliberate practice exposure to triage skill, MCE interdisciplinary communication, and resource-limited acute care skills. This curriculum when offered continuously will both train for and inform improvement in MCE responses.
Supplementary material
For supplementary material accompanying this paper visit https://doi.org/10.1017/dmp.2022.42
Acknowledgements
The study team thanks the following for their assistance in this project: Mary Devine MPH, Michelle Carestia, MPA, Marisa Brett-Fleegler, MD, Karen Dull MD, Kate Dorney MD, Alexander Hirsch MD, Mark Waltzman, MD, and the Boston Children’s Hospital SIMPEDS Simulator Program.
Author Contributions
J.L., B.K., C.P., and S.C. conceived the study and designed the curriculum. J.L., C.P., D.D., and S.C. supervised and coordinated the simulation and data collection. M.M. provided statistical advice. J.L. and S.C. performed data management and quality checks. J.L. and M.M. analyzed the data. J.L. drafted the manuscript, and all authors contributed substantially to its revision. J.L. takes responsibility for the study as a whole.
Funding
None.
Conflict(s) of interest
None.
