Learning From Simulating Mass Casualty Events: A Systematic Search and a Comprehensive Qualitative Review

Abstract

Mass casualty events (MCE) strain available health-care resources requiring extraordinary measures. Simulated exercises are used to improve preparedness. We sought to identify learning points and common themes arising from such exercises in literature. Reporting of action points to improve response plans were investigated. Type of exercises, environments, and departments were also explored. We systematically searched 3 databases and applied our eligibility criteria. Inclusion criteria were in-situ MCE simulations of clinical response to traumatic MCEs, including scene management, prehospital care, and in hospital care. Exclusion criteria were nonmedical response, infectious outbreaks, training courses with self-selecting participants, simulations assessing mechanical tools, and mathematical modeling. A total of 6883 titles were identified and screened. Eighty-three studies were read in full. Twenty-two articles were included. We identified numerous learning points, which were collated and categorized into 11 themes. Fifty-nine percent of the papers reported actions that would be or had been implemented. MCE simulation exercises have been found to improve familiarity and confidence among participants. The 11 themes identified from published exercises overlap with areas of improvement from real events. MCE simulations in the literature appear to focus on carrying out the exercise itself rather than learning points possibly missing opportunities to improve response plans.

Improving disaster preparedness is a United Nations (UN) priority as stated in the Sendai Framework for Disaster Risk Reduction 2015-2030. ¹ The UN focus for improving patient outcomes is to: “…train the existing workforce in disaster response”, “…ensure better response in emergencies” & “…promote regular disaster preparedness, response and recovery exercises …” ¹

Mass casualty events (MCE) cause significant disruption to health-care facilities, stressing both staff and resources due to their magnitude, unpredictability, and sudden onset. In the United Kingdom, the Civil Contingencies Act (2004) requires NHS organizations to show that they can effectively manage such incidents. ² At a local level, each hospital must have a plan for how to respond to an MCE and then must ensure that the staff involved have read it and understood their roles. A simulation exercise is useful in identifying any areas of weakness in translating the plan into action and to allow staff to familiarize with local protocols and understand their role. Globally, simulations take place in and outside the hospital and can involve any or all the emergency services. ^3–5 Learning points should be identified from such exercises and action points implemented to improve response.

We conducted a systematic search of the literature pertaining to mass casualty preparedness. The primary aim of this review was to identify mass casualty simulation learning points reported in literature and to establish the common themes among these learning points. Secondary aims were to establish type of exercises, locations, and departments favored for such exercises and whether lessons learned have led to modification of the institution’s major incident plan. The possibility of MCEs leading to an increase in MCE exercise publications was also investigated.

Methods

A systematic search of the literature regarding simulation in MCE preparedness was carried out. Three databases were used for the search (PubMed, Cochrane Library, and Embase), as well as grey literature and reference lists from identified articles. The search was carried out between September 2, 2019, and September 13, 2019. Inclusion and exclusion criteria are described in Table 1.

Table 1. Search inclusion and exclusion criteria

Literature that demonstrated major incident or mass casualty incident simulations were included. We excluded training courses as they self-select for interested individuals and do not test the workplace. We focused on multiple traumas and chemical, biological, nuclear, and high-yield explosives (CBRNE) scenarios and excluded infectious outbreaks due to the protracted nature of epidemics.

Prehospital simulations of major incidents were compared with in-hospital simulations. Surge capacity modeling and theoretical work were excluded as we sought the human elements in responding to a major incident. We excluded all undergraduate simulations as their focus was medical education. Similarly, simulations carried out to assess the use of mechanical tools were excluded because their focus was on the tool itself.

Some studies focused on the mechanisms of how to run a simulation in comparison with assessing preparedness or departmental training. We excluded such studies as they did not examine a health-care response.

Recent major incidents including terror attacks may have influenced the direction of research into major incident response. Thus, we included data from the past 30 years, looking from January 1, 1989, to September 13, 2019.

The following search strategy was applied to all 3 databases (Figure 1). All identified studies were exported into Mendeley Desktop (Mendeley Ltd, London, UK). Study selection was carried out through screening titles, removal of duplicates, and exclusion of papers unrelated to our study aims. Following this, all remaining abstracts were reviewed, and full text articles selected for formal assessment against the eligibility criteria. A second independent review was also completed to reduce selection bias.

Figure 1. Search terms applied across the three reviewed databases.

Data collection was performed in Microsoft Excel (Microsoft Corporation, Redmond, WA) using the headings provided in Table 2. Studies were separated into type of scenario, participating departments, and number of departments involved. Learning points from exercises were collated, individually considered and assigned a theme of best fit. Selected articles were also screened for randomized controlled trials. The limitations of all the studies within the inclusion criteria were noted to anticipate possible bias effects.

Table 2. Data extracted from selected literature

Results

Following the search, 6883 titles were identified and 22 articles were included. ^3,5–25 None of the cross-matched references met the eligibility criteria; thus, no further papers were included (Figure 2). No randomized controlled trials were identified.

Figure 2. Flow chart of the search strategy and study selection process.

The primary aim was to identify mass casualty simulation learning points reported in the literature and to establish the common themes among these learning points. In the 22 studies, learning points were established through third party evaluators, videography, structured 1-to-1 interviews, evaluation questionnaires, or debrief meetings. Two studies, Petinaux et al. and McElroy et al., reported use of structured debrief tools such as After Action Review. The established learning points are summarized in Appendix 1. Eleven themes were identified from the 22 studies. In the order of highest to lowest number of learning points reported, these themes are:

1. 1. Resource management or use: Lack of equipment and personnel for patient transfer and patient care were identified in the prehospital simulations. Limited use of MCE plans was reported. Early dispatch of ambulances and helicopter and emergency medicine services were highlighted as beneficial.

2. 2. Communication: Communication breakdowns affected victim triage, distribution, treatment and decontamination, and staff availability. Staff training on using communication systems was a limiting factor.

3. 3. Preparedness, learning, and research: Lower performance was seen as a consequence of lacking comprehensive response planning. Lack of knowledge of response plans identified need for further staff education. Postexercise debriefs were identified as extremely valuable for improvement.

4. 4. Medical care: Gaps in knowledge and areas for service improvement were identified such as ocular care, managing those with immunosuppression, and caring for the deceased.

5. 5. Role identification: Organizations need to assign roles to individuals with clear outline of responsibilities in an MCE.

6. 6. Patient flow: Bottlenecks should be anticipated. Emergencies should be dealt with and patients should be moved to their destinations as soon as possible to avoid bottlenecks. Creation of an ambulatory unit that dealt with the walking wounded and bypassed the Emergency Department was one novel approach.

7. 7. Triage: Allocation of triage officer early on, reminder to use an age-appropriate triage tool and possible deterioration of the walking wounded with anticipation of change in triage status were identified.

8. 8. Know the environment: It is important to practice every step of performance in a simulation as if it is a real MCE. If one only pretends to get the key to a cupboard or pretend to turn the water on for a decontamination tent, one may find they do not know where the keys are or how to turn the decontamination tent water on in a real event. ⁵

9. 9. Performance evaluation: Evaluating individuals’ performance in an MCE simulation lacked standardization through the exercises. A standardized centralized training and knowledge evaluation system to aid continuous feedback and service improvement has been suggested.

10. 10. Friends and family: Re-unification of loved ones, grief counseling, and information delivery to loved ones appear to be an afterthought in MCE response planning.

11. 11. Crowd control: Relatives, media, and volunteers can create a security issue and need resources allocated to their care of containment, which was an aspect of response that was found to be lacking.

Second, 59% (13) of papers included action points implemented or stated action points to be implemented (Figure 3G). These action points included colour coding of clothing for role clarification, specific MCE orientation or creation of a template that would aid patient tracking. Some papers stated implementation of action points but did not specify the actions. The remaining 41% (9) of papers did not report any action points to be implemented for improving response.

Figure 3. A, The years in which the included papers were published. B, The number of different types of simulations carried out in the included articles. C, The number of hospital or prehospital exercises carried out. D, The number of departments involved in the simulation. E, Types of scenarios involved. *This scenario was an explosion leading to a train derailment and a chemical spill. F, The number of different focuses each paper had. G, The number of papers that (Y) stated action plans have been implemented or suggested action points that will be implemented and (NA) did not report any action points.

Three types of simulation exercises were reported. Nineteen (86.4%) were live exercises, 2 (9.1%) were tabletop (9.1%), and 1 (4.5%) was a virtual reality exercise (Figure 3B). Prehospital environments were hypothesized as more abundant; however, there were 12 (54.5%) in hospital simulations, 2 (9.1) prehospital simulations, and 8 (36.4) were both (Figure 3C).

Forty-five percent (10) of the exercises reported involved 1 department alone and 41% (9) were multi agency exercises. The remaining were 2 or 3 department exercises (Figure 3D). Sixty percent (6) of single-department simulations were based in the Emergency Department. Fifty percent (11) of all scenarios involved CBRNE (Figure 3E). Others included road traffic accidents (RTA), plane crashes, and nonspecific mass casualty incidents.

The common limitations of the studies included artificiality associated with simulations, lack of cost effectiveness of simulation training, lack of generalizability and risks associated with self-reported data collection. Risk of bias within studies included reporting bias as a subset of individuals respond to postexercise questionnaires. There was also reported lack of standardization in performance evaluation.

Discussion

Eleven themes were identified from published learning points in literature. We have also examined types, size, and year of simulations in our data set. Transferability of learning points, implementations of actions, and the future of MCE simulation are an interest in discussion.

Learning Points and Transferability

We identified numerous reported learning points from MCE simulations that were applicable and transferrable to any hospital. Identification of the themes provides an opportunity to recommend areas where hospitals could ensure they have optimized their preparedness in responding to a major incident. This corresponds to the WHO Hospital Emergency Response Checklist. ²⁶ This supports the assertion that simulation training helps to strengthen real response when the learning points from simulation and from real response can be superimposed.

The impact of human factors is pervasive throughout these 11 themes. How we respond to an MCE relies on the system in place, the people within it, and how they interact with the system and each other. These learning points highlight system error that could be minimized if identified before the volatile environment of an MCE, as well as institutional safety that contributes to effective response. ²⁷

Implementation of Actions From Identified Learning Points

Only 59% of the papers included reported action points or stated that action points were made to act upon. One of the main purposes of major incident simulation training is to improve preparedness by identifying areas that require attention. Although there might be an element of reporting bias, the target for reporting lessons learnt and action points implemented should be 100%. The lack of reporting of learning points gained from the simulation suggests a greater focus on the exercise as an endpoint, rather than the exercise being a tool for further improvement.

Central Register of Scenarios, Performance Matrix, and Lessons Learned

Designing a robust scenario for an MCE simulation may be one barrier to considering training the hospital team in this manner. It may also be why the simulation itself is regarded as the end point because so much preparation goes into making it a robust learning opportunity for the staff involved. If a central database of patient profiles, facilitator roles, scenarios, and imaging existed, this would be a positive step toward making this form of training manageable, as most of the preparation work could simply be lifted from it. The department, hospital or agency could, therefore, focus on the human factors involved in gaining engagement from all parties before the exercise, and on implementation of the learning points gained following the exercise for their specific risks.

Types of Exercise

Live exercises are time and resource rich. For this reason, we anticipated live exercises to be less commonly reported; however, this was not the case. Among the exercises that reported learning points, there was an abundance of live exercises compared with tabletop and virtual reality simulations. In contrast, a recent survey on major trauma preparedness in MCEs involving 4 developed countries reported tabletop exercises were far more common than live exercises. ²⁸ Due to the significant amount of time and effort going into planning and executing a live exercise, there may be a publication bias toward live exercises as they may be considered more desirable for publication.

Departments Involved

Exercises can be departmental (1 department), multi-departmental (1+), or multiagency (eg, across emergency services). Our findings show highest rates as single departments in the hospital (45%) or multiagency (41%). The role of agencies such as emergency services ^a , the military, public health bodies, and charities are fundamental to MCE response and the findings are not surprising. The multiagency rates are comparable to a survey report where 52% of MCE exercises in New Zealand, Australia, England, and Canada were multi-agency exercises. ²⁸ However, the same report states 48% of in hospital exercises were multi-departmental in contrast to our findings of 14%. This could be explained by not all exercises carried out being published in literature.

If the perception is that simulation is not worthwhile unless it tests the whole response, then hospitals could miss important learning opportunities. In high-performance sports, athletes are coached to break down their activity into its component parts and examine each action in detail. ²⁹ This approach can also be applied to the component parts of a hospital’s mass casualty response: call-in, triage, restocking of supplies, phone communication, role allocation, flow through radiology, expanding intensive care, etc.

Using the Clinical Space

Training courses in mass casualty response provide excellent experience and education for the participants who attend them. However, in this review, we were interested in the learning or benefit gained for the organization, and this is greatest when the exercise is conducted within the health-care setting that would be called upon in a real event. ³⁰ A PubMed search for “in-situ simulation” and screening for team training, revealed no papers published before 2000, three papers published between 2000 and 2010, and 29 published between 2011 and 2020. This reflects how simulation training has moved from a classroom environment to being embraced in the workplace. Mass casualty preparedness exercises appear to also reflect this shift. Most papers we identified were single department exercises, although three papers examined interdepartmental teamwork. Only one mentioned “whole hospital response”. It is our belief that this is reflective of the actual proportion of exercises. The logistics of an entire hospital engaging with a mass casualty exercise while contemporaneously maintaining the standard of care to current patients precludes this scale of exercise being done frequently.

Time of Publication

One would expect a rise in the number of major incident simulations taking place following real major incidents. We identified the highest numbers of publications in 2012 and 2014 followed by 2003 and then 2019. We could not correspond these dates of publication with specific mass casualty events. However, it is evident that the number of global deaths from terrorism started rising in 2011 and more than quadrupled by 2014. ³¹

Prehospital Versus in Hospital Exercises

Prehospital simulations were expected to outnumber in-hospital exercises. There is almost unlimited space to train teams in MCE response outside, but difficult to run training exercises alongside ongoing service provision inside health-care facilities. Nonetheless, more papers described in hospital than prehospital simulation exercises. We do not believe this is a true representation of the number of exercises carried out pre and in hospital. Every year, prehospital multi-agency exercises are carried out and reported in the media, however, not always in literature. ^32–34 It appears that simulation training has become standard practice in the prehospital environment; thus, the yearly exercises are not always reported in literature. It is also possible that our searched databases did not necessarily include all reports of such exercises. It is perhaps an indication that simulation exercises to enhance mass casualty preparedness are not the standard practice within hospitals and, therefore, results in more papers written and published. This is something that we believe needs to change. If “no plan survives first contact with the enemy,” that first encounter should not be facing the real enemy.

Does Simulation Training Improve Response in Real Mass Casualty Incidents?

The ultimate aim of simulation is transferable advantage in real situations. This is almost impossible to quantify due to the unpredictable nature of mass casualty incidents. Confidence in responding to the Manchester Arena attack in May 2017 was attributed to having partaken in a recent simulated exercise, but difference in performance and contribution between the staff who did and did not attend the training remains elusive. ³⁵ However, the main themes found from our systematic review are reproduced in learning points from many real mass casualty events. ^36,37 The number of debriefs from mass casualty incidents that credit recent simulations for the cohesion of their response also gives credence to this method of training. ³⁶

Limitations

Bias in evaluation of exercises was a theme in the selected studies. Their findings were mostly self-reported by the participants. Some studies also reported response bias as only a subset of individuals responded to post-exercise questionnaires. Furthermore, there was no standardized way of evaluating performance. It is plausible that the collective learning points based on these feedbacks may have been influenced by such limitations.

It is plausible to assume variance in knowledge and in pre-exercise experiences between studies. These variances included regular training courses such as Adult Trauma Life Support® in preparation for the exercise, 1-hour pre-exercise explanation before a tabletop exercise or a simulation done twice but lessons reported together. It should be appreciated that these could have had an impact on the lessons learnt from an exercise.

The greatest limitation of our study was publication bias. ³⁸ We are aware of major incident simulations that take place annually, yet they are not published in literature. We also note high number of CBRNE scenarios reported in literature, however, they are not as commonly encountered events. It is possible that only very large and effortful exercises are considered “worthy” of publication, creating a gap in important learning points that smaller scale exercises can identify.

Conclusions

Mass casualty simulation exercises do occur in the workplace and have been found to improve familiarity and confidence among those who participate. There are recurring themes of learning points from these simulations in the literature. We have identified 11 main themes from these learning points. These themes overlap with areas of improvement from real events. Second, major incident simulations in the literature have had greater focus on carrying out the exercise itself rather than focusing on the learning points and improving response plans. The end point to simulation training must be the implementation of learning points, not just the exercise.

Based on the identified literature, we recommend future work focusing on creating a centralized register for major incident simulations where scenario design, patient profiles, and accompanying data, as well as learning objectives can be accessed by anyone wishing to use this form of training. Learning points can also be reported here and collated for collective learning. Standardization and validation of performance evaluation is another area for future work.