Exercises that simulate emergency scenarios are accepted widely as an essential component of a robust Emergency Preparedness program. Unfortunately, the variability in the quality of the exercises conducted, and the lack of standardized processes to measure performance, has limited the value of exercises in measuring preparedness.

In order to help health organizations improve the quality and standardization of the performance data they collect during simulated emergencies, a model online exercise evaluation toolkit was developed using performance measures tested in over 60 Emergency Preparedness exercises. The exercise evaluation toolkit contains three major components: (1) a database of measures that can be used to assess performance during an emergency response exercise; (2) a standardized data collection tool (form); and (3) a program that populates the data collection tool with the measures that have been selected by the user from the database. The evaluation toolkit was pilot tested from January through September 2014 in collaboration with 14 partnering organizations representing 10 public health agencies and four health care agencies from eight states across the US. Exercise planners from the partnering organizations were asked to use the toolkit for their exercise evaluation process and were interviewed to provide feedback on the use of the toolkit, the generated evaluation tool, and the usefulness of the data being gathered for the development of the exercise after-action report.

Ninety-three percent (93%) of exercise planners reported that they found the online database of performance measures appropriate for the creation of exercise evaluation forms, and they stated that they would use it again for future exercises. Seventy-two percent (72%) liked the exercise evaluation form that was generated from the toolkit, and 93% reported that the data collected by the use of the evaluation form were useful in gauging their organization’s performance during the exercise. Seventy-nine percent (79%) of exercise planners preferred the evaluation form generated by the toolkit to other forms of evaluations.

Results of this project show that users found the newly developed toolkit to be user friendly and more relevant to measurement of specific public health and health care capabilities than other tools currently available. The developed toolkit may contribute to the further advancement of developing a valid approach to exercise performance measurement.

AgboolaF , BernardD , SavoiaE , BiddingerPD . Development of an Online Toolkit for Measuring Performance in Health Emergency Response Exercises. Prehosp Disaster Med. 2015;30(5):503–508.

In Maryland, there have been no military/civilian training exercises of the Medical Mutual Aid Agreement for >20 years. The aims of this paper are to describe the National Disaster Medical System (NDMS), to coordinate military and civilian medical mutual aid in response to arrival of overseas mass casualties, and to evaluate the mass-casualty reception and bed “urge” capacity of Maryland NDMS Hospitals.

Three tabletop exercises and a functional exercise were performed using a simulated, overseas, military mass-casualty event. The first tabletop exercise was with military and civilian NMDS partners.The second tested the revised NDMS activation plan. The third exercised the Authorities of State Emergency Medical System and Walter Reed Army Medical Center Directors of Emergency Medicine over Maryland NDMS hospitals, and their Medical Mutual Aid Agreement.The functional exercise used Homeland Security Exercise Evaluation Program tools to evaluate reception, triage, staging, and transportation of 160 notional patients (including 20 live, moulaged “patients”) and one canine.

The first tabletop exercise identified deficiencies in operational protocols for military/civilian mass-casualty reception, triage, treatment, and problems with sharing a Unified Command. The second found improvements in the revised NDMS activation plan.The third informed expectations for NDMS hospitals. In the functional exercise, all notional patients were received, triaged, dispatched, and accounted in military and five civilian hospitals within two hours. The canine revealed deficiencies in companion/military animal reception, holding, treatment, and evacuation. Three working groups were suggested: (1) to ensure 100% compliance with triage tags, patient accountability, and return of equipment used in mass casualty events and exercises; (2) to investigate making information tech-nology and imaging networks available for Emergency Operation Centers and Incident Command; and (3) to establish NDMS training, education, and evaluation to further integrate and support civil-military operations.

The exercises facilitated military/state inter-agency coopera-tion, resulting in revisions to the Maryland Emergency Operations Plan across all key state emergency response agencies. The recommendations from these exercises likely apply to the vast majority of NDMS activities in the US.