Guidance for Health Risk Assessment at Recurrent Mass Gatherings: The Jeddah Tool Framework

Abstract

A wide range of natural and man-made hazards increases the health risks at mass gatherings (MGs). Building on the Sendai Framework for Disaster Risk Reduction 2015-2030, the World Health Organization (WHO) developed the Health Emergency and Disaster Risk Management (H-EDRM) framework to strengthen preparedness, response, and recovery from health emergencies in the communities and emergency-prone settings, such as MGs. The Jeddah tool is derived from the H-EDRM framework as an all-hazard MG risk assessment tool, which provides a benchmark for monitoring progress made in capacity strengthening over a given period for recurrent MGs. Additionally, it introduces a reputational risk assessment domain to complement vulnerability and capacity assessment matrixes. This paper describes the key elements of the Jeddah tool to improve the understanding of health risk assessment at MGs in the overarching contexts of health emergencies and disaster risk reduction, in line with international goals.

Introduction

Over time, the definition of mass gatherings (MGs) has been broadened from the consideration of crowd size within a defined period and specified geographical area to prioritizing large gatherings that strain the health resources of the host. ^1,2 Mass gatherings tend to be characterized by similar health risks, such as crowd-related incidents, effect of exposure to severe weather events, and the increased transmissibility of communicable diseases, among others. ^3,4 Understanding disaster risk is a local and national priority under the Sendai Framework for Disaster Risk Reduction 2015-2030, which emphasizes an all-hazard and multi-sectorial approach to managing emergencies and disasters. ⁵ Building on the latter, the World Health Organization (WHO; Geneva, Switzerland) developed the Health Emergency and Disaster Risk Management (H-EDRM) framework to strengthen community preparedness, response, and recovery from health emergencies. The H-EDRM identifies hazard analysis, vulnerability assessment, and capacity assessment as key elements for periodic routine risk assessment in the community or for on-going risk assessment in vulnerable settings, such as MGs and refugee-settings. ⁶ The Jeddah tool is derived from the H-EDRM framework as an all-hazard MG risk assessment tool, which provides a benchmark for monitoring the progress made in capacity strengthening over a given period for recurrent MGs. ⁷

Report

This paper describes the key elements of the Jeddah tool to improve the understanding of health risk assessment at MGs in the overarching contexts of health emergencies and disaster risk reduction, in line with international goals.

Context and Tool’s Development

The WHO Eastern Mediterranean Regional Office (Cairo, Egypt) initially developed a health emergency risk assessment protocol to guide the identification and characterization of health risks in emergency-prone settings. The regional emergency risk assessment protocol was adapted to characterize the main health risks at the 2016 Hajj. As an outcome, WHO and the Saudi Ministry of Health (MoH; Riyadh, Saudi Arabia) through the Global Center for Mass Gatherings Medicine (a WHO Collaborating Center) jointly organized an international technical consultation for health risk assessment tools at MGs in January 2017. ⁸ The technical consultation proposed the naming of the adapted risk assessment protocol as the Jeddah tool for MG risk assessment, and the tool was modified with inputs from WHO and international experts, and validated through several MG risk assessments in the Kingdom. ^8–10

Methodology

In general, the Jeddah tool is derived from an existing framework which defines risks in the context of hazards and risk factors as ¹¹ :

$${\rm{Risk}}\ \alpha \ {\rm{Hazard}}\ {\rm{Magnitude}}\ {\rm{X}}\ {\rm{Vulnerability/Capacity}}.$$

A reputational risk assessment criterion was recently introduced to guide the estimation of overall risk. The analytical framework has been incorporated into a simplified toolkit for automated risk calculation with a corresponding user guide for strategic health risk assessment at MGs. ^7,12

Characteristics of the Jeddah Tool

Hazard Identification and Prioritization

In addition to historical threats from the MG or host community, the Jeddah tool categorized hazards into four groups alongside their potential data sources. These include endemic diseases in participant’s country of origin; current local, national, or international disease outbreaks/health threats; historical data from similar events or contexts; and threats arising from changing hazard, host, or vector characteristics. Although these groups are not mutually exclusive, such a classification facilitates the identification of most potential hazards for prioritization. The hazard prioritization score is defined by the product of frequency, magnitude, and exposure using the scoring matrix in Table 1 and Table 2. The matrix was partly developed from an existing validated risk assessment framework. ¹³ Once the prioritization scores for all listed hazards are obtained, the hazards with the highest scores are selected for further assessment of risk. The benchmark for the number of hazards included in the risk assessment is usually a trade-off between the availability of risk assessment resources, including time, and the risk management objectives at the MGs.

Table 1. Hazard Prioritization Matrix for Hajj Risk Assessment

Abbreviations: MERS, Middle East respiratory syndrome; NCD, non-communicable diseases; TB, tuberculosis.

Table 2. Criteria for Mapping Frequency, Magnitude, and Exposure

Abbreviation: MG, mass gathering.

Vulnerability Assessment Matrix

Vulnerability refers to the characteristics or circumstances of people, society, and organization that make them susceptible to the damaging effects of a hazard. ¹⁴ The initial step in vulnerability assessment is to develop a set of scientifically based indicators from the hazard and population characteristics and the predisposing factors. Two distinct sets of standardized indicators that incorporate hazard magnitude and vulnerability were developed for infectious hazards and non-infectious hazards. For the infectious hazards, the following indicator areas were considered: case-fatality rate, severe illness ratio, endemicity, route of transmission, vaccine effectiveness, vulnerable population, and effective treatment (Table 3). The non-infectious hazard vulnerability indicators have the following components: morbidity rate, mortality rate, complications/permanent disability, frequency of reporting, vulnerable population, prevalence of risk factors, and types of risk factors. ⁷ The risk areas are defined and ranked from one to five, in order of increasing vulnerability. Each indicator is assigned a weight, which is ranked quantitatively from one to five, to define the relative strengths of individual indicators. The weighted score is obtained from the product of the indicator score and the weight, and then the vulnerability assessment index is calculated as sum of all weighted scores for each hazard.

Table 3. Example of Vulnerability Indicator Domain for Infectious Hazards

Abbreviations: MG, mass gathering; STD, sexually transmitted disease.

Capacity Assessment Matrix

Capacity encompasses the strengths, attributes, and resources available within a community, society, or organization that could be harnessed to prevent or minimize the adverse effects of a hazard. ¹⁴ Generally, capacity assessment helps to identify the resources that are required for preparedness, response, and recovery from a MG health emergency. The capacity indicators areas are summarized separately for infectious hazards, non-communicable diseases (NCDs), and external causes of morbidity and mortality. The key indicator areas for infectious hazards include governance, infection prevention and control, disease surveillance, risk communication, rapid response team, laboratory capacity, and case management. The main indicator areas for NCDs include screening for chronic diseases, access to health services, core health care worker’s (HCW’s) density, in-patient bed density, case management, referral system, and health promotion. For external causes of morbidity and mortality, the relevant indicators areas are incident command and coordination, communication, prehospital triage management, core HCW’s density, in-patient bed density, risk communication, and emergency response time. These indicator areas primarily assess the availability of policies, infrastructure, and services and incorporate the health system readiness to implement such policies or provide relevant services (Table 4). Each indicator is scaled from one to five in order of decreasing capacity, with one sometimes indicating surge capacity and five showing unavailability of the assessed capacity. Each indicator is assigned a weight, which is also ranked from one to five, to define the relative strength of individual indicators. Similarly, the weighted score is obtained from the product of the indicator score and weight, and the sum of which equals the capacity assessment index (Table 5).

Table 4. Examples of Capacity Indicators Selected in the Jeddah Tool

Abbreviations: BICSL, Basic Infection Control Skill License; HCW, health care worker.

Table 5. Hypothetical Example of a Risk Scoring Matrix for Stampede during a MG Risk Assessment

Abbreviations: HCW, health care worker; MG, mass gathering.

Reputational Risk

In addition to the health risks, the tool introduced for the first time a reputational risk assessment index for MGs. The latter incorporates the dimension of hazard –whether national or international, the nature of hazards, number of expected mortalities, type of MG attendees, and expected media relevance – to rank reputational risks as low, moderate, high, and very high. ⁷

Estimating the Health Risk

To estimate the risk of each hazard, the total minimum score (if each indicator score is one), the total maximum score (if each indicator score is five), and the actual risk scores are obtained from the sum of the vulnerability and capacity weighted scores. Subsequently, the actual hazard risk is categorized as low, moderate, high, and very high using 0-25th, 25th-50th, 50th-75th, and 75th-100th percentiles, respectively, of the range between the minimum and maximum expected risk scores (Table 5).

Discussion

The Jeddah tool uses standardized quantifiable matrixes, such as hazard identification and prioritization matrix, to estimate health risk at MGs based on an all-hazard approach. Traditionally, hazards are identified from historical data based on a broad classification of hazards as either man-made or naturally occurring. ¹⁵ While the dependence on historical data may perhaps be sufficient for risk assessment at local MGs, there is an increased tendency to omit historically unrecognized emerging or re-emerging diseases and travel-related threats at international MGs. As such, the Jeddah tool incorporated a broader hazard identification framework. Due to the impracticality of conducting risk assessment for all identified hazards within the scope of available resources, the tool prioritizes and focuses on hazards that might constitute the highest threats to the MG. A useful way of ensuring that most identified hazards are selected for the risk assessment is to group similar hazards, especially those with similar mitigation and prevention characteristics, to reduce the number of hazards without compromising inclusivity.

A key attribute of vulnerability and capacity assessments is the development of scientifically derived indicators from relevant hazard and population characteristics, as well as the attributes of the health system. Despite the emphasis on the use of scientifically derived indicators, the availability of data was also an important consideration for the selection of indicators in the Jeddah tool. In comparison with the Jeddah tool, most pre-existing tools only possess a vulnerability assessment component for rapid risk assesssments. ^16,17 A few others lack appropriate benchmarks for monitoring capacity changes over successive seasons. ^13,18 Due to the damaging effect of certain health threats on the reputation of MGs, the Jeddah tool introduced for the first time a reputational risk assessment domain. During the 2020 G20 risk assessment in Saudi Arabia, a very high reputational risk of COVID-19 was an important consideration for recommendations to rather conduct a virtual G20 Leader’s Summit, despite the limited crowd size and significant preparedness of the health system.

The risk assessment process involves several steps that ensure data availability and facilitate the implementation of risk assessment recommendations. In the preparatory phase, a quick needs assessment is desirable, focusing on potential data sources and their reliability, technical resources requirements, and the existing knowledge and gaps to set clear risk assessment objectives. The involvement of a multi-sectorial group of stakeholders enhances transparency and ensures the accessibility of data and technical support to improve quality. Engaging these stakeholders early-on helps build trust, and as such, promotes the acceptability of findings and fosters their commitment to the implementation of the risk assessment recommendations through a well-coordinated effort. ⁹ In the Hajj context, the priority stakeholders include the Civil Defense (Riyadh, Saudi Arabia); MoH; Ministry of Environment, Water, and Agriculture (MEWA; Riyadh, Saudi Arabia); Municipality; Saudi Food and Drug Agency (SFDA; Riyadh, Saudi Arabia); and Saudi Red Crescents (Riyadh, Saudi Arabia). While the Civil Defense primarily provides crowd management functions and manages first responders during emergencies, the Saudi Red Crescents convey injured or ill persons from the site of an emergency to designated hospitals. Food and water safety, vector control, and liquid and solid waste management are regulated by the Municipality with the support of MEWA and SFDA. The MoH coordinate public health preparedness and response planning, including risk assessments, surveillance, and response to disease outbreaks and other emergencies, as well as health services delivery, including casualty/trauma management.

Data accuracy and reliability are prerequisites for the accurate estimation of health risks. ¹³ Interviews with heads and representatives of relevant departments for data collection should be followed by sighting and review of corresponding plans and policy documents. The usefulness of these data collection approaches exceeds the assignment of scores to vulnerability and capacity indicators; qualitative interviews and focus groups could unearth reasons or factors that underpin health system gaps to strengthen the recommendations for capacity improvement. Health risk assessment at MGs is a continuum which begins months before the event to allow enough time for the implementation of risk assessment recommendations. ¹⁵ It continues throughout the MG period and is completed afterwards with public health debriefings and follow-up of attendees to identify any threats to community health. ⁸ These concluding phases of risk assessments are often overlooked, yet they could aid the rapid detection of potential public health emergencies post-MG and improve the planning for future events. On completion of the risk assessment, a detailed report should be developed and disseminated to relevant stakeholders. Conducting a stakeholder’s workshop as part of the assessment findings dissemination process provides an opportunity for assessors to explain the findings and allow stakeholders to clarify concerns, share their opinions and perspectives, and proffer practical solutions for risk mitigation and prevention.

Limitation

Despite the extensive vulnerability and capacity needs at MGs, the availability of accurate data in the Hajj setting was a primary consideration for the selection of included indicators. Hence, it is most suitable for strategic decision making and does not necessarily represent the general capacity needs at the organizational or departmental level. Since the tool is yet to be tested outside the Saudi contexts so far, its applicability in other settings is uncertain.

Conclusion

The Jeddah tool identified specific vulnerability and capacity indicators, based on the experiences of periodic Hajj risks assessments, to provide a simplified tool for estimating MG health risk. Additionally, the tool introduces a reputational risk assessment component to complement health risk assessment, and promotes an all-hazard and multi-disciplinary approach to MGs risk assessment, in line with international goals.