In the aftermath of disasters, accurate and up-to-date information from surveillance systems for disaster-related health events is critical for public health officials, medical professionals, hospitals, and other organizations to direct resources and responses to reduce morbidity and mortality. Disasters are catastrophic events that cause death, damage to the environment and properties, interruption of services, and social and economic disruption,Reference Ardalan, Schnelle and Cittonte 1 making the collection and analysis of health-related data extremely challenging. On September 20, 2017, Hurricane Maria made landfall on the southeast coast of Puerto Rico with maximum winds just below the threshold of category 5 intensity.Reference Pasch, Penny and Berg 2 The hurricane proceeded to cross the island from southeast to northwest, leaving in its wake catastrophic damage to infrastructure, including a complete loss of power, water services, and communication island-wide. The combined direct damage and business interruption resulted in the costliest hurricane of the Caribbean on record.Reference Benfield 3
A study by George Washington University commissioned by the Governor of Puerto Rico estimated that 2975 (95% confidence interval: 2658-3290) excess deaths occurred in Puerto Rico during September 2017 to February 2018.Reference Santos-Burgoa, Sandberg and Suárez 4 According to the same study, the municipality of Ponce, in the southern part of the island, had a statistically significant increase in the crude mortality rate (17-27%) during September 2017 to February 2018 compared with previous years.Reference Santos-Burgoa, Sandberg and Suárez 4 Hospitals and other health-care institutions were frequently left with limited staff, resources, and electricity to operate and serve populations with increased health-care needs.
To enhance surveillance for adverse health events after Hurricane Maria, as well as provide updated information about the type of visits being seen in emergency settings, we implemented a hurricane surveillance system at 2 sites in Ponce, Puerto Rico: the emergency department (ED) of Saint Luke’s Episcopal Hospital (SLEH), which serves as a large regional hospital, as well as an affiliated urgent care clinic (Centro de Emergencias y Medicina Integrada, or CEMI). Both sites began hurricane surveillance the week of October 16, 2017, approximately 3 weeks after Hurricane Maria’s landfall. The sites were managed by staff in an acute febrile illness surveillance system established in 2012, and in consultation with the Centers for Disease Control and Prevention (CDC) Dengue Branch in San Juan.Reference Tomashek, Lorenzi and Andújar-Pérez 5 The initiative was modeled based on the general principles of syndromic surveillance: early detection for response, continuous gathering of pre-diagnostic information, situational awareness to monitor the effectiveness of a response, and early identification of potential outbreaks.Reference Katz, May and Baker 6
Syndromic surveillance typically supplements disease or condition-specific surveillance; however, it can also improve situational awareness during an event or disaster.Reference Wiedeman, Shaffner and Squires 7 At the time of the hurricane, there were no syndromic surveillance systems in place at the hospital system. In addition, there were frequent power outages and hospital staff often reverted to use of paper-based data collection. There were concerns about the completeness of electronic data from the medical record system, and also the timeliness of receiving and analyzing the data. We present the results of an evaluation of the surveillance system with 2 components; (1) a comparison of hurricane surveillance data to electronic medical record (EMR) data collected over the same time period and (2) a survey of stakeholders to assess the utility of the system.
METHODS
To implement the hurricane surveillance system, we recruited patients through an informed consent process in the ED of SLEH and from the CEMI urgent care clinic. All patients in the hospital ED or urgent care clinic were eligible for participation; however, study staff were not always available to recruit participants. A convenience sample of patients was recruited when study staff were present, with as many patients recruited as possible. Staff were present from 8 am to 10 pm (CEMI) or 11 pm (SLEH) all days, with decreased staff numbers on weekends and holidays. If patients were unconscious or unable to communicate, consent and interview responses were obtained from a guardian or representative. Recruitment began on October 16, 2017, and continued through March 28, 2018. Participants completed a short survey from a modified version of the CDC Natural Disaster Morbidity Surveillance Individual Form (https://www.cdc.gov/disasters/surveillance/index.html) and were asked about basic demographics, the primary reason for their visit, and if the visit was related to the hurricane. The Institutional Review Board of the Ponce Research Institute/Ponce School of Medicine Foundation approved the study protocol. The CDC reviewed the protocol and approved it as non-research.
Hurricane Surveillance Measures
The survey divided visit reasons into 5 categories: injury, acute illness/symptoms, exacerbation of chronic disease, mental health, and routine/follow-up. Visits that did not fall into any of these 5 categories were grouped as “other” visits. Questions about injury visits included details about the type of injury (eg, laceration, fracture, or sprain) and injury mechanism (eg, burn, fall or slip, foreign body, motor vehicle crash, or poisoning). Visits in the acute illness/symptoms subcategory included fever, gastrointestinal symptoms, respiratory symptoms, neurological symptoms, obstetrics and gynecology-related visits, and pain by body part (eg, headache, muscle or joint pain, abdominal pain, chest pain, and others). Visits related to exacerbation of chronic disease included participants with visits related to hypertension and congestive heart failure, diabetes, seizure, stroke, asthma, HIV, lupus, and other chronic diseases. Mental health visits included anxiety or stress, depression, drug/alcohol intoxication or withdrawal, psychotic symptoms, suicidal thoughts or ideation, and other previously diagnosed mental health issues. Participants who reported the main reason for visit was related to medication refill, blood sugar or pressure check, vaccination, or wound care were assigned to the routine/follow-up visit category. Interviewers were trained to attempt to elicit and select only 1 primary reason for visit for each participant, although selection of multiple visit categories was permitted if necessary.
After interviewing participants, staff later used the medical record system to assess the participants’ dispositions, as well as results of rapid testing for influenza and leptospirosis. Participants who received testing for influenza or leptospirosis, regardless of results, were categorized as suspected influenza or leptospirosis cases, respectively. All data were collected on mobile tablets using the EpiInfo data collection software. Data collection occurred offline on each tablet, and was synced daily to be merged into a larger dataset for analysis.
EMR Measures
To evaluate the representativeness and usefulness of the hurricane surveillance, we compared data about participant demographics and visit type collected through hurricane surveillance with data obtained retrospectively from the hospital’s EMR system in the same time period. Additionally, EMR data from the year before Hurricane Maria (September 1, 2016 to September 20, 2017) were assessed to determine baseline levels of visit types. In the EMR, the patient’s reason for visit is captured as a free-text field entered at registration. Diagnosis codes based on the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10) are assigned at the end of the visit by the attending physician, and verified or assigned by the medical record department approximately 5-10 days later.
The EMR data used for comparison was de-identified, and included patient age, sex, visit date, facility, reason for visit, and ICD-10 diagnosis codes 1-10. To categorize the ICD-10 codes, 2 epidemiologists reviewed the variables on the hurricane surveillance form and identified 1 or more matching ICD-10 codes for each variable, and assessed the number and type of visits over time and by site. SAS 9.3 was used to assign a primary reason for visit based on the first diagnosis code and the matching hurricane surveillance form category; visits that did not match any of the form variables remained uncategorized were placed into categories based on the second diagnosis code. If the first 2 diagnosis codes did not fall into any of the hurricane surveillance categories, the visit was categorized as “other.” All ICD-10 codes were reviewed by category of assignment to ensure they had been coded correctly.
To evaluate the representativeness of hurricane surveillance for specific diseases or events, including influenza, leptospirosis, and hurricane-related visits, we used SAS 9.3 to search across all diagnosis codes and the free-text reason for visit field of each record for mention of specific terms or ICD-10 codes related to the conditions of interest. Specifically, code A27, or the words “leptospirosis” or “lepto,” were used to indicate suspected leptospirosis cases; codes J9-J11 were used to identify influenza-related cases; and codes X37-X39, or the words “hurricane” or “huracan” were used to identify hurricane-related visits.
Analysis
Datasets from each tablet and site were merged into a combined hurricane surveillance dataset and analyzed using SAS 9.3. During the surveillance period, the merged dataset was used to create a 2-page weekly report containing information about recent visits and cumulative data, including patient demographics, visit types over time, patient disposition, hurricane-related visits, and the number of suspected cases of influenza and leptospirosis. Reports were distributed weekly to stakeholders, including clinicians and hospital management staff, at SLEH, CEMI, and CDC.
Cumulative hurricane surveillance data were summarized to describe the type and number of visits over time. To evaluate the representativeness of the hurricane surveillance data, the hurricane surveillance and EMR datasets were compared by patient demographics, visit category, and visit day. Visit categories were also displayed as trends over time by surveillance system and facility; the acute illness/symptoms were graphed separately from the injury and chronic visits to better show the variation over time on different scales. Mental health and routine visits were not graphed over time because of the small number of visits. Because the highest number of visits fell in the acute illness/symptoms category, the subcategories in this category were graphed over time separately.
Stakeholder Feedback and Resources Used
To assess the usefulness of the hurricane surveillance, we interviewed key stakeholders, including clinicians and hospital management staff, who received the hurricane surveillance reports with a short survey. Survey questions asked about how frequently the stakeholders opened and read the report and the usefulness of the content. Survey results were entered manually into Excel, and a descriptive analysis of findings was performed using SAS 9.3. To measure the number of resources needed to implement and maintain the hurricane surveillance system, we calculated the equipment and staff time used for hurricane surveillance over the 6-mo surveillance period, including work on-site as well as technical assistance from staff at the CDC Dengue Branch, located in San Juan, Puerto Rico.
RESULTS
Overall Findings From Hurricane Surveillance
During the 6-mo surveillance period, 5116 visits were recorded in the hurricane surveillance system, with just over half of the hurricane surveillance participants captured at SLEH (56%) compared with CEMI (44%).
Of hurricane surveillance visits, 100 (2%) were reported to be hurricane or disaster-related and 20 (0.4%) were listed as suspected for leptospirosis. The most common category of reason for visit was acute illness/symptoms, which was listed as the primary reason for visit in 4035 (79%) records. Acute visits were comprised of visits for acute pain (56%), fever (29%), respiratory issues (27%), gastrointestinal issues (20%), and visits related to obstetrics or gynecology (5%), among others. The next most common reason for visit was injury, with 573 (11%) records, followed by chronic health issues (5%), routine visits (0.8%), and mental health issues (0.6%). Approximately 6% of visits did not fall into any of the categories and were categorized as “other.” These visit reasons included dizziness, abscesses, and reported swelling and cellulitis of different body parts, among other visit reasons not falling into the predefined categories. Most participants were discharged to home (85%), although 11% were admitted to SLEH, and a small percentage left before being seen or were referred to care at another site or institution.
The overall number of hurricane surveillance visits each week hovered around 210, averaging approximately 120 participants per week from SLEH and 95 participants per week from CEMI. Recruitment at CEMI decreased slightly in the last 2 mo of surveillance, likely due to a decrease in staff availability. The type of visits remained fairly constant over time, although there was a slight increase in the proportion of acute visits, and a slight decrease in the proportion of injury visits over the 6-mo period at both sites. Suspected influenza cases increased sharply over the surveillance period, with a peak in week 17 at 58% of all visits (Figure 1). Hurricane-related visits were primarily associated with injury (56%) and acute illness/symptoms (44%); however, because participants could have more than 1 visit reason, some were also reported as related to mental health (3%) and chronic health issues (2%). The number of hurricane-related visits was highest in the first 6 weeks of hurricane surveillance (73% of hurricane-related visits), although hurricane-related visits were reported through week 21 of surveillance. The activities reported at the time of injury or illness were primarily associated with cleaning debris, trees, trash, or around the home or patio, as well as lifting objects.
FIGURE 1. Hurricane Surveillance Acute Visit Subcategories by Surveillance Week From Hurricane Surveillance Sites, Ponce, Puerto Rico, October 2017 to March 2018.
Comparison with EMR System
A total of 29,383 EMR records were received for the hurricane surveillance period and assigned a visit category based on the first or second diagnosis code. Most (92%) visits were assigned a visit category based on the first diagnosis code; an additional 3% of visits were categorized based on the second diagnosis code. Approximately 4% of the EMR visits remained uncategorized, compared to 6% of the hurricane surveillance records (Table 1).
TABLE 1 Characteristics of Participants in the Hurricane Surveillance Compared With All Patient Visits Recorded in the EMR System by Site, Ponce, Puerto Rico, October 2017–March 2018
Abbreviations: CEMI, Centro de Emergencia and Medicina Integrada; EMR, electronic medical record; SLEH, Saint Luke’s Episcopal Hospital.
Hurricane surveillance captured 17% of visits in the EMR data across both sites. However, a higher proportion of visits were captured at CEMI (29%) than SLEH (13%) (P < 0.01). (Table 1) Participants in hurricane surveillance were similar (only differed by 1-3%) by sex and age category compared with patients captured in the EMR system. The patient visit categories had comparable proportions between the 2 systems, both overall and by site; for example, acute illness visits accounted for 82% of visits at CEMI in both the hurricane surveillance and EMR systems, and 77% and 74% of the hurricane surveillance and EMR visits, respectively, at SLEH. The hurricane surveillance captured more hurricane-related visits (n = 100) compared with the EMR system (n = 21), as well as a higher number of suspected influenza cases (n = 1487) compared with the EMR system (n = 1403). Additionally, hurricane surveillance identified a slightly higher proportion of suspected leptospirosis cases (0.4%) compared with EMR (0.2%), although a higher number of cases was identified through EMR (n = 52) than hurricane surveillance (n = 20). The trends in visit category type over time were also similar between hurricane surveillance and EMR (Figure 2), although the hurricane surveillance trends at CEMI were slightly closer to the EMR data than at SLEH.
FIGURE 2. Comparison of the Proportion of Visits Categorized as Acute, Injury, or Chronic Detected Through Hurricane Surveillance and Electronic Medical Records (EMR) at Saint Luke’s Episcopal Hospital (A) and Centro de Emergencias y Medicina Integrada (B), Puerto Rico, October 2017 to March 2018.
Baseline EMR Data
The majority of visits in the EMR system from the year before Hurricane Maria were associated with acute illness/symptoms (Figure 3). An increase in acute visits was observed in December 2016 at both sites, concurrent with influenza trends for that year. During January to September 2017, the number of acute, injury, and chronic visits remained fairly steady with a sharp drop in the number of visits during and immediately after the hurricane. The number of visits began to increase above previous levels beginning in early October with a peak in visit numbers at both sites the week of October 22, 2017. Visit numbers for categories assessed remained slightly elevated at both CEMI and SLEH in the 6 mo after the hurricane that corresponded to the hurricane surveillance period.
FIGURE 3. Acute, Injury, and Chronic Visits From Electronic Medical Records (EMR) Before and After Hurricane Maria Seen at Saint Luke’s Episcopal Hospital (A) and Centro de Emergencias y Medicina Integrada (B), September 2016 to March 2018.
Stakeholder Feedback
In February 2018, surveys were sent to 16 hospital stakeholders who received the weekly hurricane surveillance reports. All respondents confirmed that they received the weekly hurricane surveillance reports. More than half (56%) said they always opened the report. Six (38%) reported they always read the entire report; the remaining 63% reported sometimes reading the entire report. When asked about the amount of information in the report, 15 (93%) responded that the report contained the right amount of information; only 1 respondent indicated that the report contained too much information.
Respondents were also asked open-ended questions about how they used the hurricane surveillance report information and which information was most and least useful. Most (73%) respondents said they used the information in the report. Specifically, respondents said the reports were used to help in the assessment and differential diagnoses of patients, compare report data to private practice data, have discussions with faculty, and follow numbers and trends for specific diseases or areas including influenza, leptospirosis, and general ED visit and chief complaint numbers. Suggestions for improvement included more interpretation of graphs, better description of visits in the “other” category, inclusion of socioeconomic data, and publishing a summary end-of-year report describing the cumulative visits.
Resources Used
Implementation of the system involved use of 6 Samsung Galaxy Tab A tablets for data collection, which were available for use through the CDC Dengue Branch. Multiple staff were involved in data collection and management at the site level. At SLEH, this included full-time coverage by 2 research assistant positions and a nurse, as well as a coordinator for 10 h weekly. Staff inputs at CEMI included 1 research assistant and 1 nurse full-time, as well as a coordinator for 10 h weekly. Staff time at CEMI decreased slightly during the last 2 mo of surveillance, with less than full-time coverage. Two coordinators contributed 42 h of work during project initiation and implementation. Coverage on weekends was variable, with more complete coverage in the first month after implementation. Of note, hurricane surveillance staff were also involved in other roles and projects and time was not spent exclusively on hurricane surveillance. The total estimated cost for staff time at the site for implementation and maintenance of the surveillance system was $78,260.86. The tablets used for data collection had been purchased previously, and are not included in the cost estimates.
The CDC provided technical assistance in the form of database development, data management, data analysis, weekly report development and dissemination, and troubleshooting technical or questionnaire-related issues. During surveillance implementation, this took ~50% of the time of 3 epidemiologists, as well as 30% of a public health advisor and 5% of a supervisory epidemiologist’s time. After implementation, time decreased to 5-20% of the 5 staff members’ time for the 5 mo of hurricane surveillance maintenance.
DISCUSSION
After Hurricane Maria, we implemented a surveillance system to collect information about the reason for patients’ visits in 2 facilities to inform clinicians, hospital administrators and other stakeholders in emergency health care and recovery efforts. Although the hurricane surveillance did not capture all visits, in comparison to the EMR data, we found it to be very similar by patient age, sex, and visit category, including trends over time. Very broad categories were used in this comparison because of the complexity of the ICD-10 codes; however, we believe that smaller categories of visit types would be similar as well. Most visits were associated with acute illness or symptoms, including pain; the second most common reason for visit was injury. Hurricane surveillance was more sensitive than EMR in identifying hurricane-related visits and suspected leptospirosis cases, which was one of the original goals of the surveillance platform. In addition, we identified a higher proportion of suspected influenza cases.
In addition to finding hurricane surveillance to be representative of all visits, the system provided a timely source of information about the type of visits and suspected cases with reports sent to stakeholders at the beginning of the week following data collection. This occurred more quickly than the ICD-10 codes are available for patient visits (~10 days). In addition, this type of data to categorize visit types by ICD-10 codes, or any other type of ongoing syndromic surveillance, is not currently available through the EMR system. Substantial data management and cleaning had to be done to be able to analyze and interpret the EMR data; an additional challenge in this setting is that free text responses such as the reason for visit are captured in a mixture of Spanish and English, and algorithms used to search for words or phrases in only English or Spanish are not sufficient. Although a system could be implemented to analyze visits in the EMR system in real-time, this would take a commitment of staff time and resources to implement and maintain. Syndromic surveillance was not in place in Ponce at the time of the hurricane, but this system allowed its implementation after the hurricane.
In our setting, hurricane surveillance provided a representative, timely, and manageable method of providing surveillance data in a system without existing syndromic surveillance capacity, and where there was a high demand for information about the health effects of the hurricane. The cost of implementing the system over the 6-mo period, at nearly $80,000, was higher than anticipated. The rapid implementation and maintenance of this system was facilitated by the presence of existing projects, study staff, and infrastructure for patient recruitment. While the public health surveillance literature frequently refers to the use of automated data sources for syndromic surveillance, it can also include manual components, some of which have proven feasible in low-resource settings.Reference Katz, May and Baker 6 The hurricane surveillance data were collected electronically through tablets to facilitate analysis but relied heavily on recruitment and manual data collection through staff. This system was effective given the setting; however, it would be ideal to set up an automatic data capture system for syndromic surveillance, similar to syndromic surveillance systems in other EDs,Reference Thomas, Yoon and Collins 8 for future disasters or emergencies.
The baseline EMR data analyzed from the pre-hurricane period helped provide some background for the trends before and after Hurricane Maria. It demonstrated a steep drop and subsequent increase in cases after the hurricane, with visit numbers remaining elevated during the 6-mo surveillance period. The increase in visit numbers for an extended period after the hurricane was believed to be due to the closure of other smaller health-care facilities after the hurricane, with the result of more patients seeking care at CEMI and SLEH. Direct effects of the hurricane could also have contributed to increased visit numbers, although they appeared to be a small proportion of total visits from hurricane surveillance. At the facility level, surveillance at CEMI captured a higher proportion of EMR visits compared with SLEH, as the clinic had less patient visits overall; however, a higher overall number of patients participated in the surveillance at SLEH.
The information provided by the system was found to be useful by stakeholders and could be useful in other emergency or disaster situations, particularly in low-resource settings without an existing syndromic surveillance system or without access to real-time electronic visit data. The system successfully detected an increase in influenza cases that corresponded to increased influenza activity in Puerto Rico 9 and would have been missed if only relying on EMR. This early notification about influenza activity may have increased clinician awareness about influenza cases earlier at the facilities, and made clinicians more likely to consider influenza in other patients. Because our surveillance identified suspected, rather than confirmed, leptospirosis cases, we were unable to determine if the number of cases after the hurricane represented an increase from pre-hurricane levels. However, as a result of the hurricane surveillance, steps were taken to enhance the existing acute febrile illness surveillance system’s capacity to identify suspected leptospirosis cases for confirmatory testing.
Future directions at this facility could include setting up an automated syndromic surveillance system that could provide baseline data about visit types on an ongoing basis to better be able to identify changes during public health emergencies. If this is not possible, stakeholders could make preparations in advance to implement the system more rapidly if needed, such as having a standing institutional review board approval for surveillance implementation in specific scenarios, electronic form templates created and ready, training materials for staff, and a protocol for recruitment and data management and analysis.
There are a number of limitations to the hurricane surveillance system. There was a delay in implementation for 3 weeks after the hurricane hit, and the hurricane surveillance was not in place in time to capture the likely higher proportion of hurricane-related visits in that initial period; however, the hurricane surveillance was in place for the majority of the post-hurricane period. Categorization of hurricane-related visits was based on the patient’s response and is likely an underestimate, as visits such as exacerbation of existing health issues or illnesses associated with the disaster setting might not have been directly attributed to the hurricane by the respondent.
In addition, the wording of the question could have missed visits that were related to the hurricane but were not work-related. Although the hurricane surveillance data appeared very similar to the EMR data by visit category and patient demographics, we were unable to compare the severity of the conditions and assess patient outcome status in the 2 surveillance systems, and the hurricane surveillance might have been more likely to include participants with less severe conditions because they were willing and able to respond to questions. There could have been errors in how the EMR data were categorized using ICD-10 codes, as the ICD-10 codes did not always match specifically with the categories in the hurricane surveillance form; however, 2 epidemiologist coauthors reviewed the ICD-10 assignments and tried to conservatively assign the category that best fit the diagnosis code. Although we captured an increase in suspected influenza cases, we were unable to evaluate the impact of vaccine distribution disruptions or decreased vaccine efficacy.
CONCLUSIONS
In the wake of Hurricane Maria, we implemented a surveillance system to characterize the number and type of visits to an ED and urgent care clinic, and provide information to help guide response efforts. For example, clinicians reported using the data to guide decision-making in their clinical practice, and an increase in influenza cases was identified that mirrored surveillance data from other systems. The data collected were representative of all visits seen in the facilities and were found to be timely and useful for stakeholders in the hospital system. Our findings indicate that ED-based surveillance, although potentially costly, can provide representative and timely data after a disaster.
Acknowledgments
We recognize the assistance and support of the SEDSS team and the ZIPER team in data collection, and the CDC EpiInfo team in the creation and troubleshooting of the mobile questionnaire on tablets.
Disclaimer
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2019.79
