Integrating Behavioral Health Into Monitoring and Surveillance During Public Health Emergencies: Challenges and Opportunities

Abstract

Objective

Limited guidance exists for public health agencies to use existing data sources to conduct monitoring and surveillance of behavioral health (BH) in the context of public health emergencies (PHEs).

Methods

We conducted a literature review and environmental scan to identify existing data sources, indicators, and analytic methods that could be used for BH surveillance in PHEs. We conducted exploratory analyses and interviews with public health agencies to examine the utility of a subset of these data sources for BH surveillance in the PHE context.

Results

Our comprehensive search revealed no existing dedicated surveillance systems to monitor BH in the context of PHEs. However, there are a few data sources designed for other purposes that public health agencies could use to conduct BH surveillance at the substate level. Some of these sources contain lagging indicators of BH impacts of PHEs. Most do not consistently collect the sociodemographic data needed to explore PHEs’ inequitable impacts on subpopulations, including at the intersection of race, gender, and age.

Conclusions

Public health agencies have opportunities to strengthen BH surveillance in PHEs and build partnerships to act based on timely, geographically granular existing data.

Public health emergencies (PHEs, which are events, whether natural or human-made, that pose a risk to health—for example, hurricanes, pandemics, and terrorist attacks) can negatively impact behavioral health (BH). For some individuals, BH impacts, defined as new or worsening mental health conditions or substance use disorders, can be severe and long-lasting.¹ After a PHE, approximately 1 in 4 adults will develop clinically significant post-traumatic stress syndrome and anxiety, and about 1 in 5 will develop depressive symptoms.² Over time, symptoms of post-traumatic stress syndrome typically decrease, but depression and anxiety can persist. Both the direct impacts of the COVID-19 pandemic and the necessary public health measures to control its spread could have exacerbated the need for BH services. For instance, depression symptoms in children and adolescents increased during the pandemic, particularly among girls,³ and drug overdose mortality was higher in 2020 than in previous years.⁴

Currently, no dedicated surveillance systems exist to monitor BH during and after PHEs.⁵ Even outside of the PHE context, in lieu of having a comprehensive surveillance system for BH, public health officials rely on a patchwork of surveys and surveillance systems⁶ that have limitations, including long delays between data collection and release, nonrepresentative sampling, and lack of validated BH measures.^{5, 7}

With improved monitoring and surveillance of BH precursors and impacts, public health officials can understand BH trends and better anticipate potential impacts of PHEs, identify acute increases in BH needs, evaluate the effectiveness of BH interventions, position resources to prevent adverse BH impacts from future PHEs, and begin to address long-standing BH inequities that may emerge in PHEs.

To plan for, respond to, and recover from PHEs, public health agencies and their BH partners need timely local data on BH needs to develop tailored public health interventions. However, little is known about the strengths and limitations of existing data sources, indicators of population-level BH, and methodological approaches for conducting BH surveillance before, during, and after PHEs. Furthermore, there is limited guidance on how public health agencies can use BH surveillance data to better prepare for and respond to PHEs.

To fill these gaps, we conducted a mixed-methods study to answer 2 primary research questions (RQs):

RQ 1: What indicators, data sources, and analytic approaches could be used for BH surveillance in the PHE context?

RQ 1.1. What types of BH indicators are needed for surveillance in the PHE context?

RQ 1.2. What existing data sources contain these BH indicators, are available at the substate level, and are updated frequently enough to be useful in the PHE context?

RQ 1.3. How can public health agencies access and analyze these data, and what are the potential strengths and limitations of the different data sources?

RQ 2: How could better BH surveillance before, during, and after PHEs support decision-making and public health action?

RQ 2.1. How can these data be used to inform public health decision and action?

RQ 2.2. What do public health officials perceive as best ways to use BH analyses to inform decisions and public health action?

Methods

We conducted a literature review and environmental scan to identify data sources, indicators, and analytic approaches that could be used for BH surveillance in the PHE context. We then conducted exploratory analyses and interviews with public health officials to examine the utility of a subset of these data sources for BH surveillance in PHEs (Figure S.1). The corresponding author’s Human Subjects Review Committee exempted this study from further review (2019-0920-AM03).

RQ 1: Identifying BH Indicators, Existing Data Sources, and Analytic Approaches

RQ 1.1: Review of systematic reviews and meta-analyses to identify indicators

To develop a conceptual model of potential types of BH indicators for surveillance in the PHE context, we conducted a keyword search of the PubMed, PsycINFO, and Web of Science databases (Supplemental Online Appendix Table S.1). To focus on the most current information available within a rapidly changing landscape, we limited the search to English-only systematic reviews and meta-analyses from the prior 5 years: 2014 to November 2019 (the date the search was conducted). We screened resulting references for relevance and systematically abstracted information from included articles.

RQ 1.2 and 1.3: Environmental scan to identify data sources and analytic methods

We then performed an environmental scan consisting of a second literature review and interviews with representatives of organizations that collect or aggregate data relevant to BH (e.g., Poison Control Centers, the National Retail Data Monitor), representatives from public health agencies, and experts in public health and BH. The purpose was to identify data sources that contained relevant BH indicators and analytic methods that have been used to examine those indicators.

Literature search. We searched the peer-reviewed literature published in English from 2010 to 2020 using PubMed and PsycINFO and the search terms in Supplemental Online Appendix Table S.2. We also conducted searches for grey literature on Google Scholar for selected data sources using search terms tailored to each source (Supplemental Online Appendix Table S.2). After screening articles for relevance, we again systematically abstracted information from included articles.

Findings from the search were used to narrow down the list of data sources to a subset that was assessed to be promising for BH surveillance in the PHE context, defined as meeting the following 3 criteria: available (1) at the sub-state level, (2) more frequently than once a year, and (3) for most jurisdictions in the United States.

Informational interviews with organizations that collect or aggregate data relevant to BH. We held 21 informational interviews lasting 45 to 60 minutes with representatives of organizations that collect or aggregate data relevant to BH, focusing on the subset of promising data sources. We identified these individuals through targeted web searches and chain-referral sampling. After obtaining verbal consent, we used a semi-structured discussion guide (Supplement 1) to assess how public health agencies could access and analyze these data, and strengths and limitations of the data sources. We analyzed our detailed notes using a qualitative descriptive approach.

In-depth interviews with public health agencies and experts. We conducted 11 interviews lasting 45 to 60 minutes with state and local public health agencies and experts in public health and BH, selected for their experience with these data sources and analytic approaches for BH surveillance. After obtaining verbal consent, we used a semi-structured discussion guide (Supplement 1) that covered challenges with BH surveillance, potential data sources, and analytic methods. Interviews were audio-recorded and transcribed. Transcripts were coded in Dedoose for common themes.

Findings from both the informational and in-depth interviews informed the design of our exploratory analyses (RQ 2.1).

RQ 2: Examining the Utility of the Promising Data Sources

RQ 2.1: Exploratory analyses

To answer RQ 2.1, we conducted exploratory analyses of the promising data sources. We obtained datasets for locations that had experienced one or more PHEs, had sufficient population sizes for robust analyses, and were in different regions of the country. Some of these sources were publicly available, whereas others required data requests and data use agreements. For each source, we followed a structured process to examine the utility of the data source for BH surveillance in PHEs (Supplemental Online Appendix S.3). We documented our qualitative assessments at each step, which we integrated with interview findings described below (RQ 2.2) to arrive at our final assessment of the sources. Each source required slightly different analytic methods, which are presented elsewhere.^8–11

RQ 2.2: Interviews with public health agencies and data experts

We conducted 60-minute in-depth interviews lasting 45 to 60 minutes with 37 state, tribal, and local public health representatives; subject matter experts in public health and BH; and data source experts. We identified interview participants through chain-referral recommendations and the literature search for RQs 1.2 and 1.3. After obtaining verbal consent, we used a semi-structured discussion guide (Supplement 2) to gather input on how findings from the exploratory analyses (RQ 2.1) could inform public health action, focusing on the utility of these findings for BH surveillance in PHEs. Interviews were audio-recorded and transcribed, and transcripts were coded in Dedoose for common themes.

Results

RQ 1: Identifying BH Indicators, Existing Data Sources, and Analytic Approaches

RQ 1.1: A conceptual model of BH indicators

Through the literature review and environmental scan, we identified approximately 40 types of BH indicators that could be surveilled in the PHE context. We developed a conceptual model that groups these indicators into 3 categories (Figure 1):

• • Upstream indicators: community strengths, vulnerabilities, and other social determinants of BH that may influence the likelihood that a future PHE will impact population-level BH. These include risk and protective factors such as unemployment or job loss, housing instability, and social cohesion, several of which have been linked to BH.^12-14

• • Midstream indicators: early signs of BH impacts after the PHE has occurred for which there is still time to intervene and mitigate their effects. These include “calls for help” when BH is declining, such as to poison control centers (PCCs) for intentional ingestions, or measures of coping strategies such as taking over-the-counter sleep aids for insomnia associated with stress or depression.¹⁵

• • Downstream indicators: measures of a PHE’s later impacts on BH. These include emergency department (ED) visits for a BH crisis, calls for emergency medical services (EMS) to respond to an overdose¹⁶ or BH emergency, or deaths related to overdose²¹ or suicide.¹⁷

Figure 1. Conceptual framework of indicators of community strengths and vulnerabilities and behavioral health impacts of public health emergencies.^{1, 2}

*Promising data sources for which we conducted exploratory analyses

^**Promising data sources that we assessed through the environmental scan and in-depth interviews but for which we did not conduct exploratory analyses

¹Abbreviations: BH: behavioral health; PHE: public health emergency; UI: unemployment insurance; OTC: over-the-counter; PCC: Poison Control Centers; EMS: emergency medical services; ED: emergency department

²Promising data sources that could be repurposed for BH surveillance during and after PHEs are bolded

Not all the indicators in Figure 1 are available in the data sources discussed next. However, they are included for completeness, as public health agencies may have access to local data sources that are not available in other jurisdictions.

RQ 1.2: Existing data sources

The literature review, environmental scan, and formative interviews yielded 27 data sources containing one or more of the BH indicators we identified but no single dedicated data source for monitoring BH in the PHE context.

Applying the 3 criteria described previously to those 27 sources (i.e., available (1) at the substate level, (2) more frequently than once a year, and (3) for most jurisdictions in the United States) yielded 8 data sources that were considered promising for use in BH surveillance during and after PHEs:

• • unemployment insurance claims

• • calls to 2-1-1 centers

• • calls to poison control centers for intentional ingestions

• • over-the-counter medication sales of sleep aids

• • pharmacy data on psychotropic medication fills

• • EMS activations

• • ED visits, and

• • web searches for BH-related terms.

These sources contained indicators in the upstream, midstream, and downstream categories.

Two of these sources, EMS activations and ED visits, had national dashboards or well-established national surveillance programs, so we did not obtain these data for primary data analysis and instead used in-depth interviews (RQ 2.2) to solicit examples of how they have been used for BH surveillance in PHEs.

Next, web search data (i.e., aggregated through the Google Trends website) was excluded from the list of promising data sources after our exploratory analyses failed to detect a BH signal in these data, and we found that the data source did not have sufficient spatiotemporal granularity to be useful to state and local public health officials.

Finally, based on the in-depth interviews, we added calls to the 9-8-8 Suicide and Crisis Lifeline to the above list of promising data sources, bringing the total back to 8. The 9-8-8 dialing code launched in July 2022 and therefore did not “go live” in time for us to obtain data for analysis.

Table 1 summarizes how to access the data sources, their spatio-temporal granularity, and their strengths and limitations. Bolding indicates the 5 sources with which we conducted our own exploratory analyses (unemployment insurance claims, calls to 2-1-1 centers, calls to poison control centers for intentional ingestions, over-the-counter medication sales of sleep aids, and pharmacy data) (Supplemental Appendix S.3). Our assessment of EMS activations, ED visits, and calls to 9-8-8 was based on the environmental scan (RQ 1.2 and RQ 1.3) and semi-structured interviews (RQ 2.2).

Table 1. Summary of the most promising data sources for behavioral health surveillance in the context of public health emergencies^a

^a Adapted from Acosta et al.(¹⁹).

Abbreviations: BH, behavioral health; CDC, Center for Disease Control and Prevention; ED, emergency department; EMS: emergency medical services; ICD-10, International Classification of Diseases, 10th Revision; NEMSIS, National Emergency Medical Services Information System; NPDS, National Poison Data System; NRDM, National Retail Data Monitor; NSSP, National Syndromic Surveillance Program; OTC, over-the-counter; PCC, Poison Control Centers; PHE, public health emergency; UI, unemployment insurance.

Overall, existing data sources lacked data on sociodemographic factors that would be necessary to examine BH inequities. Only unemployment insurance claims and ED visit data consistently contained information on age, gender, and race of the claimant or patient. Age and gender were available in the data on poison control center (PCC) calls and prescription medication fills; however, race was not. Only gender was consistently collected from callers to 2-1-1 centers. Age, race, and gender were inconsistently included or not available in data on sleep aid sales, EMS activations, and suicide and crisis hotline calls.

Although other data sources contain BH indicators (e.g., those compiled by the Council of State and Territorial Epidemiologists through an initiative to strengthen surveillance of mental health and substance use),¹⁸ these indicators are not available frequently enough to be useful in the PHE context (Supplemental Online Appendix Table S.4).

RQ 1.3: Analytic approaches

We identified over 100 studies that described possible analytic approaches to monitor BH indicators (Table 2).¹⁹ Most studies used retrospective and observational repeated cross-sectional or longitudinal analytic approaches aggregating information across space (e.g., census tract) and/or time (e.g., weekly), rather than relying on prospective data collection using a counterfactual or comparison group. Analytic methods varied based on the question(s) to be explored or the indicator(s) to be monitored; the type(s) and volume of data available for analysis; the timing in relation to a PHE; and the available resources, capacity, and technical proficiency that the PH agency and its partners bring to this work. Commonly used methods for time series analyses of BH indicators included data-smoothing techniques followed by statistical modeling such as interrupted time series analyses or ITSA, difference-in-difference, and ARIMA (i.e., autoregressive integrated moving average) modeling.

Table 2. Illustrative publications that use the promising data sources to monitor behavioral health^a

^a For a complete list of publications identified through our environmental scan, see Acosta et al.(¹⁹).

^b The studies included for this data source differ from the others in that they demonstrate an association between unemployment (the indicator of interest) and behavioral health or public health emergencies.

RQ 2: Examining the Utility of Existing Data Sources

RQ 2.1: Uses of these data sources to inform public health decisions and action

Table 3 summarizes methods and key findings from our exploratory analyses using 5 of the promising data sources in the context of PHEs. Full details of these analyses can be found elsewhere.¹⁹ Across the multiple locations we examined and different PHE types, we found that changes in indicators of BH risks and impacts of PHEs can be detected in these data sources, suggesting the utility of these data to inform public health decision-making.

Table 3. Summary of exploratory analyses of five of the most promising data sources for behavioral health surveillance in the public health emergency context

^a Adapted from Acosta et al.¹⁹

Abbreviations: ARIMA, autoregressive integrative moving average; COVID-19, Coronavirus Disease 2019; CPA, change-point analysis; ITSA, interrupted time series analysis; OTC, over-the-counter; PHE, public health emergency; SSRI, selective serotonin reuptake inhibitor.

Overall, we observed anomalies in the indicators we examined across most of the promising data sources for the start of the COVID-19 pandemic, but anomalies in these indicators around more acute, localized PHEs were more difficult to detect. For instance,

• • in Los Angeles County, unemployment insurance claims peaked just after California’s business closure mandate in April 2020 and again in September 2020, coinciding with severe wildfires. In June 2020, SSRI fills began to increase.

• • in Los Angeles County, we detected 4 periods of anomalous activity in over-the-counter sleep aid sales, coinciding with the 4 large COVID-19 waves there.

• • in Broward County, Florida, calls to 2-1-1 after Hurricane Irma and the pandemic declaration showed a larger percent increase over baseline for men than for women. Calls by women remained elevated for a longer period than by men after the hurricane. The opposite was found after the pandemic declaration.

• • by early 2021, calls to PCCs for intentional exposures among youth in Dallas County, Texas, increased after the pandemic declaration, closing the longstanding gap for intentional exposures between youth and adults. We detected an increase in calls for intentional exposures in this county during the summer surge in COVID-19 cases but not shortly after the pandemic declaration. The winter storm of February 2021 was not associated with an increase in calls to PCCs for intentional ingestions.

RQ 2.2: Public health officials’ perceptions of the utility of these analyses to inform public health action

Public health agency representatives identified several public health actions that could result from timely BH surveillance efforts.

• • Before a PHE: Based on surveillance of upstream BH indicators, public health agencies and their partners could develop protective interventions, advocate for policies, and allocate resources to mitigate BH risks.

• • During and after a PHE: With better surveillance of mid- and downstream BH indicators, public health agencies and their partners could alert emergency management services, BH, and other health-care providers to prepare for possible surges in demand for services and could advocate for resources to respond to those needs.

To increase their confidence in potential surveillance signals and to inform public health action, public health practitioners reported the need to compare findings across multiple indicators. Even among jurisdictions that have explored the integration of various data streams into their BH surveillance efforts, such as Washington State²⁰ and North Carolina,²¹ interviewees noted some persistent challenges. More research is needed to (1) understand to what extent anomalies detected in these data sources correlate with “true” changes in population prevalence of BH needs and (2) use a signal in one data source (e.g., unemployment claims) to predict when other downstream signals might emerge (e.g., increases in ED visits for BH).

Discussion

Our study did not identify any existing dedicated surveillance systems to monitor BH in the context of PHEs.⁵ However, there are a limited number of data sources that public health agencies in the United States could repurpose for timely BH surveillance at the substate level. We were able to detect “signals” of population-level BH shifts in these data sources, even though these data were not originally collected for this purpose. The magnitude and duration of anomalies in the BH indicators varied by PHE, age, and gender. Consistent with our conceptual model, changes in time series of BH indicators occurred at different time points in the PHE context, with upstream indicators peaking first followed by mid- and downstream indicators.

Existing data sources were limited and primarily allowed for monitoring of lagging, rather than leading, BH indicators. Aside from unemployment claims, the data sources we identified had limited information about structural inequities and well-being, which are key factors related to BH. Only a few data sources consistently collect the sociodemographic data needed to examine PHEs’ inequitable impacts on subpopulations, including at the intersection of race, gender, and age.

Additional data sources with the spatiotemporal granularity required to be useful in the PHE context are needed to more comprehensively monitor upstream social determinants of health that influence BH in the short and longer term. Efforts to modernize and transform public health data systems in the United States^{22, 23} are underway to improve the consistency and completeness of data on sociodemographic factors to allow for subgroup analyses and better target public health interventions. These initiatives will benefit from close coordination among public health and BH stakeholders at federal, state, and local levels to clearly define roles and responsibilities around data collection, analysis, reporting, and action.

Emerging technologies and advanced analytic methods also offer opportunities for continued progress in strengthening BH surveillance, during both PHEs and routine times. For example, wastewater surveillance is increasingly being applied in jurisdictions across the country to monitor trends in levels of opioids in wastewater.²⁴ As these emerging technologies come into more widespread use, they may provide important complementary data for more integrated, timely, equitable, and actionable BH surveillance during PHEs.²⁵

To use these data for public health action, additional capabilities and partnerships are necessary. Platforms for aggregating and visualizing BH data are advancing. For instance, the National Emergency Medical Services Information System has added BH indicators to their dashboard,²⁶ and the National Retail Data Monitor has added over-the-counter sleep aids to its data categories. However, partnerships that include key local organizations from the BH and public health systems are necessary for data interpretation and dissemination.

Experts noted the importance of triangulating among different data sources. However, even after comparing findings across different data sources and using statistical methods to test for significant shifts in BH indicators, decision-makers may face uncertainty around the best course of action to take given the findings. Public health officials will need to acknowledge the gaps in existing data when co-interpreting findings with the affected populations and conveying results and recommendations to different audiences.

Limitations

This work has some limitations. First, the literature review, although rigorous in its approach, was not designed to identify all relevant articles on this broad topic, so it is possible that there are other BH indicators that could be considered for surveillance in the PHE context. Second, the exploratory analyses are intended to be preliminary examinations of the promising data sources we identified. They were conducted retrospectively, using selected past PHEs, including hurricanes, wildfires, a severe winter storm, and the onset of the COVID-19 pandemic, rather than as the PHEs unfolded, which would have allowed us to draw conclusions about their utility for BH surveillance in real time. In addition, the findings from these analyses may not be generalizable to other PHEs and to other locations.

Conclusions

Effectively monitoring and addressing BH needs, both during a PHE and in routine times, depends on timely BH surveillance data with sufficient geographic granularity to inform just-in-time decision-making. Although our study did not identify a dedicated surveillance system to monitor BH in the context of PHEs, several existing data sources could be repurposed by public health agencies to strengthen their BH surveillance efforts in the PHE context.

Implications for Public Health Practice

In the near term, while data modernization efforts are underway, public health officials could use the data sources identified in this study, supported by a tool kit with instructions on how to access and analyze each of them.¹⁹ These data modernization efforts are an opportunity to ensure that BH indicators are integrated into these changing systems so that they could be monitored and surveilled to inform the BH response to PHEs. Public health officials also could focus on leveraging existing partnerships between BH and public health systems or forming new ones, and they could access existing data aggregation platforms such as the National Emergency Medical Services Information System that are increasingly incorporating BH indicators into their dashboards. Public health workforce development efforts could focus on building data science, analytic, modeling, and informatics skills that specifically apply to BH.

Over the longer term, efforts to modernize public health data systems in the United States could explicitly include strategies to update core data and surveillance infrastructure for BH so that public health officials can monitor emerging BH impacts of PHEs, detect and intervene earlier on population-level BH concerns during and after PHEs, and access technology that reduces the collection and reporting burden for BH indicators.

With continued efforts to understand BH risks, needs, and impacts for the overall population as well as key subpopulations, public health action to combat the current BH crisis can be more tailored, proactive, and equitable.