Adapting the Evidence Academy model for virtual stakeholder engagement in a national setting during the COVID-19 pandemic

Abstract

The COVID-19 pandemic raised the importance of adaptive capacity and preparedness when engaging historically marginalized populations in research and practice. The Rapid Acceleration of Diagnostics in Underserved Populations’ COVID-19 Equity Evidence Academy Series (RADx-UP EA) is a virtual, national, interactive conference model designed to support and engage community-academic partnerships in a collaborative effort to improve practices that overcome disparities in SARS-CoV-2 testing and testing technologies. The RADx-UP EA promotes information sharing, critical reflection and discussion, and creation of translatable strategies for health equity. Staff and faculty from the RADx-UP Coordination and Data Collection Center developed three EA events with diverse geographic, racial, and ethnic representation of attendees from RADx-UP community-academic project teams: February 2021 (n = 319); November 2021 (n = 242); and September 2022 (n = 254). Each EA event included a data profile; 2-day, virtual event; event summary report; community dissemination product; and an evaluation strategy. Operational and translational delivery processes were iteratively adapted for each EA across one or more of five adaptive capacity domains: assets, knowledge and learning, social organization, flexibility, and innovation. The RADx-UP EA model can be generalized beyond RADx-UP and tailored by community and academic input to respond to local or national health emergencies.

Introduction

The COVID-19 pandemic highlighted the severity of pre-existing health disparities experienced by populations with greater morbidity and mortality from the virus (e.g., people of color, those from rural areas, and low-resourced communities) [1]. Thus, the need to address and improve health equity has never been greater [2]. Meaningful improvement requires investing in and building capacity for emergency preparedness that is adaptive and readily responsive to multiple stakeholders’ priorities, particularly those of communities most impacted by the disease.

Adaptive capacity is the condition that influences people to anticipate and respond to change, minimize consequences to moderate potential damage, recover from the consequences of stress, and take advantage of new opportunities for preparedness in times of need [3–6]. A concept often referred to and implemented in environmental or climate change research [5,6], adaptive capacity seeks to ensure that individuals, communities, and societies are actively involved in change processes, whether they be behavioral, resource-driven, or technological [5,7]. Its more recent and extended emphasis on processes includes the willingness and capacity to convert resources into effective, adaptive actions [6,8,9]. These adaptive actions emphasize the importance of resilience at the individual, community, organizational, healthcare, and government levels, where flexible infrastructural planning can enhance the ability to respond to current and future emergencies.

During the COVID-19 pandemic, the adaptive capacity process has been employed to understand and modify translational research infrastructures [10,11]. While academicians engaged in these efforts recognize the importance of strengthening [12] and enhancing communication methods [13] to mitigate the spread of COVID-19, strategies are still needed to engage communities in sharing and disseminating information. Community-engaged approaches for preventing COVID-19 allow multiple stakeholders to convene, understand, and respond in ways that drive successful and relevant research design, implementation, and translation [14–17]. The use of adaptive capacity-building, programmatic implementation strategies by academic institutions to optimize community-partnered research may lead to positive, individual-level health outcomes and behaviors as well as improved collaboration, public health planning, and interventions at an organizational level [18].

Online conferences have become an essential part of scientific communication [19], and during the COVID-19 pandemic, they have offered a safe platform for facilitating inclusive participation in community-based research [20–22]. While barriers such as lack of access to computers or the Internet, fewer opportunities for social interaction, and online fatigue, may exist [23,24], online conferences can be more accessible for low-resourced populations that must consider travel or budget restrictions, and for people with disabilities, family responsibilities, and health concerns [24], thus diversifying representation and enhancing collaboration across divers groups [25]. In a pandemic such as COVID-19, online conferences should be included as a form of engagement so that diverse stakeholders can readily and safely convene to collectively develop and implement contextually appropriate solutions for rapid impact.

We applied an adaptive capacity framework when designing and implementing the Evidence Academy (EA) as a national conference model for virtually engaging community and academic partners to share the science and practice of mitigating COVID-19 morbidity and mortality through testing and testing technologies. Part of the larger National Institute of Health (NIH)-sponsored Rapid Acceleration in Diagnostics in Underserved Populations (RADx-UP) initiative [14], we modified the EA infrastructure over time to rapidly respond to the changing needs of the EA implementers, attendees, and broader RADx-UP initiative. In this paper, we detail the structure of the RADx-UP EA conference model and its adaptive capacity framework, summarize attendee feedback and lessons learned, share how we adapted and implemented the event series, and provide implications for broad-scale dissemination.

Methods

The RADx-UP Initiative: Brief Overview

In response to the COVID-19 pandemic, the NIH launched the RADx initiative to speed innovation and implementation of SARS-CoV-2 testing technology [26]. Under this initiative, the RADx-UP program aims to ensure all Americans have access to COVID-19 testing, with a focus on communities most affected by the COVID-19 pandemic [14]. The RADx-UP Coordination and Data Collection Center (CDCC) supports over 125 community-academic project teams with education, training, implementation guidance, and coordination. The CDCC also coordinates the RADx-UP EA providing an opportunity for community-academic projects to convene, share, and disseminate information [14,26].

The RADx-UP Equity Evidence Academy (EA) Virtual Conference Model

The EA model, created in 2010, was initially designed as an interactive, in-person, regional conference to foster critical reflection and active discussion among key stakeholders around a health topic of interest. An EA event is structured to facilitate knowledge transfer, as well as collaborative identification of action steps by attendees in order to drive meaningful change [27,28]. Its format is comprised of a data profile, which is an epidemiologic summary of the health topic of interest for the regional area, followed by a day of data-driven presentations, breakout sessions, and roundtable discussions. This model may be a viable tool for large-scale, rapid dissemination of information, and collective decision-making and action planning to improve translation of research and ultimately community health.

The RADx-UP EA conference series is the first national, virtual adaptation of the EA model. Its goal is to provide an iterative, engaged conference approach that rapidly creates a shared understanding of SARS-CoV-2 testing technologies, community-engaged practices, implementation strategies, and related factors such as vaccination to advance equity in vulnerable and underserved populations. Attendees include an invited audience of RADx-UP community-academic project teams, CDCC leadership and staff, the EA Steering Committee, and leaders from the NIH.

Each EA cycle includes seven fundamental components (Fig. 1). The EA Core Team, representing staff from the CDCC, communicates with RADx-UP leadership to identify the topic that will drive the event content. The team then conducts a review of the scientific literature on the topic and generates a set of cross-cutting themes. At this time, the team also selects and convenes the EA national Steering Committee, comprised of recognized community and academic experts who assist with shaping the themes and identifying content experts to present at the EA. Prior to the EA, a data profile is created with community and academic partner input. This resource aims to provide a foundational understanding of the event’s topic and themes amongst event attendees; it is shared electronically in English and Spanish prior to the event and is published on the RADx-UP website [29]. Contents of the data profile include a table and map of all of the current RADx-UP projects, summaries of current evidence around a set of cross-cutting themes, and in-context interviews with community and academic experts to complement the thematic summaries. The EA’s content and structure are developed with input from RADx-UP CDCC staff, the EA Steering Committee, and EA core planning team and are designed to fit within the parameters of the virtual event platform. The event is delivered over the course of two days, for roughly four hours each day (see a detailed agenda in Table 1). Core components for each event include opening and closing keynote addresses; thematic breakout sessions; roundtable discussions of this information centered on identifying recommendations for action; real-time report-outs of key takeaways; opportunities for attendee networking; and evaluation surveys during and at the end of the EA event to receive input for future improvement and adaptation.

Fig. 1. The RADx-UP Equity Evidence Academy (EA) cycle.

Table 1. RADx-UP Equity Evidence Academy (EA) Sample Agenda*

*Times listed in Eastern Time (ET). The time of day and number of days for each RADx-UP EA event were selected to accommodate the five time zones of the attendees (HT, PT, MT, CT, and ET).

Following the event, a set of dissemination products are created. First, an EA summary report [30,31] is developed and distributed in English and Spanish, highlighting elements of the event, such as keynote presentations, scientific and practical evidence generated from the breakout sessions, key takeaways gleaned from event presentations and discussions, and next steps to consider. The report also includes virtual illustration posters, graphically capturing the content from the keynote presentations and breakout sessions. The Core Team and other CDCC partners work with community partners to develop and implement a second dissemination product tailored to the needs and preferences of the RADx-UP project team community partners. Finally, the Core Team gathers feedback from different stakeholders (project team community partners, CDCC staff) and, in conjunction with accessing evaluation results, identifies lessons learned and adaptive strategies for the next EA event. Then this seven-part cycle is repeated for the next EA.

The RADx-UP EA Adaptive Capacity Framework

The adaptive capacity framework for the RADx-UP EA is derived from two approaches described in climate change preparedness literature – a framework for analyzing adaptive capacity at the local level, and an approach for building adaptive capacity in tropical communities (Fig. 2). The Local Adaptive Capacity (LAC) framework [5] focuses on understanding the process of how a local system adapts during climate change interventions rather than only identifying the existing resources available to the system. This approach recognizes the importance of intangible processes in adaptation, such as how decision-making and governance occur at the community or household level, not just the national level. The LAC’s five domains include assets, innovation, social organization, flexibility, and knowledge and learning (Fig. 2).

Fig. 2. The RADx-UP Equity Evidence Academy (EA) adaptive capacity framework.* *The five domains of adaptive capacity for the RADx-UP Evidence Academy Series are adapted from two frameworks – the Local Adaptive Capacity framework by Jones et al. [5] and the framework for addressing adaptive capacity to climate change in tropical coast communities by Cinner et al. [6]. The RADx-UP EA System comprises (1) the EA Core Team: the lead team of 8-10 staff from the RADx-UP CDCC responsible for the planning, implementation, and day-to-day operations of the RADx-UP EA; (2) the RADx-UP CDCC Network: the research community in which the Core Team operates, including the leadership (principal investigators, and administrative lead); (3) the Community Engagement Core Leads: staff who offer guidance and support of the design of the EA events, identify resources, and present at the EA events; (4) the Engagement Impact Teams (EITs): staff who provide an opportunity for researchers, health system leaders, health care professionals, community members, and policymakers to discuss and share lessons learned and develop implementable strategies across the RADx-UP community-academic-partnered research projects through an equity lens; (5) the Communications Team: a team that shares news, updates, and resources, and offers guidance on website content; (6) the Evaluation Team: a team that assists with the design of the assessment tools, data analysis, and reports; and volunteers, who are faculty, staff, and community partners across the CDCC who provide input during the EA design and assist the day of the EA event); (7) the EA Steering Committee: a body of 12-16 community and academic experts selected who convene three times before the EA event to help shape the themes, identify speakers, and deliver general input on the EA design; (8) the RADx-UP Community-Academic Project Teams: members of the 125+ project teams receiving funding and other CDCC resources to conduct projects on COVID-19 testing and testing technologies; (9) NIH leaders: NIH institute directors and project officers that support RADx-UP and may present at the EA events; and (10) Presenters: content expert presenters from across the USA who are selected to deliver EA presentations.

Assets are key resources that allow the system to respond to evolving circumstances. The RADx-UP EA system is comprised of principal investigators, Steering Committee, administrative leads, and staff members. This network within which the EA Core Team operates provides high-level guidance, operational, communication, and evaluation support, and connections between the CDCC and the projects. In this dynamic system, the RADx-UP website and the virtual platform used to deliver the conference events also represent the available assets.

Innovation is fostered by systems that create an environment for experimentation, and the ability to explore unique solutions or take advantage of new opportunities. The EA Core Team works within the system, adapting and responding to deliver the RADx-UP EA in the most efficient manner possible. One of the specific domains relevant to building adaptive capacity is social organization [6].

Social organization is the way people are organized to enable cooperation, collective action, and knowledge sharing. Individuals, organizations, and communities with social capital can provide access to novel sources of information and resources, giving people voice in adaptation planning. For the RADx-UP EA, this may include input from the EA Steering Committee, the CDCC, and the EA Core Team on content-specific presenters and the organic communication network between the representative groups within the system.

Both the framework by Jones et al. [5], and approach by Cinner et al. [6], describe the importance of flexibility in adaptive capacity. Flexibility is the ability to switch between adaptation strategies and anticipate, incorporate, and respond to changes. This includes rapidly identifying and adopting changes within and between EA events to foster greater knowledge and learning (Fig. 2).

Knowledge [5] is the ability to collect, analyze, and disseminate information in support of adaption activities, and learning ⁶ reflects people’s capacity to generate, absorb, and process new information in uncertainty. For this framework, both knowledge and learning are necessary, as the morbidity and mortality of COVID-19, and technology to address the pandemic continues to change.

Adaptive Capacity Assessment

Using the RADx-UP EA framework as a guide, we assessed the adaptive capacity of our EA Core Team in the design and delivery of the EA and described the topics, attendees, operational- and delivery-specific lessons learned, and the adaptive actions taken based on the framework’s five characteristics. This approach was one of our evaluation tools to help modify our implementation across the three EA events conducted.

Results

EA Topics and Themes

RADx-UP EA events’ core topics evolved across the three EAs to align with current, national issues surrounding the pandemic. EA1 was held in February 2021 and centered on building a general understanding of SARS-CoV-2 testing and testing technologies. The next EA involved bridging infrastructures to address the systemic factors impacting vulnerable communities experiencing the pandemic (EA2, November 2021), followed by promoting COVID-19-related messaging accuracy and accessibility to accurate information (EA3, September 2022). Across all EAs, communication strategies and information accessibility were addressed as important elements related to health equity. For each EA event, there were six, cross-cutting themes. Attendees first engaged in knowledge transfer of the current evidence in these thematic areas, then co-created practical solutions to put the information discussed into action. Speakers and attendees were encouraged to collaborate as a learning community during the events by sharing examples, models, and recommendations across the discussed themes.

RADx-UP EA Attendees

The number of registered attendees across the three EA events was 319, 242, and 254, respectively (data not shown). The majority of attendees (75–80%) identified as female. Among those who responded, roughly 45% were White, 25% were Black or African American, 20% were Latino, 10% were Asian, and 5% were American Indian/Alaska Native. Greater variation across the three events occurred for Native Hawaiian/Pacific Islander, which increased from 1 to 3 and 4% over time. Nearly 40 out of 53 states, territories, and Tribal communities were represented across the three EA events.

Feedback and Lessons Learned

Table 2 summarizes attendee feedback and operational lessons learned by the EA Core Team from one event to the next. Attendee-reported strengths for EA1 included the agility of the virtual platform, quality of the speakers and themes, opportunities provided for attendees to connect with others, and overall flow of the event. The increased diversity of the speakers and topics from EA1 to EA2 was recognized strengths. After EA1, attendees reported needing more assistance using the virtual platform, more opportunities for attendee interaction, and the desire for more community representation among selected speakers. After EA2, attendees reported technological challenges with the virtual platform. CDCC staff who were volunteers at the EA2 identified the need for more effective coordination of information and expectations of roles (e.g., more involvement in the planning meetings to better understand the EA structure and consistency in volunteer facilitation during the EA event). Needs identified through attendee evaluation data were increased communication and networking during the event and improved development of community dissemination products.

Table 2. RADx-UP Equity Evidence Academy (EA) feedback, lessons learned, and modified activities by adaptive capacity domain

*Modified approaches include (a) internal operational changes to meet the functional needs of executing the EAs and/or (b) external translational changes to meet the networking and information communication needs in delivering the EAs.

Application of the RADx-UP EA Series Adaptive Capacity Framework

Table 2 also summarizes the adaptive actions taken to design and deliver each RADx-UP EA event as it evolved from the model to a national, virtual event. Across all five components of the adaptive capacity framework, we modified our activities or built our capacity to broaden reach, improve communication, increase representation, streamline processes, increase connectivity, and/or increase responsiveness. Much of the investment was in identifying and activating assets (1a–1d) and creating a structure for promoting knowledge and learning at a national level (5a–5d). EA1 was informed by leveraging the CDCC’s diverse staff and leadership, with expertise in community engagement, epidemiology, testing, policy, project management, evaluation, and communication. The CDCC’s network and the EA Steering Committee created access to recognized keynote and breakout speakers, and the CDCC staff provided access to facilitators, reporters, and notetakers serving as volunteers to fulfill needed roles during the event. The virtual platform vendor, which had experience planning and executing similarly sized virtual conferences, was secured. Adaptations in knowledge and learning (5a-5d) included EA themes with a national focus, recognized and experienced keynote speakers, availability of presentations in English and Spanish, a post-EA information dissemination webinar for community partners, and an EA post-summary. One of the more significant changes was the adaptation of the data profile (2a). We changed the data profile from a regional epidemiologic data source to a resource of evidence-based information on the key themes for the event that is relevant to diverse stakeholders across the United States; this was complemented by contextual stories from community experts and academic researchers.

In the transition from EA1 to EA2, we continued to use our assets to enhance our capacity in knowledge and learning (5d–5i) by addressing the current thematic evidence on building infrastructures to address COVID-19 testing and testing technology. In addition, within the social organization domain (3d-3f), we continued to receive input from the CDCC. Using that feedback and EA1 attendee feedback, we included RADx-UP project information in the data profile, increased the racial, ethnic, and geographic diversity of the speakers, and featured speaker panels during the EA2 breakout sessions that provided more opportunities for audience exposure to multiple perspectives. In demonstrating our flexibility (4b–4c), we were nimble in modifying the agenda by reducing the number of speakers to create more time for audience question and answer and more opportunity to connect through meet and greet sessions (3g).

From EA2 to EA3, we changed our virtual platform vendor (1g) to offer attendees more advanced networking tools such as participant profiles, a social wall, and one-to-one video call and chat capabilities between participants. The platform also provided a central hub for accessing content post-event and opportunities for comments and discussion threads to be posted by attendees. We amended the agenda by conducting thematic roundtable discussions immediately after the breakout sessions on Day 1 to encourage greater continuity and recall of content. To respond to requests for learning around community engagement strategies across RADx-UP, we developed the inaugural Advancing Community and Academic Partnerships presentation series, designed to understand and share best practices around community engagement methods in COVID-19 community-partnered research.

Discussion

Our RADx-UP EA Core Team and the CDCC network adapted a regional, in-person, interactive conference model to successfully design, deliver, and disseminate a national, virtual EA conference series model. Rooted in the principles of community engagement [14], we leveraged collaborative approaches to bring together community and academic stakeholders to collectively learn, discuss, and generate recommendations for addressing SARS-CoV-2 testing and testing technologies in populations of greatest impact.

Unlike the original, in-person regional EA conference [27,28], we fostered more in-depth discussion of recommended actions during the event, and more rapidly generated and disseminated products (e.g., toolkits and community webinars) for advancing the engaged science and practice on COVID-19 testing and testing technologies. Specifically, each EA built on the prior EA as the RADx-UP community learned more about the science of the pandemic. We engaged the attendees in developing real-time, broad, thematic recommendations that can be readily adaptable to the specific needs and circumstances of the diversely represented community-academic partnerships. Furthermore, we disseminated a menu of products from the EA events, including graphics, recordings, content summaries, a community partner-focused webinar, and other website material that is available for use by community and academic partners.

Assessing the adaptive capacity of our EA Core Team required us to understand the processes and resources necessary to improve the delivery of the EA events. Through the application of our RADx-UP EA Series Adaptive Capacity Framework (Fig. 2), we enlisted and received support from our multi-sectored RADx-UP CDCC network of stakeholders, with expertise in laboratory, clinical, and community-engaged research. Through the RADx-UP CDCC’s infrastructure [14], we relied on the domains of our adaptive capacity framework to iteratively implement EA1-EA3, using community-engaged approaches identified by Volkov and colleagues [10] as necessary for integrating community partners in the research and leadership process. As a result, we fostered an environment where community partners could authentically participate in the EA process, which included serving on the EA Steering Committee to assist with the design of the series; contributing in-context interviews for and expert reviews of the data profiles; presenting as keynote or breakout session speakers; contributing to the recommendations for advancing the science and practice; presenting projects on COVID-19 testing via oral and poster presentations on community-engaged research methods (rather than research outcomes); and disseminating information to other community partners.

Among the greatest lessons learned was the strength and limitations of the use of online technology. Our RADx-UP EA series required extensive technological and human resources to facilitate a highly interactive event, unique from traditional online conferences. While we received positive feedback from many about the opportunity to learn and connect with colleagues, technological problems occurred and presented challenges that inhibited connectivity between and movement of attendees from one virtual session to another. Adapting the agenda to meet the attendee’s needs to share with others was extremely important to creating space for more in-depth discussions.

The virtual, national conference model adapted for the RADx-UP Equity EA Series is an innovative approach to convening community and academic partners to understand health disparities in SARS-CoV-2 testing. Given that the COVID-19 pandemic is transitioning to an endemic phase and that it has prompted a cultural shift where virtual meetings are now more universally accepted as a norm, it is likely that the demand for adaptive virtual conference models will continue to grow. The EA model offers a replicable framework that other research consortiums can use to engage diverse stakeholders while also providing enough flexibility that planning teams may introduce and refine new and innovative strategies for building attendee connections in a virtual setting. This adaptive strategy can be scaled based on the purpose, reach, and available resources to enhance emergency preparedness and knowledge sharing at any level.