Introduction
Community-engaged research (CEnR) involves community stakeholders and researchers collaborating to co-design, implement, evaluate, and disseminate clinical and translational research that is culturally appropriate, valued, efficient, and effective [1]. CEnR is an important avenue for both researchers and community stakeholders to generate and disseminate knowledge in order to improve the health and well-being of patients and communities [Reference Andrews, Newman and Meadows2, Reference Rubio, Schoenbaum and Lee3]. There has been increased interest among academic institutions and community stakeholders to conduct CEnR. Supported by the National Institutes of Health, the Clinical and Translational Science Award (CTSA) program currently funds 64 institutions across the USA to further the progress of translational research [4]. One of the primary goals of the CTSA program is to foster CEnR by engaging patients and communities in every phase of the translational science and research processes [5, Reference DiGirolamo, Geller and Tendulkar6].
Establishing successful community–academic partnerships in research requires training, skills, and resources. With community engagement as a core component of all CTSA institutions, much of the work done across CTSAs in CEnR involves the development, testing, and dissemination of trainings and resources to enhance CEnR skills and knowledge among researcher and stakeholder populations, with the end goal of ensuring more successful community–academic partnerships. For more communities and researchers to effectively collaborate in CEnR, increasing access to quality CEnR education and training is crucial [1]. Therefore, the availability of resources online that are readily accessible to researchers and communities and contain accurate, up-to-date content is an essential part of the continued success of CEnR [Reference D’Alonzo7]. To this point, there is a focus by the National Center for Advancing Translational Sciences (NCATS) on shared resources and information to help advance CEnR [8]. However, the extent to which existing CEnR resources are publicly accessible online through CTSAs, and how these resources address CEnR core competencies is unclear.
The purpose of this study was to understand the extent to which CTSA programs offer publicly accessible online resources for training in CEnR core competencies. In this study, the term CEnR was used to refer to all forms of related research (e.g., community-based participatory research) [1]. The study purpose was addressed by examining the following aims: (1) identify all CEnR resources currently offered by CTSAs, with a focus on publicly accessible online resources and (2) systematically review publicly accessible online CEnR resources for whether resource content addressed identified CEnR core competency domains.
Materials and Methods
Study Design
Literature specific to CEnR education and training competencies was used to identify CEnR core competency domains. In April 2018, a search of PubMed was performed using the following search terms: community-based participatory research, community engagement, and competency. We also searched references cited in selected articles. An article was included if the primary focus was discussing competencies for training and performing CEnR. A total of eight articles were included for the final review. Common competencies and key attributes discussed in these articles were synthesized to identify core domains based on the literature [Reference Andrews, Newman and Meadows2, Reference DiGirolamo, Geller and Tendulkar6, 8–Reference Shea, Young and Powell13]. Definitions and key aspects of domains were then agreed upon by research team members. The final eight CEnR core competency domains included: knowledge and perceptions of CEnR; personal traits necessary for CEnR; knowledge and relationships with communities; training of those involved in CEnR; CEnR methods; CEnR program evaluation; resource sharing and communication; and dissemination and advocacy (Table 1).
Table 1. CEnR core competency domain definitions and characteristics
CEnR, community-engaged research.
We then conducted a systematic review of CEnR resources publicly accessible online between July 2018 and May 2019 from the websites of all 64 currently funded CTSAs to identify CEnR-specific resources. Resource characteristics were recorded including institution name, link to resource, ability to download resource, intended audience for resource, and target population of interest for a resource. Resources were included for review and analysis if all components of the resource were fully available online; they were primarily intended for education or training in performing CEnR; they were directly developed, co-developed, or sponsored by a CTSA; and they were directly available through a CTSA website. A CEnR resource was determined to be publicly accessible online if all components of the resource were available online. Resources were excluded if the website used to access the resource required the study team to request information or permission from the host CTSA in order to access the resource, and that access was not received within 5 days; the resource required in-person training, or blended online components with off-line formats, such as in-person training; or, the resource was not intended for CEnR education and training.
Overall, 647 CEnR resources available between July 2018 and May 2019 from 64 CTSAs were identified. Of those, 376 resources were identified as being publicly accessible online. A total of 214 of 647 (33.1%) publicly accessible online CEnR resources from 35 of 64 (54.7%) CTSAs met the criteria for inclusion in review (Fig. 1) [14-48]. Most resources reviewed were in the text (i.e., downloadable literature, PowerPoints, etc.), webinar, or recorded lecture format.
Fig. 1. Flowchart for the inclusion of publicly accessible online resources for CEnR.
CEnR, community-engaged research; CTSA, Clinical and Translational Science Award.
For each resource, the CEnR core competency domains addressed were evaluated. For each resource included in the final sample, the content was independently reviewed by two reviewers and subsequently scored by the extent to which it addressed each CEnR core competency domain using a dichotomous score as follows: (1) Addressed – the domain was the main topic discussed in the resource, or was substantively discussed, but was not the main focus of the resource or (2) Not addressed – minimally discussed (i.e., only discussed in a few sentences or bullet points) or not discussed at all. Reviewers’ domain scores for the same resource across reviewers were then compared. To resolve scoring discrepancies, reviewers of the resource convened to reach an agreement on the final domain score.
Finally, reviewers were asked to keep field notes regarding observations made in accessing and reviewing resources. These notes were then compared and discussed by the study team to determine what, if any, common themes were noted among reviewers.
IRB approval for this study was not required.
Statistical Analysis
Domain scores for study sample resources were aggregated and analyzed with descriptive statistics (frequencies and percentages) to assess the extent to which each of the eight identified CEnR core competency domains was addressed across resources.
Results
Of the 214 resources included for review, 60% (n = 130) were directly downloadable documents. The remaining 40% required additional steps to access the resource (i.e., a video played on YouTube or an interactive module that could not be downloaded) and the resource itself was “housed” online.
The intended audiences for the resources reviewed included community partners (33%; n = 71), academic partners (62%; n = 134), and clinician partners (8%; n = 18). For 13% (n = 29) of resources, the intended audience was unclear. About 12% (n = 27) of resources discussed the conduct of CEnR with specific communities or populations, including Latinx communities, African-Americans, and military veterans.
The CEnR core competency domains most commonly addressed in the study sample of resources were “CEnR methods” (50.5%; n = 108) and “Knowledge and relationships with communities” (40.2%; n = 86). The CEnR core competency domains least commonly addressed in the study sample of resources were “Dissemination and advocacy” (11.2%; n = 24) and “CEnR program evaluation” (12.1%; n = 26). Further details regarding the percentage of core domains addressed across all sample resources are provided in Fig. 2.
Fig. 2. Percentage of core domains addressed across all sample resources by domain.
CEnR, community-engaged research.
Comparison of reviewers’ field notes revealed that a lack of standardized CEnR nomenclature became problematic when reviewing and scoring resources. Scoring discrepancies for at least one domain within a resource initially occurred in 50.9% (n = 109) of resources. All scoring discrepancies were resolved. Of the resources in which scoring discrepancies initially occurred, 71.6% (n = 78) were ultimately determined to address the domain in question, while 28.4% (n = 31) were ultimately determined to not address the domain in question. The core domains with the most observed initial scoring discrepancies included “Knowledge and perceptions of CEnR” (20.2%; n = 22) and “Personal traits necessary for CEnR” (18.3%; n = 20). The core domains with the fewest initial scoring discrepancies were “CEnR program evaluation” (3.7%; n = 4) and “Dissemination and advocacy” (9.2%; n = 10). Further details regarding initial reviewer discrepancies in scoring the extent to which domains were addressed within resources are provided in Fig. 3.
Fig. 3. Initial reviewer discrepancies scoring the extent to which domains were addressed within resources by percentage.
CEnR, community-engaged research.
Additionally, the comparison of reviewers’ field notes revealed CTSA website navigability played a major role in excluding potential resources from the analysis. Issues related to the navigability of CTSA websites that were noted by reviewers included: inconsistent organization and nomenclature related to labeling CEnR resources, convoluted hyperlinks and website organization to access CEnR resources, and restricted access to CEnR resources (e.g., needed faculty/staff credentials to access resources).
Discussion
Engagement of communities across the continuum of translational research is vital in ensuring that such research meaningfully improves the health and well-being of those communities [49]. To accomplish this, community and academic partners must understand all aspects of CEnR so that community partners are able to understand and actively contribute to all phases of the research process. Education and training in every domain of CEnR core competency are then essential for both communities and researchers to ensure that all stakeholders are competent and comfortable contributing to CEnR.
Through this study, we generated a comprehensive inventory of CEnR resources that are publicly accessible online to researchers and other stakeholders interested in learning about and conducting CEnR. Our comprehensive review of these resources elucidated the CEnR core competency domains that are addressed in currently available resources. Results suggest that CTSAs provide numerous resources specific to CEnR through online formats. “CEnR methods” and “Knowledge and relationships with communities” were the most commonly addressed domains by these online resources. Our findings also indicate that major gaps exist in resources addressing the domains of “CEnR program evaluation” and “Dissemination and advocacy.” Furthermore, the domains of “Knowledge and perceptions of CEnR” and “Personal traits necessary for CEnR” were difficult to initially score with consistency. Finally, considering that only 33.1% of CEnR resources reviewed were publicly accessible online, and those resources only came from 54.7% of CTSAs, this demonstrates that there is a need for CTSAs to substantially improve public access to online CEnR resources.
Several challenges pertaining to accessing and classifying publicly accessible online CEnR resources were noted. First, the original source of the materials was not always clear to the reviewer. While many tools were created by the institution hosting them on their website, some resources were shared from other sources and thus redundant between sites. In the future, it will be useful for tools to be identified with the date of creation and source institution(s) so stakeholders can identify content developers and content modifications to facilitate adaptation and dissemination of CEnR resources. Second, we encountered challenges in successfully navigating CTSA websites to access CEnR resources. Marked differences were observed in how CTSA websites were structured to house and allow access to CEnR resources which led to challenges in navigation. Third, several CTSA websites offered CEnR resources by providing “hyperlinks to hyperlinks,” where instead of the resource being directly available through the CTSA website, one would have to follow several hyperlinks in order to locate the original resource of interest. Even among our trained reviewers and study team, efforts to systematically follow these hyperlinks became time-consuming and confusing. In order to maximize the efficient use of online CEnR resources, clear guidelines for formatting access to materials on CTSA sites to be shared by multiple stakeholders are recommended to ensure minimal navigability issues for all potential stakeholders. Areas of website navigability have been previously identified, and include: “the clarity with which the target of a navigational element is described by that element (clarity of target), the clarity with which a navigational element conveys the underlying structure of site information (clarity of structure), and the degree to which the site content is appropriately subdivided or hierarchically organized with respect to the relationships between the content sections (logic of structure).” [Reference Wojdynski and Kalyanaraman50].
Finally, as our team reviewed resources, we found that the CEnR core competency domains we identified based on our review of CEnR competency literature did not adequately cover resource content concerning social determinants of health, cultural competency, and other aspects of community diversity essential to successful CEnR. The potential for CEnR to meaningfully account for the important roles of social determinants of health and cultural competency in healthcare research has long been recognized [Reference Seifer and Sisco51, Reference Wallerstein and Duran52]. The NCATS strategic plan emphasizes that translational research efforts should include an additional study of clinical implementation and social determinants of health [8]. Therefore, future CEnR resources need to include standardized nomenclature and training materials with a domain and associated competencies specific to social determinants of health and cultural competency.
Inconsistencies in the language used to define and describe various aspects and competencies related to CEnR was a limiting feature of resources included in this study. More than half of the resources had at least one domain scoring discrepancy between reviewers after the initial review, which suggested some challenges related to domain nomenclature. Due to the lack of standardized nomenclature to describe key CEnR concepts, it was challenging for reviewers to compare CEnR resources between CTSAs to ensure all similar resources were being captured in the study sample. This also created difficulty in identifying the CEnR core competency domains addressed within each resource, leading to many of the initial scoring discrepancies observed. Inconsistent language makes it difficult for stakeholders to search for relevant resources and difficult for novice CEnR stakeholders to understand key terminology and concepts. For example, we found that a variety of terms were used to refer to community partners (e.g., community stakeholders, patient partners, experience ambassadors, community advisors, etc.). Such a variety of terms and descriptions could be confusing to academic and community partners seeking to understand how community partners should be educated and participate CEnR projects. In a recent study, Eder and colleagues (2018) surveyed key informants at CTSAs in an effort to understand the consistency of CEnR definitions used among participating institutions. Their findings indicate there is a distinct need in CEnR for clearly delineated, specialized nomenclature in order to accurately describe and perform CEnR [Reference Eder, Evans and Funes53]. A standardized set of terminology and curricula of the core skills and resources are needed for CEnR to facilitate addressing gaps in CEnR and improving this body of knowledge [Reference Faridi, Grunbaum and Gray9]. Our findings also indicate that efforts to harmonize language in this area of research will have a high yield for the efficiency and effectiveness of future CEnR. As CTSAs attempt to leverage the use of existing resources and develop new ones, the standardization of CEnR language and concepts will be essential, and will align with the strategic plan of the NCATS to streamline the development of best practices, and facilitate more sharing and collaboration in translational science [8].
Current CEnR literature also does not describe how institutional factors, such as mentorship and resource availability to foster community outreach and engagement, can affect the development of community engagement. This is relevant to the creation of CEnR resources because universities that have a strong foundation of commitment to CEnR might emphasize competencies differently or develop curriculum differently than institutions with a weaker foundation [Reference Parker, Margolis, Eng and Henríquez-Roldán11]. The competencies needed for successful CEnR should describe specific attitudes, knowledge, and behaviors that a researcher must have to achieve both the goals of CEnR and of the partnership between the researcher and the community. Determining gaps in competency coverage of available resources can help address training opportunities for researchers and community partners prior to entering a partnership [Reference Chari, Matthews and Blumenthal54]. Much of the literature from our review recommends building a conceptual framework that identifies key competencies of CEnR that can be mapped to specific domains of knowledge [Reference Shea, Young and Powell13]. To the co-authors’ knowledge, this is the first study to synthesize recognized components of CEnR competency into such defined domains. Identifying and defining core competency domains from the relevant literature will help in standardizing CEnR terminology for common use. There are additional efforts underway to identify core domains of CEnR competency using differing methodology, including a recent multi-institutional report from the Joint Working Group [Reference DiGirolamo, Geller and Tendulkar6, Reference Faridi, Grunbaum and Gray9, Reference Parker, Margolis, Eng and Henríquez-Roldán11, Reference Richmond, Strelnick and Tilson55]. Future CEnR research must synthesize such reports in a concerted effort to standardize CEnR competencies.
Some CTSAs provide a variety of outstanding CEnR education and training resources for community, academic, and clinical stakeholders. For example, the University of California San Francisco provided access to a variety of lectures and written materials intended for wide dissemination and use in education and training in CEnR [34]. The University of Illinois at Chicago also provided a variety of written training materials for CEnR, as well as several resources for working with specific vulnerable populations [36]. However, such quality was not uniform across resources, and there was great variation observed in the content, quality, and formats of the CEnR resources reviewed. Such variation presents stakeholders with difficulties in determining whether CEnR resources from CTSAs could be adapted for use in education and training in their potential CEnR projects. Identifying standard expectations for CEnR education and training resources to achieve would aid stakeholders in determining whether a resource could be utilized for their purposes. Therefore, there would be a benefit in developing grading criteria to help stakeholders assess the practical utility and adaptability of available CEnR resources. Future research should develop tools that help stakeholders evaluate the utility and adaptability of CEnR resources and should incorporate key aspects of CEnR resources observed in this study: content as it relates to core domains of CEnR competencies, intended audience, format, and accessibility.
The findings of this study should be interpreted while considering the following limitations. While the CEnR core competency domains used in this study were identified and defined using the expert consensus of the study team members, other CEnR experts may have identified and defined domains differently. Despite the interest in performing CEnR and the wealth of CEnR resources available from CTSAs, there is limited data and published literature related to the core competencies necessary for a successful scholarship of CEnR, as well as extreme variation in the methods and quality of literature concerning CEnR [Reference DiGirolamo, Geller and Tendulkar6, Reference D’Alonzo7]. Additional CEnR research is needed to establish universal CEnR core competency domains and their definitions. Since this study focused on the resources developed and provided by CTSAs, resources developed or provided by non-CTSA organizations that support CEnR education and training were not assessed. Finally, this study focused on the content of CEnR resources, and not the effectiveness of resources in providing adequate education and training. Future studies should examine the extent to which the intended audiences of these resources are able to successfully perform CEnR to fully assess the value of resources for use in CEnR education and training.
Conclusion
This study suggests that CEnR resources are abundantly available online from CTSAs. Many available resources addressed at least one of the CEnR core competency domains identified by our study team. However, the lack of clear organizational structure within CTSA websites and standardized CEnR competency nomenclature render those resource platforms difficult to navigate, even for CEnR experts. Variability in the language used to describe essential components of CEnR is likely to be a major barrier to standardizing education and training for community stakeholders and researchers participating in CEnR. Future work should focus on consensus building toward a standardized nomenclature and guidelines to described CEnR competencies, and on incorporating competencies related to social determinants of health into CEnR education and training.
In order to build and sustain abilities to share resources for training and education in CEnR across CTSAs, these efforts are necessary. This study elucidated gaps in CEnR education and training that must be addressed. Gaps were identified not only in the content related to competencies in publicly accessible online CEnR resources, but also in the alignment of online resource access to ensure ready to access these invaluable resources generated by CTSAs. This study, combined with the knowledge generated through expert consensus panels, may be a platform on which to develop future resources, create standardized CEnR nomenclature, and bring forth the potential development of a comprehensive, systematically organized repository of tools specific to CEnR.
Acknowledgments
R.J. Piasecki received support from the Predoctoral Fellowship in Interdisciplinary Training in Cardiovascular Health Research (T32 NR012704), the Philip D. Raso Scholarship provided by Nurses Educational Funds, Inc., and the Ruth L. Kirschstein Predoctoral Individual National Research Service Award (1F31NR018362-01A1).
E. Quarles, A. Bilheimer, and L. Carter-Edwards received support through the North Carolina Translational and Clinical Science Institute (NC TraCS). NC TraCS at the University of North Carolina at Chapel Hill is funded by the National Center for Advancing Translational Sciences through the Clinical and Translational Science Awards (CTSA) program (UL1TR002489).
C.R. Dennison Himmelfarb received support through the Johns Hopkins Institute for Clinical and Translational Research (ICTR). The Johns Hopkins ICTR is funded by the National Center for Advancing Translational Sciences through the Clinical and Translational Science Awards program (UL1TR003098 and UL1TR001079).
Disclosures
The authors have no conflicts of interest to declare.
Open access
