Objectives/Goals: The demands on MICHR’s Evaluation team are profuse and varied. Quarterly team meetings were used to keep track projects, identify new projects, and relay important new initiatives from MICHR leadership. The MICHR Translational Innovation team took on the task of assessing the Evaluation team’s processes to design better workflow and effectiveness. Methods/Study Population: The process included 5 stages, Empathize, Define, Ideate, Prototype and Test. Sixteen interviews were conducted with MICHR faculty and staff. Interviews were coded and summarized. Seventeen themes were mapped and distilled into 5 key insights. From the key insights, design principles were identified to guide a design session with Translational Innovation staff and Evaluation staff. New work processes were proposed, designed, and tested by both teams. The Evaluation team “test-drove” the prototype and iterative design sessions were conducted to determine which new elements were successful. The Evaluation team was positioned to begin utilizing the newly designed process at the beginning of MICHR’s new grant year. Results/Anticipated Results: The MICHR Evaluation team is instrumental to the development, conduct, and dissemination of Clinical & Translational Science (CTS), a primary objective of MICHR’s work. Three types of evaluation projects were identified through the design process: required reporting, CQI/program improvement, and CTS/impact evaluation. The service design process enabled the Evaluation team, and MICHR program leads to better identify and prioritize collaborations between the Evaluation and program teams that improved the quantity and quality of MICHR CTS outputs. Discussion/Significance of Impact: Generating CTS is critical to the missions of NCATS and MICHR. Thoughtfully designing processes that facilitate and increase CTS output that can be shared and duplicated across the consortium is invaluable.

Objectives/Goals: In alignment with the Clinical and Translational Science Awards (CTSA) UM1, continuous quality improvement (CQI) needs to be integrated into the elements and hub evaluation. As a first step to operationalizing this process, the University of Cincinnati hub (CCTST) evaluation team developed a systematic approach to prioritizing and sequencing tasks for aligning evaluation methods with CQI. Methods/Study Population: A scoring sheet was developed to provide a framework for prioritizing and developing the sequence and timeline for supporting elements in aligning their evaluation methods with CQI. The scoring sheet assessed: 1) impact – defined as the results from completing the task and implementing enhancements; 2) effort – defined as the amount of resources (time, personnel, and materials) needed to complete the tasks; 3) reach – defined as number of individuals (e.g., CTSA employees, members, researchers, trainees, and community members) impacted by the tasks or products of the tasks; 4) urgency – defined as a task that is time-sensitive due to a deadline and has a clear consequence if not completed on time. Each component was assessed using a 3-point scale (e.g., minimal, moderate, and high). Results/Anticipated Results: In Fall 2024, the CCTST evaluation team met with respective elements to collect data on their: 1) need from the evaluation team to support aligning their evaluation methods with CQI, 2) challenges and barriers to improving evaluation and aligning with CQI methods, 3) number of hours per month available to support improving evaluation methods, and 4) current resources to dedicated to conducting an evaluation. Next, the evaluation team will transcribe the data from the meetings and code the data into the scoring sheet for each element. The scoring sheet is anticipated to produce a score that will be used to develop the sequence, timeline, and initial tasks for supporting elements in improving evaluation methods and aligning with CQI over the first year of the UM1. Discussion/Significance of Impact: CTSA hubs’ evaluation teams operating at full capacity may encounter barriers to implementing CQI efforts. This systematic approach – assessing the impact, effort, reach, and urgency to sequence evaluation and CQI alignment – can support evaluation teams by ensuring a balanced workload and optimizing operations for quality improvement.

The Mountain West Clinical & Translational Research Infrastructure Network (MW CTR-IN), hosted at the University of Nevada, Las Vegas, and supported by the National Institute of General Medical Sciences at the National Institutes of Health, began as a partnership among 13 major U.S. public universities across 7 MW Institutional Development Award (IDeA) States, stretched across 1/3rd of the U.S. land mass and encompassed almost 1/3rd of all IDeA States. The mission of the MW CTR-IN is to build and enhance infrastructure capacity to increase CTR in the MW region. This case study describes the Clinical Pilot Projects Program (CP3) processes and tools used to support this objective through its selection of applications to fund, supporting applicants and funded investigators, and providing guidance and oversight during the funding period. The MW CTR-IN has funded 152 single-investigator pilot projects, 7 multisite pilot projects, 13 developmental team grant projects, and 14 community-engaged projects. These projects have also led to over $92M in extramural grant funding, 308 presentations, and 1,124 peer-reviewed publications. The methodologies and expertise we gained can assist other CTR networks in developing efficient pilot project programs that have been evaluated and demonstrated to improve CTR initiatives, especially through the use of a custom portal.

Objectives/Goals: We operationalized our evaluation using the RE-AIM framework, by defining its dimensions as nodes within a net effect diagram in the form of radar charts to visually display group variation among nodes. This enhanced our ability to measure the reach, effectiveness, and implementation of our efforts in under-resourced Los Angeles communities. Methods/Study Population: We applied the RE-AIM framework to guide the operationalization of evaluation methods, defining the RE-AIM dimensions generally with a focus on reach, effectiveness, and implementation. We developed and defined a standardized scoring system for metrics that contributed to the RE-AIM dimensions of focus, using data from our activities such as health education workshops targeting diverse, under-resourced populations in Los Angeles. Our standardized scoring system ranged from 1 to 5, reflecting the degree of success within each metric/dimension. Scores were mapped in net effect diagrams in the form of radar charts to enable comparative analysis and visualization, highlighting a variety of grouped variables (i.e., language, locations, and adaptation). Results/Anticipated Results: The operationalized and developed scoring system allowed us to standardize assessment across the RE-AIM dimensions, making it possible to visualize our impact through net effect diagrams. These diagrams illustrated variations in reach, effectiveness, and implementation across different community engagement activities stratified by group variables, providing insights into our impact and areas for adjustment. Preliminary results suggest that the net effect diagram effectively captures both broad and nuanced impacts and serves as a viable application of the RE-AIM framework. The use of standardized scoring enhances data comparability and offers a dynamic visual tool for monitoring ongoing and future initiatives while serving as a tool to display and report our impact. Discussion/Significance of Impact: Operationalizing evaluation with the RE-AIM framework and implementing a standardized scoring system allows us to visualize and monitor effectiveness in real time. This system supports data-driven decision-making for our sustainable, impactful community engagement initiatives ultimately contributing to our goal of improving health equity.

Intentionally enhancing and supporting the early careers of individuals from populations underrepresented in science and medicine (URSM) is essential to achieving health equity. The Health Equity Leadership and Mentoring (HELM) Program at the University of Minnesota and the University of Utah is designed to foster academic excellence and build leadership capacity of postdoctoral fellows, clinical fellows, and early-career faculty who identify as URSM and/or who are committed to careers in health equity research and clinical care. HELM models a culture of psychosocial safety to create a sense of belonging and uses evidence-based and culturally aware mentoring and career development strategies with the goal of retaining diverse faculty. HELM proved agile and adaptive during the Covid-19 pandemic and has been successful in states with and without legislation limiting diversity programs. Across the 2 institutions, the HELM program has supported over 200 trainees and early-career faculty through mid-2024. Among HELM participants who joined the program as faculty, 85%–95% have remained in their faculty positions.

Objectives/Goals: Evaluate use and reuse of data in biomedical fields to explore how data advances translational benefit for science and patients. Methods/Study Population: In this study, we use the metadata for 5 million data citations in the Data Citation Corpus as a source to analyze the use of datasets from biomedical fields across a number of facets, including date, affiliations, and funders. We identify examples of datasets that have been used across different research fields by analyzing the disciplinary scopes of the datasets and the publications that cite them and explore patterns of reuse of datasets beyond the original research project from which it was created. Results/Anticipated Results: The analysis will provide insights into datasets showing broad usage in biomedical fields, the patterns of data use over time, and the affiliations and funders associated with highly used datasets. The analysis will also dive into examples of datasets showing use across different fields and usage beyond the original project. The results will showcase examples of valuable datasets in translational research and serve as a basis for exploring different considerations around the impact of open datasets. Discussion/Significance of Impact: The exploration of data citations will provide insights into open data showing translational reach and an example of how the Data Citation Corpus can support evaluation of the impact of research data.

Objectives/Goals: To identify and characterize future potentially high impact research generated by the Clinical and Translational Science Awards (CTSA) Program, we evaluated the Altmetric Attention Scores (AAS) of recent articles associated with the Program and conducted an initial assessment of the attributes associated with high AAS for Program articles. Methods/Study Population: We used the NIH Query, View, Report (QVR) tool to identify recently-published scientific papers that cited support from a CTSA Program grant. The unique publications identified by QVR were used to construct an Altmetric Explorer report. We examined the relationship between the AAS and other variables, including number of authors, number of grants supporting the publication, number of CTSA program institutions supporting the publication, and if the publication included group authorship. Results/Anticipated Results: Our analyses confirmed that the Program indeed supports potentially high impact research, as indicated by the highest scoring papers, across a wide range of diseases and conditions. Nearly all the highest scoring papers were focused on a specific disease or condition rather than broader methodological research, despite the disease-agnostic focus of the CTSA program. We also found that the Program significantly contributed to critical research on the once-in-a-century COVID-19 pandemic. We confirmed the entire CTSA consortium is contributing to potentially high impact research, with all institutions represented in the highest scoring publications. Discussion/Significance of Impact: Understanding the impact of the CTSA Program presents a unique challenge – the program supports biomedical research infrastructure and training programs whose outcomes and impact can be difficult to track or measure. These data offer early signals of impact and can assist evaluators with designing future evaluations.

Objectives/Goals: Posttraumatic stress disorder (PTSD) is common during pregnancy and postpartum, leading to adverse birth outcomes. Despite effective interventions like narrative exposure therapy, PTSD often goes untreated due limited training opportunities and lack of community support. Expanding training for PTSD is crucial to improving access to care. Methods/Study Population: Six 3-day PNET trainings were delivered to 57 participants over a 23-month period. Workshop attendees represented a variety of professions (19% Social Workers, 19% Mental Health Graduate Trainees, 18% Psychologists, 18% Counselors, 12% Doulas, 11% Physicians, and 5% Home Visitor/Parent Educators) with varying levels of specialty experience from diverse locations (2 countries and 13 states). Key workshop outcomes included participant one-week post-workshop satisfaction, perceptions of acceptability, appropriateness, and feasibility of the intervention, and pre- to one-week post-workshop perceptions of connectedness to trauma treatment and perinatal healthcare communities. Data will be explored at 6 months post-workshop to evaluate longer-term effects on connectedness. Results/Anticipated Results: The majority of workshop attendees (84%, M =  4.76, range 1–5) reported being “extremely satisfied” with the training and 98% indicated they would “recommend it to others.” Most attendees found NET to be acceptable (M  =  4.64, range  =  1–5), appropriate (M  =  4.37, range  =  1–5), and feasible (M  =  4.49, range  =  1–5) to use within their practice. Paired t-tests revealed a significant increase in a sense of connectedness to both the trauma treatment and perinatal healthcare communities from pre- to post-workshop. Discussion/Significance of Impact: Findings indicate that the PNET workshop is feasible and effective in training interdisciplinary providers on perinatal PTSD evidence-based interventions. By training a range of professionals and fostering a sense of connectedness, the PNET workshop has the potential to make effective trauma treatments accessible to underserved populations.

We deliberated a case report of seven cases to investigate whether inhibitors of tumor necrosis factor-α (TNF-) could reduce pregnancy dangers caused by antiphospholipid syndrome (APS). Patient 1 was diagnosed with NC-OAPS and Hashimoto, Patient 3 was with SN-APS and Hashimoto, Patient 2, 3, 4 were with SN-APS, Patient 5 and 6 were with OAPS, and Patient 7 was with OAPS and PCOS. Patient 4 took the longest period to report the disappearance of symptoms (7 days), followed by patients 1 and 5, and lastly, 2, 3, 6 and 7; after treatment, TNF-α decreased to varying degrees in 7 patients, among which Patient 1, 3, 6, 7 reached the ideal level (< 8.1) and Patient 5 reached the highest level (123.04); Patient 6 and 7 were ongoing pregnancies. The fetuses were born to the desired gestational age except the fetus from Patient 1. A total of 5 patients underwent cesarean delivery. The average height of the newborns was 48.20 cm and the average weight was 2.50 kg. The Apgar scores ranged between 8 and 10. The ongoing pregnancies as a limitation of the dataset. Collectively, we found that TNF-α Inhibitors could prolong gestational period.

Objectives/Goals: Electronic health record (EHR)-based recruitment can facilitate participation in clinical trials, but is not a panacea to trial accrual challenges. We conducted a root cause analysis to identify EHR-based accrual barriers and facilitators in a pragmatic randomized trial of metformin for those with prostate cancer and glucose intolerance. Methods/Study Population: We quantitatively analyzed enrollment drop-offs among eligible patients who either did not complete a consent (with analysis of EHR-embedded consent process) or who completed a consent but were not enrolled (with analysis of EHR implementation of a Best Practice Alert). We summarized data from the EHR by eligibility, provider encounters, and alerts, and generated CONSORT diagrams and tables to trace the enrollment pathway. We supplemented quantitative findings with a thematic analysis of semi-structured individual interviews with eligible patients (n = 10) and study providers (n = 4) to identify systematic barriers to recruitment and enrollment of eligible patients. Results/Anticipated Results: CONSORT diagram analysis found that 24% of potentially eligible patients (268 of 1130) had an eligible study encounter but were not enrolled. Additionally, BPAs were not triggering for some eligible patients. Interviews revealed that study providers wanted more detailed information about which study arm their patient would be assigned to, and about next steps after enrollment, especially relating to additional lab tests and follow-up care needed. Patient interviews suggested that patients often did not remember completing the consent process and felt overwhelmed with appointments and information; patients expected providers to actively bring up research opportunities during appointments. Discussion/Significance of Impact: While pragmatic EHR-embedded trials are often characterized as lower-burden, these trials still require active engagement by providers, as well as ongoing attention from both research and informatics teams to ensure that EHR-embedded processes are functioning as designed, and that they are effective in recruiting study participants.

Academic health centers (AHC) characterized by an integrated mission serving education, research and clinical care reflect these values in the institution’s vision, decision-making and culture. Embracing this strategy, the University of Alabama at Birmingham (UAB) created a novel, competitive funding initiative through its faculty practice with the Health Services Foundation General Endowment Fund (HSF-GEF). This partnership with the faculty practice leveraged faculty and staff creativity to inform and lead capacity-building and innovation in patient-oriented and laboratory research, clinical care development and education aligned with the best interests of the enterprise. Since 1996, the HSF-GEF has invested over $66M in 442 peer-reviewed proposals led by transdisciplinary teams representing strategic advances with strong potential to generate future extramural support, to improve healthcare delivery, to enhance research capacity and to promote active learning. Beyond financial return on investment, program evaluation revealed benefit on culture, collaboration, camaraderie and infrastructure. By engaging the broad workforce to articulate, select and implement projects, UAB has fostered a purpose-driven culture of collaboration within the AHC that thrives on broad representation, enthusiasm, and ingenuity as well as peer engagement across multiple schools in the academic community.

Objectives/Goals: The University of Illinois Chicago’s Center for Clinical and Translational Science has implemented an innovative approach to program evaluation. We blend high-impact quantitative and in-depth qualitative approaches to identify local and national impacts and areas for improvement that are not captured solely by traditional quantitative methods. Methods/Study Population: CCTS manages service requests and investigator demographics through an in-house system that our evaluation program utilizes to report on service requests, investigator satisfaction, and investigator demographics to service groups, CTSA and campus leadership, and other stakeholders throughout the year. Through this system, we are able to regularly survey and interview investigators about their experiences and solicit feedback about the service process. During interviews, we focus on questions about receiving services, recommendations for CCTS and colleagues, and plans to work with CCTS in the future. This mixed-methods approach helps us lay the foundation to expand evaluation beyond reporting and establish a robust CQI program that focuses both on CCTS staff needs and improving investigator experiences. Results/Anticipated Results: Soliciting both quantitative and qualitative feedback from investigators has enabled our service groups to make significant changes to their internal processes to ensure that investigators are aware of services and supports available. Our quantitative data show us that investigators return time and again to CCTS for services and supports. Yet the feedback we receive through short, targeted interviews also helps identify challenges that investigators experience that could improve the services they come to us to receive. We have already used this system to recommend improved marketing of existing services within certain service groups that were highly requested by investigators, which increased utilization of that service. Discussion/Significance of Impact: Our mixed-method approach to evaluation allows us to easily and rapidly identify areas for improvement within our service groups, an instrumental part of implementing a CQI program that is focused on staff-identified areas of improvement. This approach can be easily replicated by other CTSA hubs with minimal impact to existing resources.

Objectives/Goals: Dissemination and implementation scientists often conflate or confuse pragmatic trials and effectiveness-implementation trial designs. This study evaluates the barriers and facilitators affecting these scientists’ collaborative work to design, plan for, and conduct these different kinds of trials. Methods/Study Population: This is a sequential mixed-methods study. For the quantitative evaluation secondary data collection and surveys of roughly 200 investigators constituting an Implementation Science Network were carried out to identify research needs and impacts associated with the Translational Science Benefits Model. Surveys were prepopulated with respondents’ grant awards and prompts to define the study designs being used. Interviews of respondents are being conducted to identify barriers and challenges they faced in conducting different implementation trials and to develop case studies of their resultant research agendas. A peer-reviewed interview protocol designed for Clinical and Translational Research Institutes to conduct case studies of translational research is being used for this qualitative evaluation. Results/Anticipated Results: The 182 ISN members submitted 1590 research proposals since 2020, 52% of which were funded. ISN members responding to surveys (N = 30) self-identified many of these studies as being Hybrid 3 (29%), Hybrid 1 (17%), or Pragmatic trials (7%), although the largest proportion included studies classified “other” (33%), and some could not be classified (12%). Surveys of ISN members also indicated that many want to conduct pragmatic trials (36%) or hybrid trials (8%) but need more opportunities to collaborate (19%). Twelve (40%) ISN members agreed to be interviewed and another 11 (37%) indicated that they would do so in fall 2024 if available. Initial findings suggest that regular interactions with colleagues helped investigators new to the field understand how varied study designs could advance their implementation science. Discussion/Significance of Impact: These findings will show how U-M implementation scientists collaborate to conduct implementation trials. If the kinds of barriers faced by investigators differs by trial type, research supports and initiatives can be tailored to better support all implementation scientists in the CTSA Consortium.

Objectives/Goals: As translational science (TS) emerges as a field, there is a need for research organizations to understand how to develop capacity for and support the advancement of TS. To support such institutional and infrastructural change, this poster outlines a Translational Science Promotion and Research Capacity (T-SPARC) framework. Methods/Study Population: The T-SPARC framework was developed by members of the Duke University Clinical and Translational Science Institute (CTSI) primarily from CTSI Pilots, Team Science, Evaluation, and Administration, all of whom had identified the need for building institutional capacity for TS at our institution. The group reviewed literature on TS to ensure grounding in current knowledge, drafted an initial TS logic model, and then determined the value of developing a framework addressing building TS institutional capacity. The group then identified other frameworks/models related to behavioral, organizational, and system change; examined scholarship addressing the building of research capacity in colleges and universities; and iterated on a TS-focused framework in multiple working sessions. Results/Anticipated Results: The resultant T-SPARC framework provides a foundation to 1) inform the development of interventions and programs advancing TS and 2) evaluate their effectiveness. It outlines: organizational levels for TS capacity building (large-scale systems, research institutions, teams, and individuals); intervention activities (policies and processes, funding, collaboration and partnership, and training); proximal outcomes (knowledge/attitudes, behaviors, resources/infrastructure, and connections); next-stage outcomes (e.g., interdisciplinary team processes, and research infrastructure); and ultimate goals (fewer translational impediments, improved public health, and health equity). It ingrates TS principles as foundational to, and outcomes of, capacity-building efforts. Discussion/Significance of Impact: T-SPARC, as a framework for building capacity in TS, provides added foundation for advancing the conceptualization and practice of TS. Ultimately, T-SPARC seeks to advance broader goals of reducing longstanding challenges in the translational research process and improving health outcomes.