Emerging healthcare interventions: Patient-Centered Outcomes Research Institute’s programmatic initiative

Abstract

The Patient-Centered Outcomes Research Institute (PCORI) is a nonprofit, nongovernmental organization established by the U.S. Congress to fund comparative clinical effectiveness research focusing on patient-centered outcomes through the engagement of stakeholders. Evaluation of emerging healthcare innovations is one of PCORI’s five National Priorities for Health. One such initiative is PCORI’s Emerging Technologies and Therapeutics Reports program, established to provide timely overviews of evidence on new drugs and other healthcare technologies. This article provides an overview of completed and ongoing Emerging Technologies and Therapeutics Reports including lessons learned to date. In addition to systematic searches, systematic selection of studies, and transparent reporting of the available evidence, informed by a select number of stakeholders (i.e., key informants), these reports focus on contextual factors shaping the diffusion of emerging technologies that are often not reported in the medical literature. This article also compares processes and methodologies of health technology assessments (HTAs) from a selected number of national and international publicly funded agencies with a goal toward potential future enhancement of PCORI’s Emerging Technologies and Therapeutics Reports program. HTAs vary considerably in terms of funding, types of assessments, the role of manufacturers, stakeholder engagement, timeline to complete from the start to the finish of a draft report publication, and communication of uncertainty for informed decision making. Future Emerging Technologies and Therapeutics Reports may focus on rapid reports to support a more expedient development of evidence. Future research could explore the role of contextual factors identified in these reports on targeted evidence generation.

Background

The Patient-Centered Outcomes Research Institute (PCORI) is a nonprofit, nongovernmental organization established by the U.S. Congress to fund patient-centered comparative clinical effectiveness research (CER) that can help a broad range of stakeholders make informed healthcare decisions and improve healthcare delivery in the United States (1). PCORI involves patients, caregivers, and the broader healthcare community across the continuum of PCORI’s work, from research topic selection to dissemination and implementation of results.

To inform future research investments in CER, from December 2018 to early 2019, PCORI launched two initiatives to monitor new and emerging interventions which may impact health care in the near term in the United States. The first initiative, PCORI’s Health Care Horizon Scanning System (HCHSS), surveils new interventions and closely monitors evidence of those with high potential for disruption in healthcare (i.e., to the current standard of care) in terms of patient outcomes, health disparities, care delivery, infrastructure, access, and/or cost (2;3). The second initiative, PCORI’s Emerging Technologies and Therapeutics Reports, is an extended application of horizon scanning, which provides broad summaries of evidence and identifies contextual issues arising from the use of new drugs, devices, and other healthcare technologies in the United States (4). More recently, PCORI’s Board of Governors approved the organization’s five new National Priorities for Health, which include the evaluation of emerging innovations in addition to existing interventions (5).

The objectives of Emerging Technologies and Therapeutics Reports are to understand benefits, unintended consequences, barriers to care, burdens and potential economic impacts, and disparities in care outcomes that may be associated with emerging interventions (4). In addition to providing timely summaries of evidence on new or emerging interventions that may disrupt health care in the United States, these reports identify gaps for future research funding opportunities and highlight additional challenges, opportunities, and uncertainties that would further inform policy makers and other stakeholders in their health-related decision making. The audience includes patients, caregivers, clinicians, health systems, policy makers, payers, and others. Globally, health technology assessment (HTA) organizations use horizon scanning and its accompanying reports to help identify candidate assessment topics and set priorities to serve their missions. While the common interest of PCORI and HTA organizations around the world is to serve their respective missions, PCORI does not assess the cost-effectiveness of technologies for reimbursement or for incorporation into care.

The purpose of this article is to examine current processes in place for PCORI’s Emerging Technologies and Therapeutics Reports and to identify areas that can advance PCORI’s mission such as incorporating patient values and patient-centered outcomes and to increase evidence through primary research funding of candidate topics. This review provides a brief description and lessons learned from completed and ongoing projects. The review examines methodologies and processes of relevant global horizon scanning programs and publicly funded HTA organizations around the world, to understand similarities and differences across programs that could serve as a source of potential future directions for our program. Although these reports are an extended application of horizon scanning, most of the processes (topic nomination through report development) and some methodologies (systematic search and selection of studies) in the initial steps mirror HTAs or rapid reports conducted by various international HTA organizations.

Introduction to HTA

HTAs assess the value of new and existing technologies in terms of safety, efficacy, and cost-effectiveness to healthcare decision makers. Generally, HTAs include diverse aspects of medical, economic, organizational, social, and ethical considerations (6). The purpose of this review is not to assess the cost-effectiveness of emerging interventions, but to understand the overall similarities and differences across different HTA organizations in terms of processes and methodologies of evaluations, and to identify areas that could potentially be incorporated into PCORI’s Emerging Technologies and Therapeutics Reports.

The definition of HTA has evolved over time. The major HTA international organizations such as the International Network of Agencies for Health Technology Assessment and Health Technology Assessment International currently define HTA “as a multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its life cycle. The purpose is to inform decision making in order to promote an equitable, efficient, and high-quality health system (7).” They also define health technology as an intervention that can prevent, diagnose, or treat medical conditions; promote health; provide rehabilitation; or organize healthcare delivery.

Overview of publicly funded national and international HTA organizations

Currently, there is no publicly funded national HTA organization in the United States. Several public institutions in the United States conduct both evidence reviews and make evidence-based recommendations, including the Advisory Committee on Immunization Practices, the U.S. Preventive Services Task Force, and the Medicare Evidence Development & Coverage Advisory Committee (MEDCAC) of the Centers for Medicare and Medicaid Services. The Agency of Healthcare Research and Quality (AHRQ) conducts evidence reviews and HTAs for the MEDCAC deliberations that evaluate the effectiveness of interventions through designated Evidence-based Practice Centers but does not make evidence-based recommendations. The Drug Effectiveness Review Project at Oregon Health and Sciences University conducts comparative effectiveness reviews to inform evidence-based coverage decision making by over a dozen state Medicaid programs. There are multiple private organizations that conduct HTA in the United States including the Institute for Clinical and Economic Review, ECRI, Hayes Inc., and Blue Cross and Blue Shield Association (BCBSA Evidence Street®).

Horizon scanning programs

Outside the United States, publicly funded HTA organizations either run their own or utilize available horizon scanning programs to inform their research investments. PCORI’s horizon scanning adapted concepts and approaches developed for the AHRQ horizon scanning system that was informed by EuroScan databases (8). PCORI continues to explore horizon scanning programs by various international organizations with an intent to incorporate their methodologies into the HCHSS and Emerging Technologies and Therapeutics Reports to meet the needs of the United States and to serve PCORI’s mission. Table 1 succinctly summarizes a selected number of programs that conduct horizon scanning around the world (9–14).

Table 1. Review of selected horizon scanning programs

Abbreviation: PCORI, Patient-Centered Outcomes Research Institute.

PCORI-funded emerging technologies and therapeutics reports

Brief background

In the United States, AHRQ created and operated the national Healthcare Horizon Scanning System to inform its Effective Health Care Program from December 2010 to December 2015. Built on concepts and approaches developed for the AHRQ horizon scanning system, PCORI initiated its HCHSS in December 2018 and defined its project scope to focus on PCORI’s high-priority conditions and topics and interventions with high potential for disruption in health care in the United States.

As an extension to its horizon scanning program, to obtain timely evidence summaries on emerging interventions and to inform its CER investments on primary research, PCORI initiated the Emerging Technologies and Therapeutics Reports in 2019. Figure 1 illustrates the general methodologies in place for these reports.

Figure 1. PCORI’s process for developing Emerging Technologies and Therapeutics Reports. This figure describes the following processes: topic scoping and development, topic refinement, evaluation, report development, stakeholder engagement in report development, and dissemination. Abbreviation: PCORI, Patient-Centered Outcomes Research Institute.

Stakeholder engagement

Stakeholders nominate the topics for Emerging Technologies and Therapeutics Reports and are engaged throughout the development of the report. A broad range of stakeholders including patients, caregivers, and other representatives offer subject matter expertise based on real-world or lived experience; explore, refine, or validate the scope of the project; and provide feedback throughout the continuum of the development of the report, raise concerns, or suggest opportunities for continued alignment of these reports with their priorities, with an intent to enhance the reports’ utility to inform healthcare decisions. These reports not only focus on the examination of clinical outcomes, but also include patient-centered outcomes deemed important, as informed by stakeholders (1).

Description of the general approach

PCORI engages stakeholders for the nomination of potential topics that have the highest potential for disruption in health care. Although there are many different definitions of disruptive innovation (15), PCORI’s HCHSS defines healthcare disruption as “disruption to the current standard of care in terms of patient outcomes, health disparities, care delivery, infrastructure, access, and/or costs (2).” PCORI’s working definition of new and emerging technologies includes “new and emerging interventions or therapies that are currently cleared by the U.S. Food and Drug Administration (FDA) for phase II/III trials, or those that may be approved in the next 1–3 years, and interventions or therapies in clinical use without the need for the FDA approval (such as off-label or repurposed for use in a particular condition or diagnostics) but lack sufficient evidence of efficacy.” After deliberations, PCORI staff conduct preliminary literature searches and narrow down eligible topics that have the highest potential for disruption, develop an initial set of guiding research questions, further evaluate topics of interest through stakeholder engagement, and contract the project report development to an external vendor for further in-depth evaluation of the topic (Figure 1). As required by PCORI, the contracted vendor utilizes systematic literature search and selection to identify relevant studies from published and gray literature sources to answer guiding research questions. The systematic literature search and selection methodology is described in detail in the appendices of each report to ensure that the yield is replicable and reproducible. During the conduct of the report, selected representatives (about 8–10 members) from different stakeholder groups with either relevant subject matter expertise or lived experience serve as key informants (KIs). KIs help to refine guiding research questions, identify relevant outcomes, and highlight relevant contextual issues in these interventions. The report includes a succinct summary of therapeutics or other technologies, a section on contextual issues identified by KIs, and a descriptive review along with evidence maps (i.e., visual charts) to depict the breadth of the available evidence. The objective of the evidence gathering is to provide a better understanding of the body of evidence supporting currently approved technologies of interest, including evidence gaps, but also to inform about those that are in the pipeline to be approved soon by the FDA. The reports undergo a technical review by selected KIs to ensure that guiding questions have been fully addressed.

The final report is publicly posted on PCORI’s Web site and is followed by the development of an issue brief and other communication and dissemination initiatives including blog posts, social media initiatives, and PCORI’s email newsletters. The issue brief succinctly summarizes the final report’s key findings in three to four pages using a lay language format and simple infographics, with an emphasis on highlighting findings that are most relevant to a range of stakeholder groups, including clinicians, patients and caregivers, researchers, health system managers, payers, and policy makers.

To date, PCORI has completed five Emerging Technologies and Therapeutics Reports, and the description of these reports is available in Table 2. The five completed reports include Landscape Review and Evidence Map of Gene Therapy as two-part reports (16;17) with two accompanying articles (18;19); Proteomic Testing in Cancer and Cardiovascular Diseases (20); Artificial Intelligence (AI) in Clinical Care (21); Genomic Sequencing to Guide Cancer Management (22); AI in the COVID-19 Response (23); and an accompanying article (24).

Table 2. Completed emerging technologies and therapeutics reports

Abbreviations: AAV, adeno-associated virus vector; AI, artificial intelligence; CAR-T, chimeric antigen receptor T cell; CRISPR, clustered regularly interspaced short palindromic repeats; NGS, next-generation genomic sequencing; RNAi, antisense, ribonucleic acid interference; ZFN, zinc finger nuclease

Reflections from evaluation of completed reports

These reports provide a means for conveying patient-centric information about an emerging innovation while supporting the identification of evidence gaps that can be used to generate new topics for future research (Figure 2). PCORI has the opportunity to develop a systematic process for nomination and working with stakeholders to establish a pipeline for future projects as well as establish various approaches to select the topics of interest. PCORI’s reports have examined multiple interventions to date. Future-focused rapid reports of single technological evaluations may yield a more expedient process for identifying evidence and addressing gaps pertaining to emerging interventions. Currently, PCORI’s Emerging Technologies and Therapeutics Reports program does not have a process in place to assess the uptake of its report findings among different stakeholders, and future efforts are needed to assess the impact of these reports on evidence generation to address gaps. Additionally, PCORI needs to develop criteria for when to update or “sunset” an existing report.

Figure 2. PCORI’s process of the evaluation of technologies and therapeutics. This figure describes the short-, medium-, and long-term products from PCORI. Abbreviations: M, million; PCORI, Patient-Centered Outcomes Research Institute.

Comparative overview of processes across HTA organizations

We provide a comparative overview of selected national and international HTA organizations, to understand similarities and differences across programs as well as to consider incorporating relevant processes and methodologies into PCORI’s Emerging Technologies and Therapeutics Reports.

We selected five publicly funded international HTA organizations that are comparable in terms of healthcare practices and demographics to which our findings will be applied, including AHRQ from the United States (25), Canadian Agency for Drugs and Technologies in Health from Canada (26), the National Institute for Health and Care Excellence (NICE) from the United Kingdom (27), Institute for Quality and Efficiency in Health Care (IQWiG) from Germany (28), and Pharmaceutical Benefits Advisory Committee from Australia (29) (Table 3). We obtained information from the methodology reported on the Web sites of these agencies and from relevant publications (30–32). The programmatic process of conducting HTA varied considerably across the reviewed bodies in terms of funding, types of assessments, the role of manufacturers, stakeholder engagement in various stages of HTA, and timeline to completion of a draft report publication.

Table 3. Comparison of the evidence assessment process of technologies and therapeutics by various publicly funded organizations

^a Fee exemptions or fee waivers for rare disease medications or those that are of public interest, respectively.

^b Applicable for the assessment of clinical effectiveness and the safety of pharmaceutical drugs that are commissioned through G-BA according to AMNOG.

^c For the assessment of all other topics (excluding pharmaceutical drugs), submission of proposals occurs via the IQWiG Web site and is conducted through the ThemenCheck Medizin (TopicCheck Medicine) program.

^d Referred to NICE using criteria reviewed by the Dept of Health.

Abbreviations: AHRQ, Agency for Healthcare Research and Quality; AMNOG, The Act on the Reform of the Market for Medical Products (Arzneimittelmarkt-Neuordnungsgesetz); CADTH, Canadian Agency for Drugs and Technologies in Health; G-BA, Gemeinsamer Bundesausschuss (Joint Federal Committee); HTA, health technology assessment; IQWiG, Institute for Quality and Efficiency in Health Care; NHS, National Health Services; NICE, National Institute of Health and Care Excellence; PBAC, Pharmaceutical Benefits Advisory Committee; SR, systematic review.

While the international HTA organizations in this review are publicly funded, a few charged a fee or an optional fee from industry or manufacturers for the evaluation of their products (26;28;29). Only two HTA organizations assessed clinical effectiveness alone (25;28), whereas the remaining three organizations assessed both clinical and cost-effectiveness. All organizations engaged manufacturers, although the level of their involvement varied across the evaluated HTAs and included patients or patient advocacy organizations as stakeholders during their evaluations. All but one HTA organization stated their topic selection criteria (29). All organizations utilized in-house staff expertise in varying capacities to conduct their evaluations or to compile their reports. All HTAs included a review period for review by one or more stakeholders, including manufacturers, payers, patient groups, and physicians and hospital representatives. AHRQ and NICE allow general public comments for their reports. IQWiG engages the public for its public HTA program (the ThemenCheck Medizin) or during deliberations of draft guidance through its supporting agency Federal Joint Committee (G-BA). The communication of uncertainty for informed decision making also varied across agencies.

Comparison of PCORI’s processes of evaluation of emerging technologies with publicly funded HTA organizations

PCORI’s Emerging Technologies and Therapeutics Reports initiative has many similarities to publicly funded organizations reviewed here. However, there are some key differences between the reviewed HTA organization programs and PCORI’s program that serve their respective missions. While most of the publicly funded HTA organization programs listed here generally focused on the effectiveness and/or value of technologies, in the role of informing coverage decisions, PCORI’s reports serve to understand the landscape of evidence to support the generation of topics for funding primary research, to inform stakeholders, and to understand the real-world issues during diffusion and aid in healthcare decision making. PCORI uses KI interviews to understand contextual issues such as regulatory factors that vary by technologies (therapeutics, diagnostics, devices, etc.), systems issues (e.g., need for specialty centers during the administration of CAR-T therapies), and patient-relevant issues (e.g., the durability of response with gene therapies and the need for re-administration).

Key considerations and challenges in evaluating emerging interventions

In summary, our review identified some strengths and limitations of PCORI’s Emerging Technologies and Therapeutics Reports and offered lessons, informed by HTA programs, to improve processes. The review identified potential ways PCORI’s reports and methods could contribute to HTA organizations. PCORI’s Emerging Technologies and Therapeutics Reports convey patient-centric information about an emerging innovation and identify evidence gaps that can be used to generate new topics for future primary research.

Even though policies and practices of engaging patients and their families during the conduct of HTAs are evolving (33), PCORI conducts research guided by patients, caregivers, patient advocacy groups, and the broader healthcare community. PCORI’s Emerging Technologies and Therapeutics Reports do not constitute a full HTA, but they do incorporate standard HTA processes from topic nomination through a final draft review and their methodologies such as systematic searches and selection. PCORI’s reports can serve as models to conceptualize and incorporate patient centricity in many ways during HTA development that could spur additional primary evidence generation. PCORI currently funds the Science of Engagement awards to further the evidence base for effective stakeholder engagement in research.

To date, PCORI’s reports have examined broader topics of multiple interventions. Rapid reports on single technologies are conducted routinely by many of the publicly funded HTA organizations that we reviewed, and this could be an approach to strengthen PCORI’s program (30). Implementing rapid reports could be important for two reasons – first, it could lead to a reduction in the time frame for report development, thereby expediting the process of identifying evidence and addressing gaps. Second, a focused review of single technologies in an expedient time frame can provide patients, their families, clinicians, and other providers with timely information that facilitates informed healthcare decision making (e.g., clinical trial participation).

PCORI’s program could be strengthened by incorporating HTA processes of topic nomination, topic selection, and technology evaluation and adapting them to the needs of patients and other stakeholders in the United States. Working with relevant stakeholders, PCORI could develop a systematic process for the nomination and selection of topics for future projects. Currently, PCORI’s Emerging Technologies and Therapeutics Reports program does not have a process in place to assess the uptake of its report findings among different stakeholders, and future efforts are needed to evaluate the impact of these reports on evidence generation to address gaps. Additionally, PCORI needs to develop a framework for when to update or “sunset” an existing report. Currently, updating a report under the Emerging Technologies and Therapeutics Reports program, PCORI relies on internal deliberations of the current relevance of key questions evaluated in the completed reports and the available new evidence.

Healthcare horizon scanning is a systematic process to identify new and emerging healthcare interventions that address unmet medical needs and have the highest potential for impact on health care. From horizon scanning, selected candidate topics would need to be evaluated if and how these new interventions can be of use, or how they should be monitored. PCORI and HTA organizations utilize healthcare horizon scanning to identify candidate assessment topics and develop accompanying reports on emerging healthcare interventions to set priorities. These reports can increase the awareness of available innovative care delivery practices, increase uptake, and further promote innovation. Horizon scanning and its accompanying emerging technology reports can adequately prepare users for future changes and can assist in policy making and in prioritizing resources by identifying important needs or gaps. For example, from PCORI’s perspective, identified gaps can inform prioritizing how resources should be allocated for future patient-centered outcomes research to increase evidence for emerging interventions. PCORI’s Emerging Technologies and Therapeutics Reports fit into the larger milieu of its broader U.S. Congressional remit of primary comparative CER and dissemination and implementation of research findings.

Conclusion and future directions

The field of healthcare innovations witnessed possibly unprecedented scientific discoveries and a rapid proliferation of emerging interventions during the COVID-19 pandemic. These healthcare innovations are being continuously evaluated and incorporated into clinical practice to support care for patients. Despite their rapid diffusion into practice, there is often a dearth of evidence in the field of emerging interventions in terms of patient-centered care, improving the individual experience of care, improving population health, and reducing healthcare costs.

In its first decade, PCORI made substantial investments in CER and the establishment of a program to support the evaluation of emerging innovations. PCORI’s stakeholders challenged us to build upon this foundation, which is acknowledged in the recently adopted PCORI’s National Priorities for Health that emphasize increasing evidence for emerging interventions.

PCORI aims to leverage its existing resources using data from diverse, real-world sources to understand the impact of emerging innovations more fully (19). While multiple technologies have been reviewed to date in PCORI’s Emerging Technologies and Therapeutics Reports, future-focused rapid reports of single novel interventions may support expedient development of evidence through primary research, and translation of evidence to practice. The recent congressional reauthorization mandates PCORI to collect the full range of patient-centered outcomes, including patient-centered economic burdens, and future Emerging Technologies and Therapeutics Reports could take these into consideration. Additionally, in the future, PCORI’s evaluation research could explore the effect of contextual factors identified in these reports on targeted evidence generation for emerging healthcare innovations.