Value and Implementation of Patient and Public Involvement and Engagement in Health Technology Assessment for Japan: implications from systematic searches

Abstract

Objectives

This study comprehensively reviewed reports on patient and public involvement and engagement (PPIE) in health technology assessment (HTA) overseas and identified the status and possible future measures, of PPIE in Japanese HTA.

Methods

The series of reviews targeted systematic reviews (SR#1), references in SR#1 (SR#2), and new articles after SR#1 (SR#3). The MEDLINE and Embase databases were searched through August 2024 using the terms “patient involvement/engagement,” “patient participation,” “community participation,” “public involvement/engagement,” and “health technology assessment.” The implementation details were extracted from information published on the websites of the HTA agencies.

Results

Three review articles in SR#1, 12 references in SR#2, and 10 articles in SR#3 were selected. The opportunities for countries, including Japan, to participate in discussions on the HTA process did not differ significantly; however, information on PPIE in Japan was scarce and did not indicate their purpose and value.

Conclusions

Collected articles indicated that the value of PPIE in HTA includes relevance, equity, fairness, legitimacy, and capacity building. The participation of patient and public representatives in Japanese discussions since 2005 appeared to be very limited to consider PPIE in HTA. In countries that implement PPIE in HTA, the value of PPIE is explicit: the process guidelines are specific and provide an appropriate environment for input that includes education, training, and feedback. Future reforms of the Japanese system will require discussions on PPIE purpose and value, implementation, and creating an environment in which a diverse range of patients and the public can easily express their views.

Background

Health technology assessment (HTA) schemes were first employed in the United States (1;2) and have been institutionalized in Australia and Canada (1;3;4). After its institutionalization in the United Kingdom (5), HTA has been introduced in various countries in Europe and the Asian region (1). Patient and public involvement and engagement (PPIE) has been implemented in clinical research (6;7), clinical practice guideline development (7;8), and decision-making in various areas of healthcare, with PPIE in HTA taking a variety of forms (1;2;9).

In Japan, pharmaceuticals and medical devices approved by the regulatory authority have been covered by public healthcare insurance without consideration of cost-effectiveness for a long time. The prices are set based on predetermined rules, with proposed prices that have been agreed upon in discussions between companies and the Ministry of Health, Labour, and Welfare (MHLW) and then authorized by the Central Social Insurance Medical Council (Chuikyo) (10,11). A system for the evaluation of the cost-effectiveness of healthcare technologies (Japanese HTA) was introduced in 2019. Japanese HTA is designed to complement the existing system (that is on, to adjust the price once it has been decided using the results of cost-effectiveness evaluations), and the products and price adjustment ranges of the determined price, and they are finally approved by the General Assembly in Chuikyo (12–14). Patient and citizen representatives have been participating in various committees in Chuikyo since the 2005 reform of the system in response to the bribery scandal (15–17). These representatives participate as members of the Expert Committee on Cost-Effectiveness Evaluation (CEE) (Expert Committee of CEE), which deliberates on individual items; the Special Committee of CEE (Special Committee of CEE), which examines the system’s form; and the General Assembly, which approves discussions at Expert Committee and Special Committee of CEEs. (18,19).

Currently, Chuikyo mainly comprises three parties: the payer, who bears the cost of medical care; the medical practitioner, who provides medical care; and the public interest, which coordinates between the two. The General Assembly of Chuikyo includes seven members representing the insured (including patient harmed by an adverse drug or medical accident and citizen/worker representatives), insurers, and employers (payer); seven members representing medical practitioners (including five doctors, dentists, and pharmacists); six members specializing in law, politics, economics, and so forth (public interest); and four expert members from medical professionals not included in the medical practitioners. The personnel structure of the Special Committee of CEE consists of six representatives of the payer (including patient and citizen representatives), six representatives of the medical practitioners, four public interest representatives, and four pharmaceutical and medical device industry representatives. The personnel of the Expert Committee of CEE include experts in medical economics, clinical medicine, medical statistics, medical ethics and law, as well as patient and citizen representatives (17 – 19).

However, the Japanese HTA system is in its infancy, and available information is fragmented, making it difficult to understand how PPIE is being implemented. Owing to the deteriorating fiscal situation in Japan, an expanded scope of the HTA is currently being considered (20), and the value of the PPIE in the HTA system is expected to increase.

PPIE in HTA agencies has been examined from the perspectives of the value of involvement (21–23), methods of involvement (2; 21;24), and resources for involvement (25). The present review synthesizes the findings from existing studies and several country agency documents on PPIE in HTA to confirm PPIE value, implementation, and need and identify measures needed for the future of PPIE in Japan.

Objective

By organizing the value, implementation methods, and necessary resources for PPIE in HTA, this study will clarify the current state of PPIE in the Japanese HTA system and identify the measures and resources required to improve the system.

Methods

Information on PPIE value, implementation, and required resources was extracted through umbrella and subsequent reviews. Details of the implementation, which were difficult to ascertain from the literature, were extracted in a predefined format from the public information available from the websites of HTA agencies. The differences between Japan and other countries were examined by adding Japanese information to the extracted data.

Umbrella and scoping reviews

Search method

Umbrella review : The MEDLINE and Embase databases were systematically searched to identify systematic reviews published up to 31 August 2024. The search terms were: “patient participation,” “patient involvement,” “patient engagement,” “community participation,” “public involvement,” “public engagement,” and “health technology assessment.”

Scoping review : A complementary literature search of the MEDLINE and Embase databases was also performed to identify original articles or documents published after the umbrella review (between 1 January 2020 and 31 August 2024). The search terms were “patient participation,” “patient involvement,” “patient engagement,” “community participation,” “public involvement,” “public engagement,” “technology assessment,” “the Pharmaceutical Benefits Advisory Committee (PBAC),” “the Canadian Agency for Drugs and Technologies in Health (CADTH),” and “the National Institute for Health and Clinical Excellence (NICE).” As 96 percent of the articles from the umbrella review search yielded not-HTA-eligible articles, the scoping review employed “technology assessment” as a single phrase in the search, supplemented by abbreviations for the three target agencies.

Article selection

Articles on PPIE in the HTA that met the following criteria were included:

Umbrella review : The included articles were required to be reviews of original articles published from countries belonging to the Organization for Economic Cooperation and Development or in Asia. Study designs, including guidelines, qualitative studies, case studies, and cross-sectional surveys, were excluded.

Scoping review : The included articles were required to be original articles or documents (that is on guidelines, qualitative studies, case studies, and cross-sectional surveys) published in Australia, the United Kingdom, or Canada.

The exclusion criteria were nontargeted outcomes, including PPIE on patient-reported outcomes (patient preferences and patient experiences), PPIE on medical procedures not covered by the Japanese HTA system (that is on vaccinations, tests, fertility treatments), or PPIE focused on ethical issues, such as efficiency and equity of healthcare; local-level HTA; publication style (that is on editorial, commentary, and discussion); and scope (prioritization not considered in the Japanese HTA).

The umbrella review included reviews obtained directly from the database (primary screening articles) and the references of the selected reviews. The references (secondary screening articles) were reviewed using the same criteria as in the scoping review and added to the article set.

Data extraction and integration

The following contents were extracted from the articles from the umbrella (primary and secondary screening) and scoping reviews: (i) author, (ii) publication year, (iii) country, (iv) study type, (v) main theme, (vi) participants (patient or public), (vii) PPIE values, (viii) PPIE implementation (target HTA stages and participation type), and (ix) resources (guidelines, education and training, information sharing or dissemination from agencies, actions for patients and public inputs, and feedback or process evaluation). Items (i)–(vi) were mandatory, while (vii)–(ix) were collected if available.

Website search

Search methods

The search target was information from the websites of three HTA agencies: PBAC in Australia, CADTH in Canada, and NICE in the United Kingdom, as well as the Centre for Health Economics and Evaluation (C2H) of the National Institute of Health and Medical Sciences in Japan as documents on the evaluation/deliberation process on the website of Chuikyo in Japan. This information was updated on 31 August 2024.

Information integration

The descriptions of PPIE in the assessment/deliberation process were consolidated into the following items based on previous articles (2;21): institution name, role (recommendation/resolution), PPIE participants, PPIE-related statement, PPIE for conducting HTA; PPIE on review assessment results and the formulation of recommendations (or reports for drug price adjustments in Japan); PPIE in the dissemination of decision or assessment results; and internal evaluation/feedback.

In the United Kingdom, NICE reviews and formulates recommendations and makes decisions on reimbursement. In Australia and Canada, the roles of HTA agencies (PBAC and CADTH) are limited to reviewing and formulating recommendations, with decisions being made by a separate agency (Table 2). As the drugs to be evaluated are selected in Japan based on predefined selection criteria (13), PPIE information from the subsequent process was collected and integrated in this review.

Results

The search for the umbrella review identified 3,220 candidate articles, from which three reviews were finally extracted (primary screening). A total of 54 references were obtained from the three extracted reviews. Most of these 54 references referred to information from Australia, Canada, and the United Kingdom. Therefore, the scope was narrowed to articles from these three countries, and 12 articles were included in the review (secondary screening).

The scoping review targeted articles from three agencies in Australia (PBAC), Canada (CADTH), and the United Kingdom (NICE) from 2020 onward because Gagnon et al. (25), which was selected in the umbrella review, comprehensively reviewed reports up to 2020. The search for scoping reviews identified 1,984 candidate articles; however, among these, only 10 articles were included. The sequences of these searches are presented in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) chart in Figure 1.

Figure 1. PRISMA flow diagram.

PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analyses.

The articles selected for the series of reviews included qualitative studies (9 articles) and others (16 articles), with the latter comprising both multicountry studies (five articles in total), including reviews and guidance development, and single-country studies (11 articles in total). Table 1 provides an overview of the five articles on multicountry reviews and guidance development. Reviews from single countries and qualitative studies are outlined in the Supplementary Appendix. Information on the HTA process and PPIE in different countries is presented in Table 2 and 3.

Table 1. Outline of the multinational literature review

Abbreviations: CADTH: Canadian Agency for Drugs and Technologies in Health, EUnetHTA: European Network for Health Technology Assessment, EUPATI: European Patients’ Academy on Therapeutic Innovation, GRIPP: Guidance for Reporting Involvement of Patients and the Public, HTA: health technology assessment, HTAi: HTA international, INAHTA: International Network of Agencies for Health Technology Assessment, ISPOR: International Society for Pharmacoeconomics and Outcomes Research, n/a: not applicable, orgs: organizations, PPI: patient and public involvement, RedETS: Spanish Network of Health Technology Assessment

Table 2. Outline of the Web search for information on PPIE in HTA in four countries

Abbreviations: HTA: health technology assessment, n/a: not applicable, PPIE: patient and public involvement and engagement.

* Academics representing the public interest participate as individuals. They are involved in the adjudication and therefore do not comment on other occasions as a rule in Japan.

^** Citizen is a representative of a labor organization.

Table 3. Outline of the Web search on PPIE at each stage of HTA

Abbreviations: C2H: Center for Health Economics and Evaluation of the National Institute of Health and Medical Sciences; CADTH: Canadian Agency for Drugs and Technologies in Health; Chuikyo: Central Social Insurance Medical Council; HTA: health technology assessment; n/a: not applicable; NICE: National Institute for Health and Clinical Excellence, PPIE: patient and public involvement and engagement.

The following section describes the findings from the information obtained on PPIE value, implementation, and required resources.

PPIE value.

Based on research reports by patient groups (26), the value of patient involvement is relevance, equity, fairness, legitimacy, and capacity building (27). HTA agencies have reported that PPIE ensures legitimacy and fairness in resource allocation decisions (28) and provides a background to the technology under evaluation (29). Pinho-Gomes et al. summarized the value of PPI in representation, transparency, relevance, equity, fairness, and reconciling different types of knowledge. (21). These PPIE values are posted as policies or statements on the website of each agency in Australia (30), Canada (31), and the United Kingdom (32). In contrast, Chuikyo’s website in Japan lacks such statements, and it was not until 2021 that PPIE-related comments were confirmed in the minutes of the meetings.

PPIE implementation (target stage/form of participation):

The results of this review showed that all stages of HTA are subject to PPIE. More specifically, this includes technology identification and prioritization, scoping, evaluation by companies, review by HTA agencies, dissemination of decision or HTA findings, and evaluation of the PPIE process (21;22;25;26;33;34). The details are in Table 2. Guidelines and reports from HTA agencies in countries other than Japan confirm the involvement of patients and the public in each process.

Gagnon et al. (25) broadly classified the forms of involvement as (i) direct participation of patients and the public in decision-making processes (that is on meetings) and (ii) provision of patient and public perspectives through documents (that is on indirect participation). The unit of participation (individual or collective) and the collection of views vary depending on the agency; however, both individuals and groups generally can express their views (25;33;34). Under the Japanese HTA, the views of patients and the public can be reflected in meetings by representatives of patients and labor organizations. Academia, who represents the public interest, participates as an individual and is responsible for coordinating between the payer and the medical practitioner; thus, their opinions on such coordination are limited to what is necessary. (15;16). Only a few mentions regarding PPI were identified in the minutes.

Resources required for PPIE.

The resources required to implement PPIE are time, material, human, and financial (25;34). Based on the feasibility in Japan, this review summarizes the resources needed, including guidelines, education and training, information sharing and dissemination from facilities, information from patients and the public, and process evaluation.

(a) Guidelines.

Many of the articles included in the present review referred to PPIE in the HTA process guidelines of Australia (35), Canada (36;37), and the United Kingdom (38;39), or guidelines by the European Patients’ Academy on Therapeutic Innovation (EUPATI) (27), the European Network for Health Technology Assessment (EUnetHTA) (40), the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), and Health Technology Assessment International (HTAi) (33). However, Japan has no such guidelines on the process itself nor the PPIE, and is limited to addressing technical issues of evaluation.

(b) Education/training.

Low et al. (41) comprehensively summarized information regarding education/training programs in PPIE implementation provided by the EUPATI, ISPOR, HTAi, and HTA agencies. The authors reported that accessibility, inclusiveness, transparency, and interpersonal relationships and committee dynamics (33;42) required consideration and that tailored support was important (27;43;44). Moreover, education and training is also needed for participants, including researchers, staff, HTA reviewers, and committee members (21;27). No information on education/training by relevant Japanese institutions could be found.

(c) Information from institutions.

Much of the information on the implementation of HTA in countries implementing PPIE is shared collectively (45). In addition, simplified versions are also provided when disseminating the evaluation results to the public (27;34). Through this process, PPIE is considered to contribute to promoting the understanding of the HTA system (25). In Japan, relevant information is provided separately by the three meetings of the Chuikyo (that is the General Assembly (46), the Expert Committee of CEE (47), and the Special Committee of CEE (48) and by the C2H (49–51), which compiles official analysis reports. However, only some of the minutes of the Expert Committee CEE meetings have been made public, and no lay summaries have been seen for the reports.

(d) Patient and public input.

A range of opportunities and support should be available to provide information to patients and the public. Regarding patient and public responses, guidances (25;27;41) on information for patients and the public (Summary of Information for Patients) and templates for patient input (52–54) are provided. However, collating the collected opinions and using them in discussions and reports remain a challenge, even in other countries (21;26;55–57). As noted in ‘PPIE implementation’, the relevant institutions in Japan do not have a mechanism for receiving input directly from patients and the public specifically regarding the evaluation of cost-effectiveness.

(e) Process evaluation and feedback.

The PPIE process can be evaluated through qualitative research, such as interviews (21;25;33;34;55), quantitative evaluations, such as questionnaires (22;26;29;33), and reports. The research is conducted by external evaluators (22;43;55–57) and HTA agencies themselves (39;58) and some provide feedback (59).

Discussion

Based on the reports of PPIE in HTA in other countries, the current PPIE in Japan and the measures and resources needed to improve it are described below.

PPIE purpose and value.

PPIE values in HTA generally include relevance, equity, legitimacy, and capacity building, although these differ between countries (21;27). The websites of HTA agencies other than that in Japan consolidated information on PPIE and indicated its purpose and value. Although patients and citizens participate in Chuikyo meetings in Japan, no information on PPIE is provided in the related websites.

Three years after the introduction of HTA in Japan in 2019, a reference to patient involvement was finally confirmed in the minutes of Special Committee of CEE regarding the revision of the system in 2021 (60). In that meeting, patient and citizen representatives questioned whether there was a possibility of future involvement in the form of patients providing input for appraisals, as are done in other countries. The secretariat of the Committee replied, “the purpose of patient participation had not been specifically discussed in the committee before and that it was an issue for the future.” In response, patient and citizen representatives indicated that the background of patient involvement and engagement included factors of consideration for “patient discrimination and people with disabilities.” During the final stages of institutionalization, various issues were discussed regarding “how to deal with factors other than cost-effectiveness” (61), however, PPIE did not appear in the discussions.

Since the 2005 reform, patient and citizen representatives have participated in discussions at Chuikyo to reflect the voices of patients and citizens (15). These patients are participating from the payer’s side, or rather, from a position closer to the public. This differ from position of PPI patients in HTA in other countries, but the form of participation was maintained, without discussion, in the creation of the Japanese HTA system. However, as the PPIE value is diverse and meant to ensure the legitimacy of decisions, a discussion on the value and implementation of PPIE in future revisions of the Japanese HTA is inevitable. Previous revisions of the Japanese system have discussed the training of experts to perform technical assessments (60,62), but future expansions of the system should also discuss the training of experts to implement PPIE.

PPIE implementation.

HTA in the three countries other than Japan clearly describe PPIE in the guidelines and reports and include a variety of participation units and opportunities for participation. In contrast, the Japanese HTA lacks process guidelines, and the participation of patient and citizen representatives in the three meetings (General Assembly, Expert Committee of CEE, and Special Committee of CEE) must be confirmed in the respective meeting rosters and minutes (18;19;47;48). The participation in these meetings includes representatives of the public interest, labor organizations, and patient groups, or patient experts with disease-specific expertise as well as technical knowledge of pharmaceutical research and development and regulatory affairs through experience in drug litigation (1 8;1 9).

While only a few mentions of patient and citizen representatives were identified, the minutes recorded a request to establish a system that would enable them to listen to the opinions of a wide range of patient groups and to evaluate them comprehensively (60). This suggests that the representatives have the opportunity to participate in the meetings, but that participants lack sufficient resources to express their opinions. Opportunities are lacking for patients and citizens to express their views directly outside of meetings and no mechanism exists to encourage the expression of these opinions. Even if people struggle to express their opinions, no clear mechanism exists to link these opinions to measures or to provide feedback to patients and citizens. Thus, guidelines on processes, including PPIE, are needed, and patients and citizens should conduct process evaluations.

Resources required for PPIE.

In countries studied other than Japan, patients and public representatives are involved in the discussion at each stage of HTA evaluation, and environments have been developed to provide multiple methods for input to be provided. The resources required to develop these environments included process guidelines, education and training, information sharing and provision from institutions, and various opportunities and support systems for input and feedback.

As mentioned above, Japan lacks HTA process guidelines and information about education and training. PPIE information in clinical research is provided by patients and public groups (Japanese translation of EUPATI’s PPI-related educational program (63;64). Moreover, PPIE information in clinical practice guidelines (8) includes a section on HTA, which is useful for understanding the situations in other countries. However, the Japanese HTA differs significantly from other systems in terms of reimbursement and scope of covered technologies. Promoting an understanding of the Japanese HTA requires explanatory information from C2H (49) and lay (plain language) summaries of various reports on the discussions at Expert Committee of CEE (50,51).

Education and training are also necessary for those who accept to participate. Additionally, the application of information obtained through PPIE will enhance the value of PPIE.

Many HTA agencies in this review have centralized and provided information on evaluation findings (45). In Japan, relevant information is provided separately by the three meetings of the Chuikyo (46–48) and by the C2H (49–51), making it difficult for the public and patients to obtain comprehensive information and judge the content of such reports. As discussed above, a variety of input opportunities for participants is required, as well as a support system to provide feedback to participants and set up contact points for individual consultations.

Patient and citizen participation in the current Japanese HTA follows the structure of Chuikyo, which was reformed in 2005 and is very limited in terms of being considered PPIE in HTA. Although PPIE purpose, value, and implementation has not been discussed and the resources (education/training and information sharing) required to comment on the discussions are not sufficient, patient and citizen representatives have expressed their views in these discussions. However, whether their comments can lead to measures is unclear, and no mechanism is available for patients and citizens to receive feedback on subsequent responses or to evaluate the outcomes of their participation.

PPIE value and implementation must be specifically considered in discussions of future revisions. For this purpose, resources must be secured for patients and the public toward discussion. Patient and public access to HTA information, and discussion of PPIE in HTA outside the system should be increased.

Limitations

The present study has some limitations. The first is the choice of keywords in the literature search, particularly in the scoping review, which forced the use of “technology assessment” as a single phrase. However, a preliminary review confirmed that the review is broader than the single phrase “health technology assessment” and that the addition of the abbreviations of the HTA bodies (NICE, CADTH, and PBAC) prevents omissions. These measures were necessary to ensure more efficient and accurate reviews. Second, the selection of search targets excluded reports on PPIE with a focus on ethics. However, reviewing reports on ethics requires a different approach (that is on qualitative synthesis methods); thus, future reviews by experts in this area are needed. Third, the search for websites was limited to three HTA organizations. Future reviews with more collaborators will broaden the scope of this survey. Fourth, the granularity of information differed between reports obtained through the umbrella and scoping reviews. The present review complements these differences by citing systematic reviews for matters in which multiple-country cases were considered and citing individual articles for other matters. Fifth is that this review did not perform prioritization as the Japanese system automatically determines which drugs or medical devices are eligible for HTA, based on the price determined in the previous stage of the evaluation process (13). Future expansion of the Japanese system to include reimbursement eligibility will require additional review that includes prioritization. Finally, from spring 2024, Canada’s reorganization from the CADTH to Canada’s Drug Agency and Australia’s comprehensive consideration of HTA process evaluation is a common issue for non-HTA decision-making (65). As of July 2024, these changes have not been identified; however, the possibility of future changes should be noted. In Japan, “patient and public involvement” must be discussed as a common theme for future policymaking in the MHLW (66).

Conclusion

PPIE in HTA aims to provide patient and public insight into decision-making and has the value of relevance, equity, fairness, legitimacy, and capacity building. In other countries, patient and public representatives are involved in the discussion at each stage of HTA evaluation and mechanisms have been established to allow these representatives to provide input in a variety of ways.

Patient and citizen participation in the current Japanese HTA follows the previous structure, which is very limited when considered as PPIE. Considering the diverse values of PPI, when reforming the system (expanding the scope of coverage), PPIE value and implementation should be discussed and an environment should be created in which diverse patients and citizens can easily express their views.