A conceptual framework, called Innovation of Health Technology Assessment Methods (IHTAM), has been developed to facilitate the understanding of how to innovate methods of health technology assessment (HTA). However, the framework applicability has not been evaluated in practice. Hence, we aimed to explore framework applicability in three cases of method innovation that are part of the HTx project and to develop a roadmap to improve framework applicability.

The IHTAM framework was applied to three cases of innovating HTA methods. We collected feedback from case study leaders and consortium members after a training session, an approximately 1-year follow-up of periodic case study meetings, and a general assembly meeting where innovation progresses of the three cases were reported through surveys and interviews. Feedback was then summarized using an open-coding technique.

According to feedback, the framework provided a structured way of deliberation and helped to improve collaboration among HTA stakeholders. However, framework applicability could be improved if it was complemented by a roadmap with a loop structure to provide tailored guidance for different cases, and with items to elaborate actions to be taken by stakeholders. Accordingly, a 48-item roadmap was developed.

The IHTAM framework was generally applicable to the three case studies. A roadmap, with loop structure and actionable items, could complement the framework, and may provide HTA stakeholders with tailored guidance on developing new methods. To further examine the framework applicability, we recommend stakeholders to apply the IHTAM framework and its roadmap in future practice.

The sandbox approach, developed in the financial technologies sector, creates an environment to collaboratively develop and test innovative new products, methods and regulatory approaches, separated from business as usual. It has been used in health care to encourage innovation in response to emerging challenges, but, until recently, has not been used in health technology assessment (HTA). This article summarizes our learnings from using the sandbox approach to address three challenges facing HTA organizations and to identify implications for the use of this approach in HTA.

We identified three challenging contemporary HTA-related topics to explore in a sandbox environment, away from the pressures and interests of “live” assessments. We convened a pool of 120 stakeholders and experts to participate in various sandbox activities and ultimately co-develop solutions to help HTA organizations respond to the identified challenges.

Important general learnings about the potential benefits and implementation of a sandbox approach in HTA were identified. Consequently, we developed recommendations to guide its use, including how to implement an HTA sandbox in an effective way and the types of challenges for which it may be best suited.

For many HTA organizations, it is difficult to carefully consider emerging challenges and innovate their processes due to risks associated with decision errors and resource limitations. The sandbox approach could reduce these barriers. The potential benefits of addressing HTA challenges in a collaborative “safe space” are considerable.

To assess the health technology assessment (HTA) process in Greece from its establishment in 2018 until 2023 in terms of timeliness and productivity.

Data were collected from the HTA Committee’s database and other publicly available sources. The overall study timeframe was divided into three periods: (i) July 2018–January 2020, (ii) January 2020–July 2021, and (iii) July 2021–February 2023.

During the study period, a total of 1,157 applications for medicinal products (MPs) (including 219 new active substances (NAS) and orphans) were submitted to the HTA Committee. The number of HTA recommendations increased from 60 (first period) to 641 (third period), while the backlog of MPs pending HTA and price negotiations decreased from 89 and 106 (January 2020) to 8 and 44 (February 2023), respectively. The median time intervals for all application types decreased significantly over time. In February 2023, the median time for clinical data assessment of NAS (excluding orphans) almost halved from 207 days in the first period to 114 days; median times for NAS and orphans from regulatory approval to HTA application were 420 and 457 days, and from HTA application to reimbursement 228 and 417 days, respectively.

The performance of the HTA process in Greece improved significantly over time, with increased MP appraisals, backlog reduction, and decreased timelines. Delays in reimbursement of NAS were mainly caused by the long gap between regulatory approval and HTA application. Overall, HTA review times in Greece are now on par with that of well-established European HTA systems.

This study investigated how patient representatives have experienced their involvement in medicines appraisal and reimbursement processes with the Council for Choices in Health Care in Finland (COHERE) and the Pharmaceuticals Pricing Board (PPB) and how authorities perceive the role of patient organizations’ input.

Semi-structured thematic individual and pair interviews were conducted in 2021 with representatives (n = 14) of patient organizations and government officials (n = 7) of the Ministry of Social Affairs and Health. The interview data were analyzed using qualitative content analysis.

Patient representatives expressed their appreciation for the PPB and the COHERE in creating consultation processes and systematic models that support involvement. However, there were many challenges: patient representatives were uncertain about how their submissions were utilized in official processes and whether their opinions had any significance in decision-making. Patients or patient organizations lack representation in appraisal and decision-making bodies, and patient representatives felt that decision-making lacked transparency. The importance of patient involvement was highlighted by the authorities, but they also emphasized that the patient organizations’ contributions were complementary to the other materials. Submissions regarding the medications used to treat rare diseases and those with limited research evidence were considered particularly valuable. However, the submissions may not necessarily have a direct impact on decisions.

The interviews provided relevant input for the development of involvement processes at the PPB and COHERE. The interviews confirmed the need for increased transparency in the medicines assessment, appraisal, and decision-making procedures in Finland.

In France, decisions for pricing and reimbursement for medicinal products are based on appraisals performed by the National authority for health (Haute Autorité de Santé (HAS)). During the appraisal process, additional real-world evidence can be requested as “Post-Registration Studies” (PRS) when there are uncertainties in evidence that could be resolved by additional data collection. To facilitate PRS planning, a retrospective exploratory analysis was conducted to identify the characteristics of medicinal products associated with a PRS request.

This analysis encompassed all appraisals finalized between January 1, 2016 and December 31, 2021 and compared products for which the appraisal led to a PRS request with those that did not.

Six hundred positive opinions for reimbursement were identified, with a PRS request present in 17 percent (n = 103) of cases. The independent characteristics associated with a PRS request were a mild or moderate clinical benefit score, a major to moderate or minor clinical added value score, previous availability under an early access program, and certain therapeutic areas (neurology, pulmonology, and endocrinology). These findings suggest two different profiles of PRS requests: (i) products for which there is uncertainty in the size of the clinical benefit and (ii) innovative products for which a substantial benefit is expected but uncertainties persist.

These results will assist health technology developers to better anticipate data generation to promptly address uncertainties identified by HAS. It may also help HAS and other assessment agencies to work together to improve postlaunch evidence generation according to the characteristics of the medicinal products.

Systematic priority setting is necessary for achieving high-quality healthcare using limited resources in low- and middle-income countries. Health technology assessment (HTA) is a tool that can be used for systematic priority setting. The objective of this study was to conduct a stakeholder and situational analysis of HTA in Zimbabwe.

We identified and analyzed stakeholders using the International Decision Support Initiative checklist. The identified stakeholders were invited to an HTA workshop convened at the University of Zimbabwe. We used an existing HTA situational analysis questionnaire to ask for participants’ views on the need, demand, and supply of HTA. A follow-up survey was done among representatives of stakeholder organizations that failed to attend the workshop. We reviewed two health policy documents relevant to the HTA. Qualitative data from the survey and document review were analyzed using thematic analysis.

Forty-eight organizations were identified as stakeholders for HTA in Zimbabwe. A total of 41 respondents from these stakeholder organizations participated in the survey. Respondents highlighted that the HTA was needed for transparent decision making. The demand for HTA-related evidence was high except for the health economic and ethics dimensions, perhaps reflecting a lack of awareness. Ministry of Health was listed as a major supplier of HTA data.

There is no formal HTA agency in the Zimbabwe healthcare system. Various institutions make decisions on prioritization, procurement, and coverage of health services. The activities undertaken by these organizations provide context for the institutionalization of HTA in Zimbabwe.

Our objective was to explore procedures and methods used at health technology assessment (HTA) agencies for assessing medical devices and the underlying views of HTA practitioners about appropriate methodology to identify challenges in adopting new methodologies for assessing devices. We focused on the role of normative commitments of HTA practitioners in the adoption of new methods.

An online survey, including questions on procedures, scoping, and assessments of medical devices, was sent to members of the International Network of Agencies for Health Technology Assessment. Interviews were conducted with survey respondents and HTA practitioners involved in assessments of transcatheter aortic valve implantation to gain an in-depth understanding of choices made and views about assessing medical devices. Survey and interview questions were inspired by the “values in doing assessments of health technologies” approach towards HTA, which states that HTA addresses value-laden questions and information.

The current practice of assessing medical devices at HTA agencies is predominantly based on procedures, methods, and epistemological principles developed for assessments of drugs. Both practical factors (available time, demands of decision-makers, existing legal frameworks, and HTA guidelines), as well as commitments of HTA practitioners to principles of evidence-based medicine, make the adoption of a new methodology difficult.

There is a broad recognition that assessments of medical devices may need changes in HTA methodology. In order to realize this, the HTA community may require both a discussion on the role, responsibility, and goals of HTA, and resulting changes in institutional context to adopt new methodologies.

The rising costs of drugs have necessitated the exploration of innovative payment methods in healthcare systems. Risk-sharing agreements (RSAs) have been implemented in many countries as a value-based payment mechanism to manage the uncertainty associated with expensive technologies. This study aimed to investigate stakeholder perspectives on value-based payment in the Singaporean context, providing insights for future directions in health technology assessment and financing.

This descriptive qualitative inquiry involved participant interviews conducted between October 2021 and April 2022. Thematic analysis was conducted in two phases to analyze the interview transcripts.

Seventeen respondents participated in the study, and five key themes emerged from the analysis. Stakeholders viewed RSAs as moderately positive, despite limited experience with them. They emphasized the importance of clearly defining objectives and establishing transparent criteria for implementing these schemes. The current data infrastructure was identified as both a barrier and facilitator, as RSAs impose administrative burdens. To successfully implement these payment mechanisms, capacity building, and effective stakeholder engagement that fosters mutual trust and cocreation are crucial.

This study confirms previously identified barriers and facilitators to successful RSA implementation while contextualizing them within the Singaporean setting. The findings suggest that value-based payment has the potential to address uncertainty and improve access to healthcare technologies, but these barriers must be addressed for the schemes to be effective.

Health technology assessment (HTA) plays a vital role in healthcare decision-making globally, necessitating the identification of key factors impacting evaluation outcomes due to the significant workload faced by HTA agencies.

The aim of this study was to predict the approval status of evaluations conducted by the Brazilian Committee for Health Technology Incorporation (CONITEC) using natural language processing (NLP).

Data encompassing CONITEC’s official report summaries from 2012 to 2022. Textual data was tokenized for NLP analysis. Least Absolute Shrinkage and Selection Operator, logistic regression, support vector machine, random forest, neural network, and extreme gradient boosting (XGBoost), were evaluated for accuracy, area under the receiver operating characteristic curve (ROC AUC) score, precision, and recall. Cluster analysis using the k-modes algorithm categorized entries into two clusters (approved, rejected).

The neural network model exhibited the highest accuracy metrics (precision at 0.815, accuracy at 0.769, ROC AUC at 0.871, and recall at 0.746), followed by XGBoost model. The lexical analysis uncovered linguistic markers, like references to international HTA agencies’ experiences and government as demandant, potentially influencing CONITEC’s decisions. Cluster and XGBoost analyses emphasized that approved evaluations mainly concerned drug assessments, often government-initiated, while non-approved ones frequently evaluated drugs, with the industry as the requester.

NLP model can predict health technology incorporation outcomes, opening avenues for future research using HTA reports from other agencies. This model has the potential to enhance HTA system efficiency by offering initial insights and decision-making criteria, thereby benefiting healthcare experts.

Traditional therapies are crucial in maintaining and improving human well-being. China’s healthcare policymakers are attempting to use health technology assessment (HTA) as a decision-making supportive tool. The value assessment framework for Chinese patent medicine (CPM) has been developed and is being adopted and validated widely by research institutions. Subsequently, the healthcare decision-makers particularly hanker for the value framework of traditional non-pharmacological therapies.

To construct a practical value framework for traditional non-pharmacological therapies, a scoping review methodology was adopted to identify the evaluation domains and obstacles. A search, screening, and analysis process was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). Evidence was retrieved from scientific databases and HTA agencies’ websites.

The search strategy identified 5 guidelines records and 17 acupuncture HTA reports. By synthesizing the valuable reports of CPM and acupuncture evaluation in representative countries, this study found that Mainland China was promoting the comprehensive value assessment of CPM, whereas the United Kingdom, Singapore, Canada, the United States, and Malaysia had carried out the HTA evaluation of acupuncture for various conditions among which chronic pain was the most common. UK and Singapore applied the HTA results to support acupuncture reimbursement decisions. Three domains, including safety, effectiveness, and economy, were commonly adopted. The identified biggest challenge of evaluating traditional non-pharmacological therapies is the scarce high-quality clinical evidence.

This study identified value domains and issues of traditional therapies, and pointed out future research implications, to promote the development value framework of traditional therapies.

Poor nutrition links to chronic diseases, emphasizing the need for optimized diets. The EU-funded project PREVENTOMICS, introduced personalized nutrition to address this. This study aims to perform a health technology assessment (HTA) comparing personalized nutrition interventions developed through this project, with non-personalized nutrition interventions (control) for people with normal weight, overweight, or obesity. The goal is to support decisions about further development and implementation of personalized nutrition.

The PREVENTOMICS interventions were evaluated using the European Network for HTA Core Model, which includes a methodological framework that encompasses different domains for value assessment. Information was gathered via [1] different statistical analyses and modeling studies, [2] questions asked of project partners and, [3] other (un)published materials.

Clinical trials of PREVENTOMICS interventions demonstrated different body mass index changes compared to control; differences ranged from −0.80 to 0.20 kg/m2. Long-term outcome predictions showed generally improved health outcomes for the interventions; some appeared cost-effective (e.g., interventions in UK). Ethical concerns around health inequality and the lack of specific legal regulations for personalized nutrition interventions were identified. Choice modeling studies indicated openness to personalized nutrition interventions; decisions were primarily affected by intervention’s price.

PREVENTOMICS clinical trials have shown promising effectiveness with no major safety concerns, although uncertainties about effectiveness exist due to small samples (n=60–264) and short follow-ups (10–16 weeks). Larger, longer trials are needed for robust evidence before implementation could be considered. Among other considerations, developers should explore financing options and collaborate with policymakers to prevent exclusion of specific groups due to information shortages.

The development and strengthening of health technology assessment (HTA) capacity on the individual and organizational level and the wider environment is relevant for cooperation on HTAs. Based on the Maltese case, we provide a blueprint for building HTA capacity.

A set of activities were developed based on Pichler et al.’s framework and the starting HTA capacity in Malta. Individual level activities focused on strengthening epidemiological and health economic skills through online and in-person training. On the organizational level, a new HTA framework was developed which was subsequently utilized in a shadow assessment. Awareness campaign activities raised awareness and support in the wider environment where HTAs are conducted and utilized.

The time needed to build HTA capacity exceeded the planned two years accommodating the learning progress of the assessors. In addition to the planned trainings, webinars supplemented the online courses, allowing for more knowledge exchange. The advanced online course was extended over time to facilitate learning next to the assessors’ daily tasks. Training sessions were added to implement the new economic evaluation framework, which was utilized in a second shadow assessment. Awareness by decision-makers was achieved with reports, posters, and an article on the current and developing HTA capacity.

It takes time and much (hands-on) training to build skills for conducting complex assessment such as HTAs. Facilitating exchange with knowledgeable parties is crucial for succeeding as well as the buy-in of local managers motivating staff. Decision-makers need to be on-boarded for the continued success of HTA capacity building.

This study examined the application, feasibility, and validity of supervised learning models for text classification in appraisals for rare disease treatments (RDTs) in relation to uncertainty, and analyzed differences between appraisals based on the classification results.

We analyzed appraisals for RDTs (n = 94) published by the National Institute for Health and Care Excellence (NICE) between January 2011 and May 2023. We used Naïve Bayes, Lasso, and Support Vector Machine models in a binary text classification task (classifying paragraphs as either referencing uncertainty in the evidence base or not). To illustrate the results, we tested hypotheses in relation to the appraisal guidance, advanced therapy medicinal product (ATMP) status, disease area, and age group.

The best performing (Lasso) model achieved 83.6 percent classification accuracy (sensitivity = 74.4 percent, specificity = 92.6 percent). Paragraphs classified as referencing uncertainty were significantly more likely to arise in highly specialized technology (HST) appraisals compared to appraisals from the technology appraisal (TA) guidance (adjusted odds ratio = 1.44, 95 percent CI 1.09, 1.90, p = 0.004). There was no significant association between paragraphs classified as referencing uncertainty and appraisals for ATMPs, non-oncology RDTs, and RDTs indicated for children only or adults and children. These results were robust to the threshold value used for classifying paragraphs but were sensitive to the choice of classification model.

Using supervised learning models for text classification in NICE appraisals for RDTs is feasible, but the results of downstream analyses may be sensitive to the choice of classification model.

Qualitative methods allow in-depth exploration of patient experiences and can provide context for healthcare decision making. Frameworks for patient-based evidence in health technology assessment (HTA) are expanding; yet, how extensively qualitative methods are currently used is unclear. This review characterized the extent and quality of qualitative data submitted to National Institute for Health and Care Excellence (NICE) and Canadian Agency for Drugs and Technologies in Health (CADTH) for HTA.

NICE and CADTH submissions from September 2019 to August 2021 were reviewed. Submission characteristics and features of patient-based evidence included within submissions were extracted. The quality of qualitative reporting was assessed using the CASP checklist.

Patient-based evidence was included in 83/107 NICE and 119/124 CADTH submissions. A small proportion described qualitative data collection (NICE=14; CADTH=24) and analysis (NICE=6; CADTH=9) methods. One-to-one interviews were the most common data collection method, and thematic analysis was exclusively used. Thirty-three percent of NICE submissions scored >7 yes responses on CASP, versus 78 percent of CADTH submissions.

Although patient-based evidence was common in the submissions reviewed, only 14/107 NICE and 24/124 CADTH submissions involved formal qualitative data collection. Use of formal analysis methods was even rarer and reporting tended to be brief. At present, there is little guidance about qualitative evidence most likely to be informative and therefore to potentially impact decision making. Ensuring, however, that qualitative data are collected and analyzed in a systematic, rigorous way will maximize their usefulness and ensure that patient voices are clearly heard.

Health technology assessment (HTA) traditionally informs decision making for single health technologies, which could lead to ill-informed decisions, suboptimal care, and system inefficiencies. We explored opportunities for conceptualizing the decision space in HTA as a disease management question versus an intervention management question.

Semistructured interviews were conducted between April 2022 and October 2022 with purposefully selected individuals from national and provincial HTA agencies and related organizations in Canada. We conducted manual line by line coding of data informed by our interview guide and sensitizing concepts from the literature. One author coded the data, and findings were independently verified by a second author who coded a subset of transcripts

Twenty-four invitations were distributed, and eighteen individuals agreed to participate. A disease management approach to HTA was differentiated from traditional approaches as being disease-based, multi-interventional, and dynamic. There was general support for an explicit care pathway approach to HTA by informing discussions around patient choice and suboptimal care, creating a space where decision makers can collaborate on shared objectives, and in setting up a platform for open dialogue about managing high-cost and high-severity diseases. There are opportunities for a care pathway approach to be implemented that build on the strengths of the existing HTA system in Canada.

A disease management approach may enhance the impact of HTA by supporting dynamic decision making that could better inform a proactive, resilient, and sustainable healthcare system in Canada.