Health technology assessment (HTA) plays a vital role in healthcare decision-making globally, necessitating the identification of key factors impacting evaluation outcomes due to the significant workload faced by HTA agencies.

The aim of this study was to predict the approval status of evaluations conducted by the Brazilian Committee for Health Technology Incorporation (CONITEC) using natural language processing (NLP).

Data encompassing CONITEC’s official report summaries from 2012 to 2022. Textual data was tokenized for NLP analysis. Least Absolute Shrinkage and Selection Operator, logistic regression, support vector machine, random forest, neural network, and extreme gradient boosting (XGBoost), were evaluated for accuracy, area under the receiver operating characteristic curve (ROC AUC) score, precision, and recall. Cluster analysis using the k-modes algorithm categorized entries into two clusters (approved, rejected).

The neural network model exhibited the highest accuracy metrics (precision at 0.815, accuracy at 0.769, ROC AUC at 0.871, and recall at 0.746), followed by XGBoost model. The lexical analysis uncovered linguistic markers, like references to international HTA agencies’ experiences and government as demandant, potentially influencing CONITEC’s decisions. Cluster and XGBoost analyses emphasized that approved evaluations mainly concerned drug assessments, often government-initiated, while non-approved ones frequently evaluated drugs, with the industry as the requester.

NLP model can predict health technology incorporation outcomes, opening avenues for future research using HTA reports from other agencies. This model has the potential to enhance HTA system efficiency by offering initial insights and decision-making criteria, thereby benefiting healthcare experts.

To determine the level of awareness of health technology assessment (HTA) and its predictors among clinical year medical students in public universities in Klang Valley, Malaysia.

A cross-sectional study using the stratified random sampling method was conducted among clinical year medical students in four public universities in Klang Valley, Malaysia. Data on the level of awareness of HTA and its associated factors were collected using a self-administered online questionnaire. Descriptive, bivariate, and multivariate analyses were performed using IBM SPSS version 27 to determine the level of awareness of HTA and its predictors.

Majority (69 percent) of participants had a low level of awareness of HTA. The predictors of high-level awareness of HTA were attitude toward HTA (adjusted odds ratio (AOR) = 7.417, 95 percent confidence interval (CI): 3.491, 15.758), peer interaction on HTA (AOR = 0.320, 95 percent CI: 0.115, 0.888), and previous training on HTA (AOR = 4.849, 95 percent CI: 1.096, 21.444).

Most future doctors in public universities exhibit a low awareness of HTA. This study highlights the interplay between attitudes toward HTA, peer interaction, and previous training as influential predictors of HTA awareness. An integrated and comprehensive educational approach is recommended to cultivate a positive attitude and harness the positive aspects of peer interaction while mitigating the potential negative impact of misconceptions. Emphasizing early exposure to HTA concepts through structured programs is crucial for empowering the upcoming generation of healthcare professionals, enabling them to navigate HTA complexities and contribute to evidence-based healthcare practices in Malaysia and beyond.

Computerized clinical decision support software (CDSS) are digital health technologies that have been traditionally categorized as medical devices. However, the evaluation frameworks for traditional medical devices are not well adapted to assess the value and safety of CDSS. In this study, we identified a range of challenges associated with CDSS evaluation as a medical device and investigated whether and how CDSS are evaluated in Australia.

Using a qualitative approach, we interviewed 11 professionals involved in the implementation and evaluation of digital health technologies at national and regional levels. Data were thematically analyzed using both data-driven (inductive) and theory-based (deductive) approaches.

Our results suggest that current CDSS evaluations have an overly narrow perspective on the risks and benefits of CDSS due to an inability to capture the impact of the technology on the sociotechnical environment. By adopting a static view of the CDSS, these evaluation frameworks are unable to discern how rapidly evolving technologies and a dynamic clinical environment can impact CDSS performance. After software upgrades, CDSS can transition from providing information to specifying diagnoses and treatments. Therefore, it is not clear how CDSS can be monitored continuously when changes in the software can directly affect patient safety.

Our findings emphasize the importance of taking a living health technology assessment approach to the evaluation of digital health technologies that evolve rapidly. There is a role for observational (real-world) evidence to understand the impact of changes to the technology and the sociotechnical environment on CDSS performance.

Poor nutrition links to chronic diseases, emphasizing the need for optimized diets. The EU-funded project PREVENTOMICS, introduced personalized nutrition to address this. This study aims to perform a health technology assessment (HTA) comparing personalized nutrition interventions developed through this project, with non-personalized nutrition interventions (control) for people with normal weight, overweight, or obesity. The goal is to support decisions about further development and implementation of personalized nutrition.

The PREVENTOMICS interventions were evaluated using the European Network for HTA Core Model, which includes a methodological framework that encompasses different domains for value assessment. Information was gathered via [1] different statistical analyses and modeling studies, [2] questions asked of project partners and, [3] other (un)published materials.

Clinical trials of PREVENTOMICS interventions demonstrated different body mass index changes compared to control; differences ranged from −0.80 to 0.20 kg/m2. Long-term outcome predictions showed generally improved health outcomes for the interventions; some appeared cost-effective (e.g., interventions in UK). Ethical concerns around health inequality and the lack of specific legal regulations for personalized nutrition interventions were identified. Choice modeling studies indicated openness to personalized nutrition interventions; decisions were primarily affected by intervention’s price.

PREVENTOMICS clinical trials have shown promising effectiveness with no major safety concerns, although uncertainties about effectiveness exist due to small samples (n=60–264) and short follow-ups (10–16 weeks). Larger, longer trials are needed for robust evidence before implementation could be considered. Among other considerations, developers should explore financing options and collaborate with policymakers to prevent exclusion of specific groups due to information shortages.

Olaparib targets the DNA repair pathways and has revolutionized the management of metastatic castration resistant prostate cancer (mCRPC). Treatment with the drug should be guided by genetic testing; however, published economic evaluations did not consider olaparib and genetic testing as codependent technologies. This study aims to assess the cost-effectiveness of BRCA germline testing to inform olaparib treatment in mCRPC.

We conducted a cost-utility analysis of germline BRCA testing-guided olaparib treatment compared to standard care without testing from an Australian health payer perspective. The analysis applied a decision tree to indicate the germline testing or no testing strategy. A Markov multi-state transition approach was used for patients within each strategy. The model had a time horizon of 5 years. Costs and outcomes were discounted at an annual rate of 5 percent. Decision uncertainty was characterized using probabilistic and scenario analyses.

Compared to standard care, BRCA testing-guided olaparib treatment was associated with an incremental cost of AU$7,841 and a gain of 0.06 quality-adjusted life-years (QALYs). The incremental cost-effectiveness ratio (ICER) was AU$143,613 per QALY. The probability of BRCA testing-guided treatment being cost effective at a willingness-to-pay threshold of AU$100,000 per QALY was around 2 percent; however, the likelihood for cost-effectiveness increased to 66 percent if the price of olaparib was reduced by 30 percent.

This is the first study to evaluate germline genetic testing and olaparib treatment as codependent technologies in mCRPC. Genetic testing-guided olaparib treatment may be cost-effective with significant discounts on olaparib pricing.

Evidence development for medical devices is often focused on satisfying regulatory requirements with the result that health professional and payer expectations may not be met, despite considerable investment in clinical trials. Early engagement with payers and health professionals could allow companies to understand these expectations and reflect them in clinical study design, increasing chances of positive coverage determination and adoption into clinical practice.

An example of early engagement through the EXCITE International model using an early technology review (ETR) is described which includes engagement with payers and health professionals to better inform companies to develop data that meet their expectations. ETR is based on an early evidence review, a framework of expectations that guides the process and identified gaps in evidence. The first fourteen ETRs were reviewed for examples of advice to companies that provided additional information from payers and health professionals that was thought likely to impact on downstream outcomes or strategic direction. Given that limitations were imposed by confidentiality, examples were genericized.

Advice through early engagement can inform evidence development that coincides with expectations of payers and health professionals through a structured, objective, evidence-based approach. This could reduce the risk of business-related adverse outcomes such as failure to secure a positive coverage determination and/or acceptance by expert health professionals.

Early engagement with key stakeholders exemplified by the ETR approach offers an alternative to the current approach of focusing on regulatory expectations. This could reduce the time to reimbursement and clinical adoption and benefit patient outcomes and/or health system efficiencies.

To achieve universal health coverage (UHC), countries must make difficult choices to optimize the use of scarce resources. There is a growing interest in using evidence-based priority setting processes, such as Health Technology Assessment (HTA), to inform these decisions. In 2020, the Palestinian Institute of Public Health (PNIPH) and the Norwegian Institute of Public Health (NIPH) initiated a pilot to test the feasibility of coproducing an HTA on breast cancer screening in the West Bank, occupied Palestinian Territory. Additionally, a secondary aim was to test whether using an adaptive HTA (aHTA) approach that searched and transferred published evidence syntheses could increase the speed of HTA production.

The applied stepwise approach to the HTA is described in detail and can be summarized as defining a core team, topic selection, and prioritization; undertaking the HTA including adaptation using tools from the European Network for HTA (EUnetHTA) and stakeholder engagement; and concluding with dissemination.

The aHTA approach was faster but not as quick as anticipated, which is attributed to (i) the lack of availability of local evidence for contextualizing findings and (ii) the necessity to build trust between the team and stakeholders. Some delays followed from the COVID-19 pandemic, which showed the importance of good risk anticipation and mitigation. Lastly, other important lessons included the ability of virtual collaborations, the value of capacity strengthening initiatives within low- and middle-income countries (LMICs), and the need for early stakeholder engagement. Overall, the pilot was successfully completed.

This was the first HTA of its kind produced in Palestine, and despite the challenges, it shows that HTA analysis is feasible in this setting.

The development and strengthening of health technology assessment (HTA) capacity on the individual and organizational level and the wider environment is relevant for cooperation on HTAs. Based on the Maltese case, we provide a blueprint for building HTA capacity.

A set of activities were developed based on Pichler et al.’s framework and the starting HTA capacity in Malta. Individual level activities focused on strengthening epidemiological and health economic skills through online and in-person training. On the organizational level, a new HTA framework was developed which was subsequently utilized in a shadow assessment. Awareness campaign activities raised awareness and support in the wider environment where HTAs are conducted and utilized.

The time needed to build HTA capacity exceeded the planned two years accommodating the learning progress of the assessors. In addition to the planned trainings, webinars supplemented the online courses, allowing for more knowledge exchange. The advanced online course was extended over time to facilitate learning next to the assessors’ daily tasks. Training sessions were added to implement the new economic evaluation framework, which was utilized in a second shadow assessment. Awareness by decision-makers was achieved with reports, posters, and an article on the current and developing HTA capacity.

It takes time and much (hands-on) training to build skills for conducting complex assessment such as HTAs. Facilitating exchange with knowledgeable parties is crucial for succeeding as well as the buy-in of local managers motivating staff. Decision-makers need to be on-boarded for the continued success of HTA capacity building.

In response to the Omicron surge in early 2022, the HTA Philippines evaluated the acceptability of Filipinos in using self-administered antigen tests (SAAgTs) as part of COVID-19 HTAs in the Philippines.

Scoping review from literature databases was initially conducted to identify preset codes in the use of SAAgT. Preset codes were used to establish the questions for focus group discussions (FGDs). Semi-structured questionnaires were created through Delphi technique. FGDs with four stakeholder groups (i.e., nine healthcare workers [HCWs], seven representatives of at-risk groups, six economic frontliners, and seven representatives of micro–small–medium-sized enterprises) were conducted.

Discomfort in being a target of stigma and being prescribed an “illness identity” when suspected or confirmed COVID-19-positive, along with lack of confidence to perform self-test, caused hesitancy in self-testing among participants. The need for subsidies for test kits from the government or employers was emphasized to increase its accessibility. Having a designated access point and reporting system for SAAgT was highlighted to avoid nepotism (padrino system attributed to debt of gratitude), inequitable distribution, and lapses in reporting. A participatory approach to education was perceived as crucial to reduce any misconceptions associated with the use of SAAgT.

All FGD groups expressed favorable reviews on the implementation of SAAgT because it can potentially reduce the burden of health facility-administered tests. These findings were considered by the HTA Council in the recommendation of SAAgT as part of the overarching national strategies for the diagnosis and screening of COVID-19.

In the UK, the number of patients urgently referred for suspected cancer is increasing, and providers are struggling to cope with demand. We explore the potential cost-effectiveness of a new risk prediction test – the PinPoint test – to triage and prioritize patients urgently referred with suspected urological cancers.

Two simulation models were developed to reflect the diagnostic pathways for patients with (i) suspected prostate cancer, and (ii) bladder or kidney cancer, comparing the PinPoint test to current practice. An early economic analysis was conducted from a UK National Health Service (NHS) perspective. The primary outcomes were the percentage of individuals seen within 2 weeks and health care costs. An exploratory analysis was conducted to understand the potential impact of the Pinpoint test on quality-adjusted life years gained.

Across both models and applications, the PinPoint test led to more individuals with urological cancer being seen within 2 weeks. Using PinPoint only to prioritize patients led to increased costs overall, whereas using PinPoint to both triage and prioritize patients led to cost savings. The estimated impact on life years gained/lost was very small and highly uncertain.

Using the PinPoint test to prioritize urgent referrals meant that more individuals with urological cancer were seen within 2 weeks, but at additional cost to the NHS. If used as a triage and prioritization tool, the PinPoint test shortens wait times for referred individuals and is cost saving. More data on the impact of short-term delays to diagnosis on health-related quality of life is needed.

This study aims to gain insight into each attribute as presented in the value of implantable medical devices, quantify attributes’ strength and their relative importance, and identify the determinants of stakeholders’ preferences.

A mixed-methods design was used to identify attributes and levels reflecting stakeholders’ preference toward the value of implantable medical devices. This design combined literature reviewing, expert’s consultation, one-on-one interactions with stakeholders, and a pilot testing. Based on the design, six attributes and their levels were settled. Among 144 hypothetical profiles, 30 optimal choice sets were developed, and healthcare professionals (decision-makers, health technology assessment experts, hospital administrators, medical doctors) and patients as stakeholders in China were surveyed. A total of 134 respondents participated in the survey. Results were analyzed by mixed logit model and conditional logit model.

The results of the mixed logit model showed that all the six attributes had a significant impact on respondents’ choices on implantable medical devices. Respondents were willing to pay the highest for medical devices that provided improvements in clinical safety, followed by increased clinical effectiveness, technology for treating severe diseases, improved implement capacity, and innovative technology (without substitutes).

The findings of DCE will improve the current evaluation on the value of implantable medical devices in China and provide decision-makers with the relative importance of the criteria in pricing and reimbursement decision-making of implantable medical devices.

Qualitative methods allow in-depth exploration of patient experiences and can provide context for healthcare decision making. Frameworks for patient-based evidence in health technology assessment (HTA) are expanding; yet, how extensively qualitative methods are currently used is unclear. This review characterized the extent and quality of qualitative data submitted to National Institute for Health and Care Excellence (NICE) and Canadian Agency for Drugs and Technologies in Health (CADTH) for HTA.

NICE and CADTH submissions from September 2019 to August 2021 were reviewed. Submission characteristics and features of patient-based evidence included within submissions were extracted. The quality of qualitative reporting was assessed using the CASP checklist.

Patient-based evidence was included in 83/107 NICE and 119/124 CADTH submissions. A small proportion described qualitative data collection (NICE=14; CADTH=24) and analysis (NICE=6; CADTH=9) methods. One-to-one interviews were the most common data collection method, and thematic analysis was exclusively used. Thirty-three percent of NICE submissions scored >7 yes responses on CASP, versus 78 percent of CADTH submissions.

Although patient-based evidence was common in the submissions reviewed, only 14/107 NICE and 24/124 CADTH submissions involved formal qualitative data collection. Use of formal analysis methods was even rarer and reporting tended to be brief. At present, there is little guidance about qualitative evidence most likely to be informative and therefore to potentially impact decision making. Ensuring, however, that qualitative data are collected and analyzed in a systematic, rigorous way will maximize their usefulness and ensure that patient voices are clearly heard.

This study examined the application, feasibility, and validity of supervised learning models for text classification in appraisals for rare disease treatments (RDTs) in relation to uncertainty, and analyzed differences between appraisals based on the classification results.

We analyzed appraisals for RDTs (n = 94) published by the National Institute for Health and Care Excellence (NICE) between January 2011 and May 2023. We used Naïve Bayes, Lasso, and Support Vector Machine models in a binary text classification task (classifying paragraphs as either referencing uncertainty in the evidence base or not). To illustrate the results, we tested hypotheses in relation to the appraisal guidance, advanced therapy medicinal product (ATMP) status, disease area, and age group.

The best performing (Lasso) model achieved 83.6 percent classification accuracy (sensitivity = 74.4 percent, specificity = 92.6 percent). Paragraphs classified as referencing uncertainty were significantly more likely to arise in highly specialized technology (HST) appraisals compared to appraisals from the technology appraisal (TA) guidance (adjusted odds ratio = 1.44, 95 percent CI 1.09, 1.90, p = 0.004). There was no significant association between paragraphs classified as referencing uncertainty and appraisals for ATMPs, non-oncology RDTs, and RDTs indicated for children only or adults and children. These results were robust to the threshold value used for classifying paragraphs but were sensitive to the choice of classification model.

Using supervised learning models for text classification in NICE appraisals for RDTs is feasible, but the results of downstream analyses may be sensitive to the choice of classification model.

The purpose of this study is to evaluate the validity of the standard approach in expert judgment for evaluating precision medicines, in which experts are required to estimate outcomes as if they did not have access to diagnostic information, whereas in fact, they do.

Fourteen clinicians participated in an expert judgment task to estimate the cost and medical outcomes of the use of exome sequencing in pediatric patients with intractable epilepsy in Thailand. Experts were randomly assigned to either an “unblind” or “blind” group; the former was provided with the exome sequencing results for each patient case prior to the judgment task, whereas the latter was not provided with the exome sequencing results. Both groups were asked to estimate the outcomes for the counterfactual scenario, in which patients had not been tested by exome sequencing.

Our study did not show significant results, possibly due to the small sample size of both participants and case studies.

A comparison of the unblind and blind approach did not show conclusive evidence that there is a difference in outcomes. However, until further evidence suggests otherwise, we recommend the blind approach as preferable when using expert judgment to evaluate precision medicines because this approach is more representative of the counterfactual scenario than the unblind approach.

Telemedicine may improve healthcare access and efficiency if it demands less clinician time than usual care. We sought to describe the degree to which telemedicine trials assess the effect of telemedicine on clinicians’ time and to discuss how including the time needed to treat (TNT) in health technology assessment (HTA) could affect the design of telemedicine services and studies.

We conducted a scoping review by searching clinicaltrials.gov using the search term “telemedicine” and limiting results to randomized trials or observational studies registered between January 2012 and October 2023. We then reviewed trial registration data to determine if any of the outcomes assessed in the trials measured effect on clinicians’ time.

We found 113 studies and of these 78 studies of telemedicine met the inclusion criteria and were included. Nine (12 percent) of the 78 studies had some measure of clinician time as a primary outcome, and 11 (14 percent) as a secondary outcome. Four studies compared direct measures of TNT with telemedicine versus usual care, but no statistically significant difference was found. Of the sixteen studies including indirect measures of clinician time, thirteen found no significant effects, two found a statistically significant reduction, and one found a statistically significant increase.

This scoping review found that clinician time is not commonly measured in studies of telemedicine interventions. Attention to telemedicine’s TNT in clinical studies and HTAs of telemedicine in practice may bring attention to the organization of clinical workflows and increase the value of telemedicine.

Lack of evidence regarding safety and effectiveness at market entry is driving the need to consider adopting a lifecycle approach to evaluating medical devices, but it is unclear what lifecycle evaluation means. This research sought to explore the tacit meanings of “lifecycle” and “lifecycle evaluation” as embodied within evaluation models/frameworks used for medical devices.

Drawing on qualitative evidence synthesis methods and using an inductive approach, novel methods were developed to identify, appraise, analyze, and synthesize lifecycle evaluation models used for medical devices. Data was extracted (including purpose; audience; characterization; outputs; timing; and type of model) from key texts for coding, categorization, and comparison, exploring embodied meaning across four broad perspectives.

Fifty-two models were included in the synthesis. They demonstrated significant heterogeneity of meaning, form, scope, timing, and purpose. The “lifecycle” may represent a single stage, a series of stages, a cycle of innovation, or a system. “Lifecycle evaluation” focuses on the overarching goal of the stakeholder group, and may use a single or repeated evaluation to inform decision-making regarding the adoption of health technologies (Healthcare), resource allocation (Policymaking), investment in new product development or marketing (Trade and Industry), or market regulation (Regulation). The adoption of a lifecycle approach by regulators has resulted in the deferral of evidence generation to the post-market phase.

Using a “lifecycle evaluation” approach to inform reimbursement decision-making must not be allowed to further jeopardize evidence generation and patient safety by accepting inadequate evidence of safety and effectiveness for reimbursement decisions.