Health technologies are becoming increasingly complex and contemporary health technology assessment (HTA) is only partly equipped to address this complexity. The project “Integrated assessments of complex health technologies” (INTEGRATE-HTA), funded by the European Commission, was initiated with the overall objective to develop concepts and methods to enable patient-centered, integrated assessments of the effectiveness, and the economic, social, cultural, and ethical issues of complex technologies that take context and implementation issues into account. The project resulted in a series of guidances that should support the work of HTA scientists and decision makers alike.

Objectives: Some countries make considerable effort to involve patients and patient groups in their health technology assessment (HTA) processes; others are only just considering or are yet to consider patient involvement in HTA.

Methods: This commentary offers four arguments why patient involvement should be prioritized by those HTA agencies that do not yet involve patients: (1) from a patients’ rights perspective, (2) based on patient and community values, (3) centering on evidentiary contributions, and (4) from a methodological perspective.

Results: The first argument builds on the Alma-Ata Declaration, which holds that patients have a right and duty to have a say in the planning and delivery of their health care, individually and collectively. Where HTA is used to determine access to technologies and services, we argue that patients have a right to be heard. The second argues that decisions about treatments and services need to be aligned with the core values and morals of the patients whom the health system serves. The third argues that patients have unique knowledge and insights about living with a health condition and their needs for services and treatments regarding that condition, which can add to the knowledge base and value of the HTA process. The fourth argues that involvement of patients can facilitate methodological advancement of HTA, in areas such as early scientific advice and managed entry with evidence development.

Conclusions: An HTA process that includes patient perspectives can, therefore, provide added value to patients, policy makers and healthcare professionals alike.

This mini-theme contains six stories of health technology assessment (HTA) impact from member agencies of The International Network of Agencies for Health Technology Assessment (INAHTA), which were originally shared at the 2015 and 2016 INAHTA Congresses. The INAHTA impact story sharing is an innovative network activity where member agency representatives share experiences of HTA impact in a loosely structured story format. Through this process, members gain insights from other agencies on new ways of thinking about and approaching HTA impact assessment. A guide is provided to members to prepare their story, and the best story receives the David Hailey Award for Best Impact Story. This mini-theme contains stories of HTA impact from six member agencies in different parts of the world: the Health Assessment Division of the Ministry of Public Health (Uruguay), the Institute of Quality and Efficiency in Health Care (Germany), the Health Information and Quality Authority (Ireland), the Finnish Office for Health Technology Assessment (Finland), the Australian Safety and Efficacy Register of New Interventional Procedures-Surgical (Australia), and the Institut national d'excellence en santé et en services sociaux (Canada). Across the papers, common themes emerge about the importance of appropriate engagement of stakeholders and the broadening scope of HTA beyond reimbursement decision making.

The number of publications on health technology assessment (HTA) from Central, Eastern, and Southern Europe (CESE countries) is still low compared with the north and west of Europe. It is not surprising, as the idea of HTA originated from high-income Western economies and was afterward adopted by the south-eastern part of Europe, which mostly consists of middle-income countries. These CESE countries, with less capacity and experience with HTA processes, must deal with even tougher decisions on financing health technologies than north-western Europe. There may even be a lack of confidence to open discussions on their specific needs for HTA.

Objectives: The evaluation of public health interventions poses some challenges. As a consequence, health technology assessment (HTA) methods for public health interventions (PHI) have to be adapted. This study aimed to summarize the available guidance on methods for HTA of PHI.

Methods: We systematically searched for methodological guidance on HTA of PHIs. Our focus was on research synthesis methods to evaluate effectiveness. Relevant information was synthesized narratively in a standardized way.

Results: Only four guidance documents were identified specifically for HTAs of PHI. The approaches used for HTAs of PHIs are broader and more flexible than those for medical interventions. For this reason, there is a tendency to identify the intervention components and context factors that influence the effectiveness and transferability of an intervention rather than to assess its effectiveness in general. The details in the guidance vary without justification. Unjustified heterogeneity between the different guidance approaches is most pronounced for quality assessment, assessment of applicability, and methods to integrate qualitative and quantitative evidence. Descriptions for the assessment of integrity, heterogeneity, sustainability, context factors, and applicability are often vague.

Conclusions: The heterogeneity in approaches indicates that there is currently no consensus on methods to deal with the challenges of the PHI evaluations. A possible explanation for this may be that the methods are not sufficiently developed, and advantages and disadvantages of a certain method in relation to the research question (e.g., broad/focused) have not yet been sufficiently evaluated.

Background: The importance of economic evaluation in decision making is growing with increasing budgetary pressures on health systems. Diverse economic evidence is available for a range of interventions across national contexts within Europe, but little attention has been given to identifying evidence gaps that, if filled, could contribute to more efficient allocation of resources. One objective of the Research Agenda for Health Economic Evaluation project is to determine the most important methodological evidence gaps for the ten highest burden conditions in the European Union (EU), and to suggest ways of filling these gaps.

Methods: The highest burden conditions in the EU by Disability Adjusted Life Years were determined using the Global Burden of Disease study. Clinical interventions were identified for each condition based on published guidelines, and economic evaluations indexed in MEDLINE were mapped to each intervention. A panel of public health and health economics experts discussed the evidence during a workshop and identified evidence gaps.

Results: The literature analysis contributed to identifying cross-cutting methodological and technical issues, which were considered by the expert panel to derive methodological research priorities.

Conclusions: The panel suggests a research agenda for health economics which incorporates the use of real-world evidence in the assessment of new and existing interventions; increased understanding of cost-effectiveness according to patient characteristics beyond the “-omics” approach to inform both investment and disinvestment decisions; methods for assessment of complex interventions; improved cross-talk between economic evaluations from health and other sectors; early health technology assessment; and standardized, transferable approaches to economic modeling.

Objectives: The headroom method was introduced for the very early evaluation of the potential value of new technologies. It allows for establishing a ceiling price for technologies to still be cost-effective by combining the maximum effect a technology might yield, the maximum willingness-to-pay (WTP) for this effect, and potential downstream expenses and savings. Although the headroom method is QALY-based, not all innovations are expected to result in QALY gain.

Methods: This study explores the feasibility and usefulness of the headroom method in the evaluation of technologies that are unlikely to result in QALY gain. This will be illustrated with the diagnostic trajectory of complex pediatric neurology (CPN).

Results: Our headroom analysis showed a large room for improvement in the current diagnostic trajectory of CPN in terms of diagnostic yield. Combining this with a maximum WTP value for an additional diagnosis and the potential downstream expenses and savings, resulted in a total headroom of €15,028. This indicates that a new technology in this particular diagnostic trajectory, might be cost-effective as long as its costs do not exceed €15,028.

Conclusions: The headroom method seems a useful tool in the very early evaluation of medical technologies, also in cases when immediate QALY gain is unlikely. It allows for allocating healthcare resources to those technologies that are most promising. It should be kept in mind, however, that the headroom assumes an optimistic scenario, and for that reason cannot guarantee future cost-effectiveness. It might be most useful for ruling out those technologies that are unlikely to be cost-effective.

Objectives: The evaluation of telemedicine from the patient's point of view has focused on the patient pathway and patient satisfaction. However, research in this field does not provide us with the means to assess a patient's perception of the procedure if their reasoning ability is impaired. In this study, we use direct observation of a patient's behavior and mood to assess their perception of an oral teleconsultation procedure.

Methods: This study has been conducted in the context of a pilot project using an asynchronous teleconsultation to improve access to dental care for the dependent elderly, disabled people, and prisoners, some of whom may be cognitively impaired. We use a direct observation form consisting of five behavioral variables and eight affect variables to reflect the patient's experience of the oral teleconsultation procedure.

Results: A total of 135 patients were evaluated; 10 refused the procedure. Psychotic patients (n = 33) had a somewhat negative experience during the oral teleconsultation procedure. Patients who were not psychotic had a positive experience; this decreased as we moved from the autonomous to the semi-autonomous and then to the dependent sub-group. Some gender differences were also noted.

Conclusions: Improving evidence on evaluating the acceptance of the cognitively impaired is required to improve the technology development process so that it can be translated into an improved patient experience and adherence. Although the study was specifically focused on teledentistry, the approach described in this study could be adapted to other forms of teleconsultation.

Objectives: Our objectives were to assess the current implementation of health technology assessment (HTA) in Hungary and to identify country-specific patterns of challenges and potential improvements.

Methods: We applied a structure that can be used to create HTA implementation roadmaps to evaluate various issues regarding HTA implementation. A comprehensive description of the Hungarian HTA system is presented according to relevant literature and experiences of the authors.

Results: By investigating eight components of HTA implementation, we identified the most important strengths and weaknesses of the Hungarian system. More specifically, we were mainly focusing on the emergence of HTA capacity, the establishment and current role of Department of HTA, the complex process of decision making, the quality elements developed in the near past, and the activity of Hungarian experts at international collaborations.

Conclusions: We concluded that there is a sophisticated methodological and educational basis for HTA in Hungary. A permanent focus on capacity building and changes to the reimbursement procedure can further improve transparency and the scientific basis of decision making in the country.

Objectives: The impact of health technology assessment (HTA) in the judicialization of the right of health has not been deeply studied in Latin American countries. The purpose of this study is to review the process of judicialization of the access to high cost drugs in Uruguay and assess the impact HTAs have had on this process.

Methods: The methodology used for this study included a comprehensive literature search in electronic databases, local journals, internal documents developed in the Ministry of Health, as well as conducting interviews with key informants.

Results: Judicialization of the access of high cost drugs has been increasing since 2010. The strategy of the Ministry of Health of Uruguay to decrease this problem included the organization of roundtables with judges and other stakeholders on the basis of HTA, the training of defense lawyers in the use and interpretation of HTA, and the participation of a professional who develops HTA in the preparation of the defense arguments. A year after the implementation of this strategy, 25 percent of writs of protection were won by the Ministry of Health.

Conclusions: Even though the strategy implemented was effective in reducing the loss of litigations, it was not effective in reducing the growing number of writs of protection. It is essential to address this problem in a broad debate and to promote understanding between the parties.

Objectives: Current health technology assessment (HTA) is not well equipped to assess complex technologies as insufficient attention is being paid to the diversity in patient characteristics and preferences, context, and implementation. Strategies to integrate these and several other aspects, such as ethical considerations, in a comprehensive assessment are missing. The aim of the European research project INTEGRATE-HTA was to develop a model for an integrated HTA of complex technologies.

Methods: A multi-method, four-stage approach guided the development of the INTEGRATE-HTA Model: (i) definition of the different dimensions of information to be integrated, (ii) literature review of existing methods for integration, (iii) adjustment of concepts and methods for assessing distinct aspects of complex technologies in the frame of an integrated process, and (iv) application of the model in a case study and subsequent revisions.

Results: The INTEGRATE-HTA Model consists of five steps, each involving stakeholders: (i) definition of the technology and the objective of the HTA; (ii) development of a logic model to provide a structured overview of the technology and the system in which it is embedded; (iii) evidence assessment on effectiveness, economic, ethical, legal, and socio-cultural aspects, taking variability of participants, context, implementation issues, and their interactions into account; (iv) populating the logic model with the data generated in step 3; (v) structured process of decision-making.

Conclusions: The INTEGRATE-HTA Model provides a structured process for integrated HTAs of complex technologies. Stakeholder involvement in all steps is essential as a means of ensuring relevance and meaningful interpretation of the evidence.

Objectives: Our study addresses the important issue of estimating treatment costs from historical data. It is a problem frequently faced by health technology assessment analysts. We compared four approaches used to estimate current costs when good quality contemporary data are not available using liver transplantation as an example.

Methods: First, the total cost estimates extracted for patients from a cohort study, conducted in the 1990s, were inflated using a published inflation multiplier. Second, resource use estimates from the cohort study were extracted for hepatitis C patients and updated using current unit costs. Third, expert elicitation was carried out to identify changes in clinical practice over time and quantify current resource use. Fourth, routine data on resource use were obtained from National Health Service Blood and Transplant (NHSBT).

Results: The first two methods did not account for changes in clinical practice. Also the first was not specific to hepatitis patients. The use of experts confirmed significant changes in clinical practice. However, the quantification of resource use using experts is challenging as clinical specialists may not have a complete overview of clinical pathway. The NHSBT data are the most accurate reflection of transplantation and posttransplantation phase; however, data were not available for the whole pathway of care. The best estimate of total cost, combining NHSBT data and expert elicitation, is £121,211.

Discussion: Observational data from routine care are potentially the most reliable reflection of current resource use. Efforts should be made to make such data readily available and accessible to researchers. Expert elicitation provided reasonable estimates.

Objectives: Zika virus (ZikaV) is currently one of the most important emerging viruses in the world which has caused outbreaks and epidemics and has also been associated with severe clinical manifestations and congenital malformations. Traditional approaches to combat the ZikaV outbreak are not effective for detection and control. The aim of this study is to propose a cloud-based system to prevent and control the spread of Zika virus disease using integration of mobile phones and Internet of Things (IoT).

Methods: A Naive Bayesian Network (NBN) is used to diagnose the possibly infected users, and Google Maps Web service is used to provide the geographic positioning system (GPS)-based risk assessment to prevent the outbreak. It is used to represent each ZikaV infected user, mosquito-dense sites, and breeding sites on the Google map that helps the government healthcare authorities to control such risk-prone areas effectively and efficiently.

Results: The performance and accuracy of the proposed system are evaluated using dataset for 2 million users. Our system provides high accuracy for initial diagnosis of different users according to their symptoms and appropriate GPS-based risk assessment.

Conclusions: The cloud-based proposed system contributed to the accurate NBN-based classification of infected users and accurate identification of risk-prone areas using Google Maps.

Objectives: The Scottish Health Technologies Group (SHTG) produces rapid reviews on nonmedicine technologies to inform advice for decision making in the National Health Service in Scotland (NHSScotland). This study describes our experiences and lessons learned in adapting for NHSScotland two rapid Relative Effectiveness Assessments (REAs) produced as part of the European Network for Health Technology Assessment (EUnetHTA) project to test collaborative preparation and then local adaptation.

Methods: The REAs were examined for their relevance to NHSScotland and the feasibility of their adaptation evaluated using a screening toolkit. Some supplementary literature searches were conducted and additional background information on epidemiology and the specific technologies was sought. To inform decision making within Scotland, it was also necessary to identify and review cost-effective analyses.

Results: Robust evidence reviews were delivered for NHSScotland. Time saved was less than anticipated, partly due to the need to add in health economic information, and partly because of attempts to supplement and update the EUnetHTA material. The preparation of an accompanying advice statement for NHSScotland enabled the inclusion of local contextual information.

Conclusions: Collaborative production and adaptation of HTAs is feasible and would be aided by the inclusion of cost-effectiveness analyses in the original work. Agencies should develop clear processes for adapting such reviews for their context with the focus on selecting appropriate topics for adaptation, avoiding unnecessary supplementation of the original work, and ensuring local ownership of final advice.

Objectives: Internationally, funders require stakeholder involvement throughout health technology assessment (HTA). We report successes, challenges, and lessons learned from extensive stakeholder involvement throughout a palliative care case study that demonstrates new concepts and methods for HTA.

Methods: A 5-step “INTEGRATE-HTA Model” developed within the INTEGRATE-HTA project guided the case study. Using convenience or purposive sampling or directly / indirectly identifying and approaching individuals / groups, stakeholders participated in qualitative research or consultation meetings. During scoping, 132 stakeholders, aged ≥ 18 years in seven countries (England, Italy, Germany, The Netherlands, Norway, Lithuania, and Poland), highlighted key issues in palliative care that assisted identification of the intervention and comparator. Subsequently stakeholders in four countries participated in face–face, telephone and / or video Skype meetings to inform evidence collection and / or review assessment results. An applicability assessment to identify contextual and implementation barriers and enablers for the case study findings involved twelve professionals in the three countries. Finally, thirteen stakeholders participated in a mock decision-making meeting in England.

Results: Views about the best methods of stakeholder involvement vary internationally. Stakeholders make valuable contributions in all stages of HTA; assisting decision making about interventions, comparators, research questions; providing evidence and insights into findings, gap analyses and applicability assessments. Key challenges exist regarding inclusivity, time, and resource use.

Conclusion: Stakeholder involvement is feasible and worthwhile throughout HTA, sometimes providing unique insights. Various methods can be used to include stakeholders, although challenges exist. Recognition of stakeholder expertise and further guidance about stakeholder consultation methods is needed.

Objectives: The Quebec Trauma Care Continuum (TCC) was initiated in 1991 with the objective of providing accessible, continuous, efficient, and high quality services for all injury cases in the province.

Methods: The TCC design relied on three key components: (i) the designation of a network of acute care and rehabilitation facilities with specific mandates and responsibilities; (ii) the elaboration of transfer protocols, standing agreements, and governing structures to ensure fluid and optimal patient flow; and (iii) the close monitoring of several indicators to facilitate the continuous evaluation and improvement of the network.

Results: Between 1992 and 2002, in-hospital mortality following major trauma decreased from 51.8 percent to 8.6 percent, followed by an additional 24 percent drop between 1999 and 2012. We also observed a 16 percent decrease in average LOS but no change in the incidence of complications or unplanned readmissions. These changes translate into 186 lives saved per year and cost savings, due to shorter LOS, of 6.3 million CD$ per year. The risk-adjusted incidence of in-hospital mortality following major injury between 2006 and 2012 (7 percent) was the lowest of all Canadian provinces.

Conclusions: Strategic transformation of a network's structure and processes, supported by continuous monitoring of validated quality indicators, can lead to significant and sustainable improvements in clinical outcomes. It is hoped that the Quebec trauma story will inspire other jurisdictions and other healthcare sectors.

Objectives: In the Czech Republic, the health technology assessment (HTA) approaches have been implemented in evaluation of medicinal products since 2008.

The aim of this study was to provide an overview of the implementation of HTA and different levels thereof in the evaluation process conducted by the State Institute for Drug Control (SUKL) and to describe the impact of HTA on the entrance of new medicinal entities into out-patient healthcare system including highly innovative and orphan drugs.

Methods: Materials supporting this overview were collected using the records in the database of administrative proceedings of SUKL, in-house standard operating procedures, and the legislation in force. Based on these sources as well as the hands-on knowledge of the current practice, a brief description of the general rules of administrative proceedings involving HTA of varying complexity was elaborated. Characteristic features of the individual types of proceedings, basic differences in the complexity of HTA employed, and its most important challenges were summarized.

Results: In Czech Republic, HTA in the formal administrative proceedings ensures a transparent process of introduction of new medicinal products into clinical practice and leaves space for restriction of reimbursement conditions to minimize budget impact.

Conclusions: As a robust as well as pragmatic HTA methodology has been implemented by SUKL, relevant stakeholders (marketing authorization holders, Health Care Funds, clinical expert groups) are now able to influence reimbursement of new technologies.