Objectives: Model for ASsessment of Telemedicine Applications (MAST) is a health technology assessment (HTA) inspired framework for assessing the effectiveness and contribution to quality of telemedicine applications based on rigorous, scientific data. This study reports from a study of how it was used and perceived in twenty-one pilots of the European project RENEWING HEALTH (RH). The objectives of RH were to implement large-scale, real-life test beds for the validation and subsequent evaluation of innovative patient-centered telemedicine services. The study is a contribution to the appraisal of HTA methods.

Methods: A questionnaire was administered for project leaders of the pilots. It included questions about use and usefulness of MAST for (i) preceding considerations, (ii) evaluation of outcomes within seven domains, and (iii) considerations of transferability. Free text spaces allowed for proposals of improvement. The responses covered all pilots. A quantitative summary of use and a qualitative analysis of usefulness were performed.

Results: MAST was used and considered useful for pilot evaluations. Challenges included problems to scientifically determine alternative service options and outcome within the seven domains. Proposals for improvement included process studies and adding domains of technological usability, responsible innovation, health literacy, behavior change, caregiver perspectives and motivational issues of professionals.

Conclusions: MAST was used according to its structure. Its usefulness in patient centered pilots can be improved by adding new stakeholder groups. Interdependencies between scientific rigor, resources and timeliness should be addressed. Operational options for improvements include process studies, literature reviews and sequential mini-HTAs for identification of areas for more elaborate investigations.

Background: Availability of real-time electronic healthcare data provides new opportunities for rapid-cycle evaluation (RCE) of health technologies, including healthcare delivery and payment programs. We aim to align decision-making processes with stages of RCE to optimize the usefulness and impact of rapid results. Rational decisions about program adoption depend on program effect size in relation to externalities, including implementation cost, sustainability, and likelihood of broad adoption.

Methods: Drawing on case studies and experience from drug safety monitoring, we examine how decision makers have used scientific evidence on complex interventions in the past. We clarify how RCE alters the nature of policy decisions; develop the RAPID framework for synchronizing decision-maker activities with stages of RCE; and provide guidelines on evidence thresholds for incremental decision-making.

Results: In contrast to traditional evaluations, RCE provides early evidence on effectiveness and facilitates a stepped approach to decision making in expectation of future regularly updated evidence. RCE allows for identification of trends in adjusted effect size. It supports adapting a program in midstream in response to interim findings, or adapting the evaluation strategy to identify true improvements earlier. The 5-step RAPID approach that utilizes the cumulating evidence of program effectiveness over time could increase policy-makers' confidence in expediting decisions.

Conclusions: RCE enables a step-wise approach to HTA decision-making, based on gradually emerging evidence, reducing delays in decision-making processes after traditional one-time evaluations.

Objectives: Differences in the process of using liver imaging technologies might be important to patients. This study aimed to investigate preferences for scanning modalities used in diagnosing focal liver lesions.

Methods: A discrete choice experiment was administered to 504 adults aged ≥25 years. Respondents made repeated choices between two hypothetical scans, described according to waiting time for scan and results, procedure type, the chance of minor side-effects, and whether further scanning procedures were likely to be required. Choice data were analyzed using mixed-logit models with respondent characteristics used to explain preference heterogeneity.

Results: Respondents preferred shorter waiting times, the procedure to be undertaken with a handheld scanner on a couch instead of within a body scanner, no side-effects, and no follow‑up scans (p ≤ .01). The average respondent was willing to wait an additional 2 weeks for the scan if it resulted in avoiding side-effects, 1.5 weeks to avoid further procedures or to be told the results immediately, and 1 week to have the scan performed on a couch with a handheld scanner. However, substantial heterogeneity was observed in the strength of preference for desirable imaging characteristics.

Conclusions: An average individual belonging to a general population sub‑group most likely to require imaging to characterize focal liver lesions in the United Kingdom would prefer contrast‑enhanced ultrasound over magnetic resonance imaging or computed tomography. Insights into the patient perspective around differential characteristics of imaging modalities have the potential to be used to guide recommendations around the use of these technologies.

Objectives: Colloquial evidence (CE) has been described as the informal evidence that helps provide context to other forms of evidence in guidance development. Despite challenges around quality, and the potential biases, the use of CE is becoming increasingly important in assessments where scientific literature is sparse and to also capture the experience of all stakeholders in discussions, including that of experts and patients. We aimed to ascertain how CE was being used at the National Institute for Health and Care Excellence (NICE).

Methods: Relevant data corresponding to the use of CE was extracted from all NICE technical and process manuals by two reviewers and quality assured and analyzed by a third reviewer. This was considered in light of the results of a focused literature review and a combined checklist for quality assessment was developed.

Results: At NICE, CE is utilised across all guidance producing programmes and at all stages of development. CE could range from information from experts and patient/carers, grey literature (including evidence from websites and policy reports) and testimony from stakeholders through consultation. Six tools for critical appraisal of CE were available from the literature and a combined best practice checklist has been proposed.

Conclusions: As decisions often need to be made in areas where there is a lack of published scientific evidence, CE is employed. Therefore to ensure its appropriateness the development of a validated CE data quality check-list to assist decision makers is essential and further research in this area is a priority.

Objectives: This study describes the views of various stakeholders on the importance of different criteria for priority setting of HIV/AIDS interventions in Indonesia.

Methods: Based on a general list of criteria and a focus group discussion with stakeholders (n = 6), a list was developed of thirty-two criteria that play a role in priority setting in HIV/AIDS control in West-Java province. Criteria were categorized according to the World Health Organization's health system goals and building block frameworks. People living with HIV/AIDS (n = 49), healthcare workers (HCW) (n = 41), the general population (n = 43), and policy makers (n = 22) rated the importance of thirty-two criteria on a 5-point Likert-scale. Thereafter, respondents ranked the highest rated criteria to express more detailed preferences.

Results: Stakeholders valued the following criteria as most important for the priority setting of HIV/AIDS interventions: an intervention's impact on the HIV/AIDS epidemic, reduction of stigma, quality of care, effectiveness on individual level, and feasibility in terms of current capacity of the health system (i.e., HCW, product, information, and service requirements), financial sustainability, and acceptance by donors. Overall, stakeholders’ preferences for the importance of criteria are similar.

Conclusions: Our study design outlines an approach for other settings to identify which criteria are important for priority setting of health interventions. For Indonesia, these study results may be used in priority setting processes for HIV/AIDS control and may contribute to more transparent and systematic allocation of resources.

In their article “Assessment of MAST in European Patient-Centered Telemedicine Pilots” published in this journal (1), Anne Granstrøm Ekeland and Astrid Grøttland describe the results from a questionnaire study of how MAST (Model for Assessment of Telemedicine) (2) was used and perceived by the project managers in twenty-one studies of telemedicine in the European project RENEWING HEALTH.

Background: Health-related quality of life (HRQoL) is an important endpoint of many healthcare interventions. This study develops guidance on how to select appropriate HRQoL measures for inclusion in a clinical trial, given the purposes of the HRQoL measurement.

Methods: The guidance is based on a systematic literature review, discussions with members of the European Network for Health Technology Assessment (EUnetHTA) and two rounds of public consultation.

Results: A set of twelve recommendations was developed, addressing the requirements for HRQoL data for relative effectiveness assessment, for cost-utility analyses and for informing clinical decision making. Recommendations relate to the choice of the type of measure as well as to aspects such as measurement frequency, target population and presentation.

Conclusions: The purpose and context of HRQoL measurement is crucial for the relevance of the data obtained with a specific HRQoL measure. It is recommended to always include a generic HRQoL instrument in clinical trials to cover a wide range of possible future uses of the HRQoL data.

Health systems around the world cope with the challenge of difficult economic times, and the value of health technology assessment (HTA) is increasing. Making the right choices, with limited resources, in the face of increasingly complex technologies requires decisions informed by data and analyses that help us to manage the risks involved. Those who undertake and use HTA can play a greater role in helping decision makers meet these challenges; they need to think how to define innovation and respond to it, how to communicate their analyses, and, critically, how to align their work with the ambitions of their health systems. HTA can become a key health system enabler without compromising its objectivity or independence. It can say that it is too early to determine the value of a new technology when the data simply will not support a safe decision. However, it can also be bold and recommend the managed introduction of new technologies, even when the when the data is immature, provided that the health system understands the risks and there is a plausible case for believing that further research will support the value proposition. The goal for HTA is to be able confidently to do both.

Objectives: This study aims to analyze the key factors considered for the first application of the National Committee for the Evaluation of Medical Devices (CNEDiMTS) for achieving reimbursement through registration in the list of products and services qualifying for reimbursement (LPPR).

Methods: All the appraisals studied on medical devices (MD) for first inclusion in the LPPR during 2011 and 2012 were retrieved from the French National Authority for Health or Haute Autorité de santé (HAS) Web site. A list of relevant factors was analyzed for each included opinion, followed by univariate and multivariate analyses to highlight the key factors that impacted the expected benefit (EB) provided by HAS.

Results: A total of 151 appraisals were included in the study. Of them, 94 (62 percent) were granted with sufficient EB. The manufacturers were mostly from the United States (36 percent), while most of the applicants were from France (84 percent). After adjusting for other retrieved factors, it was observed that MDs complying with the technical standards, requests supported by opinion(s) from previous generation of MD, and the presence of recommendations or guidelines had more probability to obtain a sufficient EB. A lower probability was related to MDs supported by low-quality studies and with no specific health public benefit.

Conclusions: Our results confirmed that manufacturers seeking reimbursement should be aware of the expectations of the health authorities (level of evidence, technical standard, etc.) and foresee their plan of sending requests for funding so that they can provide evidence of good quality.

Objectives: There have been multiple calls for explicit integration of ethical, legal, and social issues (ELSI) in health technology assessment (HTA) and addressing ELSI has been highlighted as key in optimizing benefits in the Omics/Personalized Medicine field. This study examines HTAs of an early clinical example of Personalized Medicine (gene expression profile tests [GEP] for breast cancer prognosis) aiming to: (i) identify ELSI; (ii) assess whether ELSIs are implicitly or explicitly addressed; and (iii) report methodology used for ELSI integration.

Methods: A systematic search for HTAs (January 2004 to September 2012), followed by descriptive and qualitative content analysis.

Results: Seventeen HTAs for GEP were retrieved. Only three (18%) explicitly presented ELSI, and only one reported methodology. However, all of the HTAs included implicit ELSI. Eight themes of implicit and explicit ELSI were identified. “Classical” ELSI including privacy, informed consent, and concerns about limited patient/clinician genetic literacy were always presented explicitly. Some ELSI, including the need to understand how individual patients’ risk tolerances affect clinical decision-making after reception of GEP results, were presented both explicitly and implicitly in HTAs. Others, such as concern about evidentiary deficiencies for clinical utility of GEP tests, occurred only implicitly.

Conclusions: Despite a wide variety of important ELSI raised, these were rarely explicitly addressed in HTAs. Explicit treatment would increase their accessibility to decision-makers, and may augment HTA efficiency maximizing their utility. This is particularly important where complex Personalized Medicine applications are rapidly expanding choices for patients, clinicians and healthcare systems.

Objectives: The aim of this study was to review and compare current health technology assessment (HTA) activities for medical devices across non-European Union HTA agencies.

Methods: HTA activities for medical devices were evaluated from three perspectives: organizational structure, processes, and methods. Agencies were primarily selected upon membership of existing HTA networks. The data collection was performed in two stages: stage 1–agency Web-site assessment using a standardized questionnaire, followed by review and validation of the collected data by a representative of the agency; and stage 2–semi-structured telephone interviews with key informants of a sub-sample of agencies.

Results: In total, thirty-six HTA agencies across twenty non-EU countries assessing medical devices were included. Twenty-seven of thirty-six (75 percent) agencies were judged at stage 1 to have adopted HTA-specific approaches for medical devices (MD-specific agencies) that were largely organizational or procedural. There appeared to be few differences in the organization, process and methods between MD-specific and non–MD-specific agencies. Although the majority (69 percent) of both categories of agency had specific methods guidance or policy for evidence submission, only one MD-specific agency had developed methodological guidelines specific to medical devices. In stage 2, many MD-specific agencies cited insufficient resources (budget, skilled employees), lack of coordination (between regulator and reimbursement bodies), and the inability to generalize findings from evidence synthesis to be key challenges in the HTA of medical devices.

Conclusions: The lack of evidence for differentiation in scientific methods for HTA of devices raises the question of whether HTA needs to develop new methods for medical devices but rather adapt existing methodological approaches. In contrast, organizational and/or procedural adaptation of existing HTA agency frameworks to accommodate medical devices appear relatively commonplace.

Background: Healthcare use and costs are about 81% higher for morbidly obese individuals compared to non-obese persons, and 47% higher compared to the non-morbidly obese population. The benefits of bariatric surgery for health are well established, but its mid-term impact on healthcare use and costs remains controversial.

Methods: This study examines the trends in healthcare use and costs in a Brazilian cohort during a 4-year period before and after surgery. Healthcare use and direct costs related to inpatients and outpatients were retrieved from a healthcare insurance company database from which all cohort members were selected.

Results: Between 2004 and 2010, 4,006 individuals underwent bariatric surgery. Most patients were female (80%) with a mean age of 36.2 years and a mean body mass index of 42.8 kg/m2. Elevated blood pressure was present in 38% of cases and diabetes was found in 12.5% of subjects. Hospital admissions increased consistently after surgery, even after excluding hospitalizations for esthetic surgery and pregnancy-related care. The most prevalent conditions in this group were gastrointestinal diseases. Emergency department visits increased after bariatric procedures, in particular for genitourinary and hematologic problems. Adjusted costs were higher after surgery as assessed during a 4-year follow-up period.

Conclusion: Results indicate that costs and hospital admissions after bariatric surgery increase following this procedure, even when elective interventions are excluded. Healthcare providers and policy makers need to be aware that a decrease in obesity-related diseases following bariatric surgery does not reduce healthcare use and costs.

This article gives a patient organization's perspective on health technology assessments (HTAs) and the role that such organizations can, and should, play in them.

Objectives: Developing a search strategy for use in a systematic review is a time-consuming process requiring construction of detailed search strings using complicated syntax, followed by iterative fine-tuning and trial-and-error testing of these strings in online biomedical search engines.

Methods: Building upon limitations of existing online-only search builders, a user-friendly computer-based tool was created to expedite search strategy development as part of production of a systematic review.

Results: Search Builder 1.0 is a Microsoft Excel®-based tool that automatically assembles search strategy text strings for PubMed (www.pubmed.com) and Embase (www.embase.com), based on a list of user-defined search terms and preferences. With the click of a button, Search Builder 1.0 automatically populates the syntax needed for functional search strings, and copies the string to the clipboard for pasting into Pubmed or Embase. The offline file-based interface of Search Builder 1.0 also allows for searches to be easily shared and saved for future reference.

Conclusions: This novel, user-friendly tool can save considerable time and streamline a cumbersome step in the systematic review process.

Objectives: Equity is one of the founding principles in most healthcare systems. Financial constraints entail an increased risk of exacerbating inequities and a greater need for evidence-based decisions. It is, therefore, both important and timely to enquire how equity can be addressed in health technology assessment (HTA) practice. We aimed to explore related practices from a broad range of HTA agencies, identify exemplary approaches and common concerns, and offer insights for future considerations.

Methods: HTA agencies for which both methodological guides and HTA reports were publicly available were selected from an initial comprehensive pool. Information was extracted on issues ranging from a general commitment to fairness to specific measures targeting both methodological and process-related elements.

Results: Methodological documents and ninety-eight reports from nineteen agencies were analyzed. Our findings indicate that equity was not a standard consideration in HTA report production. The nature of specific approaches and the amount of resources invested into including an equity perspective varied considerably. Specific measures (e.g., appropriate information sources, analytical tools, and schemes) were mentioned by almost half of the agencies analyzed. Albeit sporadic, both horizontal and vertical equity considerations were identified in included HTA reports.

Conclusions: While varying legal contexts and institutional principles can lead to different interpretations of equity at the decision point, a combination of methodological and process-related practices could contribute to more equity-sensitive evaluations, especially in conjunction with enhanced dissemination of existing methodological tools. Networking initiatives on behalf of existing collaborating platforms could play an important role in this direction.

Objectives: The suitability of general HTA methodology for medical devices is gaining interest as a topic of scientific discourse. Given the broad range of medical devices, there might be differences between groups of devices that impact both the necessity and the methods of their assessment. Our aim is to develop a taxonomy that provides researchers and policy makers with an orientation tool on how to approach the assessment of different types of medical devices.

Methods: Several classifications for medical devices based on varying rationales for different regulatory and reporting purposes were analyzed in detail to develop a comprehensive taxonomic model.

Results: The taxonomy is based on relevant aspects of existing classification schemes incorporating elements of risk and functionality. Its 9 × 6 matrix distinguishes between the diagnostic or therapeutic nature of devices and considers whether the medical device is directly used by patients, constitutes part of a specific procedure, or can be used for a variety of procedures. We considered the relevance of different device categories in regard to HTA to be considerably variable, ranging from high to low.

Conclusions: Existing medical device classifications cannot be used for HTA as they are based on different underlying logics. The developed taxonomy combines different device classification schemes used for different purposes. It aims at providing decision makers with a tool enabling them to consider device characteristics in detail across more than one dimension. The placement of device groups in the matrix can provide decision support on the necessity of conducting a full HTA.

Objectives: There is growing interest in implementing hospital-based health technology assessment (HB-HTA) as a tool to facilitate decision making based on a systematic and multidisciplinary assessment of evidence. However, the decision-making process, including the informational needs of hospital decision makers, is not well described. The objective was to review empirical studies analysing the information that hospital decision makers need when deciding about health technology (HT) investments.

Methods: A systematic review of empirical studies published in English or Danish from 2000 to 2012 was carried out. The literature was assessed by two reviewers working independently. The identified informational needs were assessed with regard to their agreement with the nine domains of EUnetHTA's Core Model.

Results: A total of 2,689 articles were identified and assessed. The review process resulted in 14 relevant studies containing 74 types of information that hospital decision makers found relevant. In addition to information covered by the Core Model, other types of information dealing with political and strategic aspects were identified. The most frequently mentioned types of information in the literature related to clinical, economic and political/strategic aspects. Legal, social, and ethical aspects were seldom considered most important.

Conclusions: Hospital decision makers are able to describe their information needs when deciding on HT investments. The different types of information were not of equal importance to hospital decision makers, however, and full agreement between EUnetHTA's Core Model and the hospital decision-makers’ informational needs was not observed. They also need information on political and strategic aspects not covered by the Core Model.

Objectives: A rapid scoping review was performed to support the development of a new clinical technology platform. An iterative sifting approach was adopted to address the challenges posed by the nature of the review question and the extremely large volume of search results to be sifted within the timescales of the review.

Methods: This study describes the iterative sifting approach applied in the scoping review and a preliminary validation of the methods applied.

Results: The searches performed for the rapid scoping review retrieved 27,198 records. This was the full set of records subjected to the staged, iterative sifting approach and the subsequent validation process. The iterative sifting approach involved the screening for relevance of 17,354 (i.e., 63.8 percent) of the 27,198 records. A list of fifty-three potential biomarker names was generated as a result of this iterative sifting method, of which nineteen were selected by clinical specialists for further scrutiny. The preliminary validation involved the exhaustive sifting of the remaining 9,844 previously unsifted records. The validation process identified sixteen additional potential biomarker names not identified by the iterative sifting process. The clinical specialists subsequently concluded that none were of further clinical interest.

Conclusions: This study describes an approach to the screening of search records that can be successfully applied in appropriate review and decision problems to allow the prioritization of the most relevant search records and achieve time savings. Following further refinement and standardization, this iterative sifting method may have potential for further applications in reviews and other decision problems.

Objectives: We sought to evaluate patient involvement (consultation and direct participation) in the assessment of alternative measures to restraint and seclusion among adults in short-term hospital wards (in psychiatry) and long-term care facilities for the elderly.

Methods: We conducted individual semi-structured interviews with thirteen stakeholders: caregivers, healthcare managers, patient representatives, health technology assessment (HTA) unit members, researchers, and members of the local HTA scientific committee. Data were collected until saturation. We carried out content analysis of two HTA reports and four other documents that were produced in relation with this HTA. We also used field notes taken during formal meetings and informal discussions with stakeholders. We performed thematic analysis based on a framework for assessing patient involvement in HTA. We then triangulated data.

Results: For the majority of interviewees, patient consultation enriched the content of the HTA report and its recommendations. This also made it possible to suggest other alternatives that could reduce the use of restraint and seclusion and helped confirm some views and comments from healthcare professionals consulted in this HTA. The direct participation of patient representatives enabled rephrasing of some findings so as to bring the patient perspective to the HTA report.

Conclusions: Patient consultation was seen as having directly influenced the content of the HTA report while direct participation made it possible to rephrase some findings. This is one of few studies to assess the impact of patient involvement in HTA and more such studies are needed to identify the best ways to improve the input of such involvement.

Are we in a period of significant change? Is health technology assessment (HTA) undergoing a fundamental transformation? Or should it be? Are we in the middle of a paradigm shift? These are some of the questions the paper by Facey et al in this journal (1) raises about the future of HTA, based on discussions at the HTAi Policy Forum in February 2015. To further share the deliberations of the Forum and with a view to opening this debate among the wider HTA community, a panel within the HTAi 2015 Annual Meeting in Oslo was organized. Presentations at the panel included a summary of the HTAi Policy Forum deliberations and perspectives from a patient, a clinician, and representatives of an HTA organization, industry, and a health system. This letter presents issues and thoughts raised in the panel session.