Objectives: The aim of this study was to predict the potential impact of the introduction of implantation of polyurethane scaffold for the treatment of partial meniscal lesions in the South Korean healthcare system.

Methods: The horizon scanning process was used to select a target technology and assess its potential impacts on patients and the Korean healthcare system. We identified and filtered research-phase health technologies that are not listed yet in Korean, but appear promising. After a process of prioritization, we chose the implantation of polyurethane scaffolds as a target technology. Then, through the procedures of assessment and peer review, we analyzed current evidence and its predicted potential impacts.

Results: There were eight studies included in the review: one prospective cohort and seven case-series studies. Six revealed significant improvements in function and pain relief. Of the six studies, which reported safety endpoints, four stated no major postoperative complications related to scaffold, and two reported adverse events and serious adverse events such as pain, joint swelling, et cetera. We also included the potential impact of this technology based on the experts’ consultation. They all agreed that it would satisfy the diverse needs of patients and fulfill clinical needs. However, the majority of related clinical studies were based on short-term follow-up observations without any validation process involving comparison with control groups.

Conclusions: Through a horizon scanning activity, we found that the implantation of polyurethane scaffolds is a promising technology to resolve articular cartilage defects; however, long-term evidence with comparison groups for safety and effectiveness is required.

Objectives: Economic decision-support tools can provide valuable information for tobacco control stakeholders, but their usability may impact the adoption of such tools. This study aims to illustrate a mixed-method usability evaluation of an economic decision-support tool for tobacco control, using the EQUIPT ROI tool prototype as a case study.

Methods: A cross-sectional mixed methods design was used, including a heuristic evaluation, a thinking aloud approach, and a questionnaire testing and exploring the usability of the Return of Investment tool.

Results: A total of sixty-six users evaluated the tool (thinking aloud) and completed the questionnaire. For the heuristic evaluation, four experts evaluated the interface. In total twenty-one percent of the respondents perceived good usability. A total of 118 usability problems were identified, from which twenty-six problems were categorized as most severe, indicating high priority to fix them before implementation.

Conclusions: Combining user-based and expert-based evaluation methods is recommended as these were shown to identify unique usability problems. The evaluation provides input to optimize usability of a decision-support tool, and may serve as a vantage point for other developers to conduct usability evaluations to refine similar tools before wide-scale implementation. Such studies could reduce implementation gaps by optimizing usability, enhancing in turn the research impact of such interventions.

The Austrian healthcare system relies mainly on physicians in private practice and on various services provided by hospitals. The social health insurance scheme is compulsory, covering 99% of the population. The system is very decentralized. While the federal state provides the framework, the nine autonomous provinces are responsible for administering health and social services. There is ongoing public discussion about centralizing the healthcare system to make it more efficient and to enforce structural reforms. Because of concerns about healthcare expenditures, in 1997 the Performance-Related Hospital Financing System (LKF), a system similar to the diagnosis-related group system, was introduced for hospitals, including a plan for large medical devices. It is too early to evaluate the success of this new system, although some effects of the LKF system that could have been anticipated, such as shortened lengths of stay and more hospitalizations, have been seen. Previously, health technologies have been almost uncontrolled in Austria. The evaluation of health technologies as an instrument to support or to control their dissemination and use or to help define policies is not institutionalized or systematically used. It seems clear that structural reforms of the Austrian healthcare system are needed. Health technology assessment should be part of such reforms.

The use of probabilistic analysis as a means for analyzing uncertainty within economic analysis has grown in popularity in recent years as it has been recognized as the most complete method for propagating uncertainty with respect to input parameters in terms of uncertainty about outcomes of interest. The World Health Organization (WHO) in a series of recent reports and publications have recognized the role of probabilistic analysis in what they term generalized cost-effectiveness analysis. However, there are fundamental problems with the analysis and the interpretation of such analysis as proposed by WHO. This study highlights three specific points for concern and offers constructive criticism by recommending more appropriate approaches.

Objectives: The aim of this study was to determine the aspects of expert advice that decision makers find most useful in the development of evidence-based guidance and to identify the characteristics of experts providing the most useful advice.

Methods: First, semi-structured interviews were conducted with seventeen members of the Interventional Procedures Advisory Committee of the UK's National Institute of Health and Care Excellence. Interviews examined the usefulness of expert advice during guidance development. Transcripts were analyzed inductively to identify themes. Second, data were extracted from 211 experts’ questionnaires for forty-one consecutive procedures. Usefulness of advice was scored using an index developed through the qualitative work. Associations between usefulness score and characteristics of the expert advisor were investigated using univariate and multivariate analyses.

Results: Expert opinion was seen as a valued complement to empirical evidence, providing context and tacit knowledge unavailable in published literature, but helpful for interpreting it. Interviewees also valued advice on the training and experience required to perform a procedure, on patient selection criteria and the place of a procedure within a clinical management pathway. Limitations of bias in expert opinion were widely acknowledged and skepticism expressed regarding the anecdotal nature of advice on safety or efficacy outcomes. Quantitative analysis demonstrated that the most useful advice was given by clinical experts with direct personal experience of the procedure, particularly research experience.

Conclusions: Evidence-based guidance production is often characterized as a rational, pipeline process. This ignores the valuable role that expert opinion plays in guidance development, complementing and supporting the interpretation of empirical data.

The Special Interest Group on Ethics and HTA (health technology assessment) has invited two renowned philosophers, Norman Daniels from Harvard University and Henry Richardson from Georgetown University to reflect on the role of HTA in healthcare policy making. Both acknowledge its importance, but at the same time warn against a too mechanistic deployment of HTA. In their view, the relevance of HTA to healthcare policy making would considerably be enhanced if it were subsumed within a broader deliberative framework. Why should this be so? What is there to deliberate on, who should do the deliberating, where and when, and how does this relate to the more technical elements of HTA such as evidence synthesis and economic modeling?

Objectives: The aim of this study was to examine the influence of participants' characteristics on the results produced by formal consensus methods.

Methods: The approach was an experimental study of 346 participants in 20 groups rating the appropriateness of four mental health interventions for the treatment of chronic fatigue syndrome, irritable bowel syndrome, and chronic back pain. There were four factors in the design: systematic literature review provided or not, decisions made under realistic or “ideal” resource assumptions, clinically mixed (general practitioners and mental health professionals) or homogenous group (general practitioners only), convened or mail-only group. A group's rating was defined as the median of participants' ratings. The influence of participants' characteristics (age, sex, and specialty) was examined using multilevel models.

Results: The largest differences were between the GPs and mental health professionals, both in their initial ratings of the different interventions, and in how much they altered their ratings between rounds. There were smaller but statistically significant (p<.05) differences between specialty and age groups in initial ratings for the treatment (by whatever means) of different conditions, and for certain conditions women increased their ratings more than men. Women rated intervention more favorably when assuming “ideal” rather than realistic levels of resources, but men did not.

Conclusions: Our findings support the practice of treating professional specialty as an important determinant of the results in consensus panels.

Objectives: Economic evaluations, although not formally used in purchasing decisions for medical devices in Canada, are still being conducted and published. The aim of this study was to examine the way that prices have been included in Canadian economic evaluations of medical devices.

Methods: We conducted a review of the economic concepts and implications of methods used for economic evaluations of the eleven most implanted medical devices from the Canadian perspective.

Results: We found Canadian economic studies for five of the eleven medical devices and identified nineteen Canadian studies. Overall, the device costs were important components of total procedure cost, with an average ratio of 44.1 %. Observational estimates of the device costs were obtained from buyers or sellers in 13 of the 19 studies. Although most of the devices last more than 1 year, standard costing methods for capital equipment was never used. In addition, only eight studies included a sensitivity analysis for the device cost. None of the sensitivity analyses were based on actual price distributions.

Conclusions: Economic evaluations are potentially important for policy making, but although they are being conducted, there is no standardized approach for incorporating medical device prices in economic analyses. Our review provides suggestions for improvements in how the prices are incorporated for economic evaluations of medical devices.

Objectives: The aim of this study was to implement health technology assessment (HA) in the First General City Hospital in Astana, Kazakhstan.

Methods: We organized trainings to familiarize hospital staff with the purpose and details of HTA. An HTA committee was established, with representation from hospital physicians and managers, and criteria for prioritization of health technologies determined. Clinical departments of the hospital were asked to prepare applications for new technologies for their services.

Results: The HTA committee reviewed five applications and selected a technology from one of these, on single incision laparoscopic surgery (SILS), for assessment. A short HTA report on SILS was prepared, covering its safety, clinical effectiveness, and cost effectiveness. The report was used to support a request to the Department of Health for additional funding to implement this technology within the hospital. This funding was approved and SILS was established in several hospital departments.

Conclusions: This successful initial experience with HTA has paved the way for its routine use by the hospital for informing decisions on the procurement and use of new health technologies.

The assessment of value within healthcare is undergoing a major transformation. Gone are the days when a new mechanism of action alone would be regarded as a high value innovation. Today, there is much more of an emphasis on what are the outcomes (i.e., mortality and morbidity benefits), how does this compare with the current standard of care and what is the impact to the usage of healthcare resources. This emphasis is quite understandable given the financial crisis we are experiencing with an ever growing and aging population that is placing considerable strain on the healthcare system.

Objectives: The purpose of the European network for Health Technology Assessment (EUnetHTA) is to make HTA agencies collaborate sharing methods and tools thus avoiding duplication of evaluative efforts and allowing resource savings. From 2010 to 2012, the activities of the network were carried out through EUnetHTA Joint Action 1 and Work Package 4 Strand B aimed at producing two Core HTAs with two main objectives: to test the Web based Core model and the collaborative working models. Our objective in this article is to give an historical record of the Work Package activities highlighting what worked and what did not in the collaboration of researchers’ groups coming from different agencies.

Methods: A retrospective description of all the steps for the joint production of the two Core HTAs is provided starting from the first step of selecting technologies of common interest. Primary researchers’ views on the whole process have been collected through a semi-structured telephonic interview supported by a questionnaire. Coordinators views were gathered during internal meetings and validated.

Results: Majority of respondents thought topic selection procedure was not clear and well managed. About collaborative models, small groups were seen to enable more exchange, whatever the model. According to coordinators, loss of expertise and experience during the production process, different languages, and novelty of the Online Tool were main barriers.

Conclusions: Lessons learned from this first experience in Joint Action 1 paved the path for the collaboration in Joint Action 2, as it allowed enhancements and changes in models of collaborations and coordination.

Objective: To describe how scientific evidence has influenced healthcare policy making in Belgium in the field of sickness prevention for mammography, PSA testing in prostate cancer screening, and use of ultrasound in pregnancy.

Methods: Review of published and gray literature and interviews with stakeholders and experts.

Results: At the end of 1999, a systematic national/regional screening program had not yet been implemented for any of the three screening strategies. A systematic breast cancer screening program is being prepared for implementation only in Flanders. This limited impact can be attributed to the fragmentation in healthcare policy, the different options among the different regions, fragmentation in healthcare practice, the strong impact of healthcare stakeholders (provider groups and sickness funds) on decision making, and limited attention to scientific evidence in health policy and technology assessment.

Conclusions: Health technology assessment has had very little impact on policy and practice in use of mammography, PSA testing, and ultrasound in pregnancy in Belgium.

Objective: We have used multidisciplinary expert panels to assess the health benefits from two different emergency medical service programs in Norway. This gave the opportunity to study the reliability of the expert panel method.

Methods: Two panels assessed case reports for 18 children, and two other panels assessed case reports for 64 adult patients. The assessments of each case report were compared. These assessments were also compared with assessments of the same case reports, done by the same panels 1 and 9 years earlier.

Results: Two different panels agreed on the benefit/no benefit conclusion in at least 75% of the patients, both for children and adult patients (kappa 0.88–0.50). For groups of patients assessed to have some health benefit, the magnitude of the benefit estimates differed by 25% between the panels. When the same panels assessed the same patient groups twice, 1 and 9 years apart, their estimates of total benefit differed up to 30%. However, estimates for single patients, as well as estimates from single panel members, varied considerably more.

Conclusions: Use of multidisciplinary expert panels is a useful method for estimating health benefits on program level or for groups of patients. But assessments from single panelists, and for single patients may be seriously biased.

Objectives: A randomized controlled trial (RCT) showed magnetic resonance imaging for patients waiting for knee arthroscopy did not reduce the number of arthroscopies. Our study aimed to identify decisions made by orthopedic surgeons about whether patients on a waiting list should proceed to arthroscopy, and to describe surgeons’ decisions.

Methods: Five surgeons were asked to Think Aloud (TA) as they made their decisions for twelve patients from the original RCT. Audiotapes of the decision making were transcribed for analysis.

Results: For five patients, surgeons agreed about proceeding with arthroscopy, although reasoning differed. In no cases did surgeons agree about not proceeding to arthroscopy. Agreement was more likely in patients with clinically diagnosed meniscal abnormality, and less likely in patients with osteoarthritis.

Conclusions: Surgeons’ decisions were influenced by patient wishes. For some patients, the decision to proceed with arthroscopy was based solely on clinical diagnosis; MRI may not be advantageous in these instances. Surgeons disagreed more often than they agreed about the decision to proceed with arthroscopy, particularly when OA was diagnosed. This has implications for decision making in the current NHS patient choice environment. Patients may choose a treatment provider from a list of available providers at time of original clinical assessment and diagnosis. The treating surgeon does not necessarily re-examine the patient until the day of surgery. Given the variation between surgeons about the merits of proceeding with arthroscopy, surgeons may end up in the invidious position of providing surgery to patients whom they do not believe will benefit from arthroscopy.

Objectives: As health care budgets continue to face close scrutiny, any new acquisition must be evaluated for both costs and outcomes. This study was undertaken to demonstrate the application of an economic framework for the evaluation of a multileaf collimator as an example of a new technology that can have both quantifiable and nonquantifiable benefits for patients, staff, and cancer care institutions.

Methods: Using financial data from the Northeastern Ontario Regional Cancer Centre (NEORCC) and a recognized staffing model, a commercial spreadsheet, developed to economically characterize the principal radiotherapy processes has been used to determine the net incremental annual cost of a multileaf collimator (MLC).

Results: The incremental annual cost of purchasing an MLC is estimated at approximately $85,000 (1997 CDN $). Without increasing patient throughput, this increases the average cost of a course of radiotherapy by approximately CDN $200. Savings can be accrued by decreasing mold room activity, increasing the hourly patient capacity on each treatment machine, and decreasing sick time due to strain injuries.

Conclusions: Although the clinical outcome of techniques facilitated by MLCs, such as intensity-modulated radiation therapy, are unknown at this time, an economic context within which to objectively evaluate this technology is presented. The framework presented suggests a method of quantifying outcome-justified expenditures, such as improved patient outcome and greater treatment flexibility, which, when offset against the incremental annual equipment cost, may be used to help justify the acquisition of multileaf technology.