Objectives: This article provides a detailed understanding of the differences in selected formulary submission guidelines supplied by various health technology assessment (HTA) agencies and indicates how these differences can impact the evidence base used to populate the HTA.

Methods: Detailed summaries of the recommended methods for evidence generation, organized by topic areas relevant for clinical and economic data, for twelve countries in Europe, North America, and Australia where HTA processes are well developed were prepared. Using these summaries, we provide examples of the likely impact these differences in recommended methods could have on the evidence base used to evaluate new health technologies.

Results: Areas where recommendations differed included methodologies for systematic literature reviews (e.g., preferred databases and study designs for inclusion); selection of appropriate comparators; guidance on critical appraisal and synthesis of clinical evidence; appropriate sources for health value measures, resource use, and cost data; and approaches to uncertainty analyses. Performing literature searches that capture all relevant studies and then creating subsets of the literature based on a listing of country-specific requirements could allow for direct comparison of the evidence bases associated with the different guidelines.

Conclusions: If the formulary submission guidelines were followed as written, different (although overlapping) bodies of evidence likely would be generated for each country, which could contribute to disparate assessments and recommendations. This comparison of the formulary submission guidelines could contribute to an understanding of why clinical and reimbursement decisions vary across countries.

The Problem: There is a shortage and maldistribution of medically trained health professionals to deliver cost-effective maternal and child health (MCH) services. Hence, cost-effective MCH services are not available to over half the population of Uganda and progress toward the Millennium Development Goals for MCH is slow. Optimizing the roles of less specialized health workers (“task shifting”) is one strategy to address the shortage and maldistribution of more specialized health professionals.

Policy Options: (i) Lay health workers (community health workers) may reduce morbidity and mortality in children under five and neonates; and training for traditional birth attendants may improve perinatal outcomes and appropriate referrals. (ii) Nursing assistants in facilities might increase the time available from nurses, midwives, and doctors to provide care that requires more training. (iii) Nurses and midwives to deliver cost-effective MCH interventions in areas where there is a shortage of doctors. (iv) Drug dispensers to promote and deliver cost-effective MCH interventions and improve the quality of the services they provide. The costs and cost-effectiveness of all four options are uncertain. Given the limitations of the currently available evidence, rigorous evaluation and monitoring of resource use and activities is warranted for all four options.

Implementation Strategies: A clear policy on optimizing health worker roles. Community mobilization and reduction of out-of-pocket costs to improve mothers’ knowledge and care-seeking behaviors, continuing education, and incentives to ensure health workers are competent and motivated, and community referral and transport schemes for MCH care are needed.

Background: To ensure rapid access to new potentially beneficial health technologies, obtain best value for money, and ensure affordability, healthcare payers are adopting a range of innovative reimbursement approaches that may be called Managed Entry Agreements (MEAs).

Methods: The Health Technology Assessment International (HTAi) Policy Forum sought to identify why MEAs might be used, issues associated with implementation and develop principles for their use. A 2-day deliberative workshop discussed key papers, members' experiences, and collectively addressed four policy questions that resulted in this study.

Results: MEAs are used to give access to new technologies where traditional reimbursement is deemed inappropriate. Three different forms of MEAs have been identified: management of budget impact, management of uncertainty relating to clinical and/or cost-effectiveness, and management of utilization to optimize performance. The rationale for using these approaches and their advantages and disadvantages differ. However, all forms of MEA should take the form of a formal written agreement among stakeholders, clearly identifying the rationale for the agreement, aspects to be assessed, methods of data collection and review, and the criteria for ending the agreement.

Conclusions: MEAs should only be used when HTA identifies issues or concerns about key outcomes and/or costs and/or organizational/budget impacts that are material to a reimbursement decision. They provide patient access and can be useful to manage technology diffusion and optimize use. However, they are administratively complex and may be difficult to negotiate and their effectiveness has yet to be evaluated.

Objectives: The aim was to learn about perspectives of consumers contributing to the work of the Division of Research, Audit and Academic Surgery of the Royal Australasian College of Surgeons. The research arm of the Division has worked with consumers since it was formed in 1998.

Methods: Nine consumers who worked with the Division over the past 5 years completed (1) a written survey focused on their background and past experience, and (2) a semi-structured phone interview focused on their motivations for becoming involved in this work; their role; the evolution of the role of consumers in healthcare research; and what health information for consumers should contain.

Results: Participants came from various backgrounds and had different motivations for being involved. A common theme was concern about uncertainties in surgery and the need to provide consumers with information about potential benefits and risks of a procedure. Participants believed that a consumer presence was vital in research on surgical procedures, and that the content and wording of consumer information must be chosen carefully in order for the public to use it in a meaningful way. They also acknowledged the changing role of the consumer, who was rapidly becoming a partner in the doctor–patient relationship.

Conclusions: In surgical research and audit, the consumer perspective is unique and informed by a wealth of experience. The findings of this study may be of interest to other health technology assessment and associated agencies seeking to involve consumers within their own research process.

Objectives: The aim of this study was to establish a list of priority topics for methods research to support decision making at the National Institute for Health and Clinical Excellence (NICE).

Methods: Potential priorities for methods research topics were identified through a focused literature review, interviews, an email survey, a workshop and a Web-based feedback exercise. Participants were members of the NICE secretariat and its advisory bodies, representatives from academia, industry, and other organizations working closely with NICE. The Web exercise was open to anyone to complete but publicized among the above groups.

Results: A list of potential topics was collated. Priorities for further research differed according to the type of respondent and the extent to which they work directly with NICE. Priorities emerging from the group closest to NICE included: methodology for indirect and mixed treatment comparisons; synthesis of qualitative evidence; research relating to the use of quality-adjusted life-years (QALYs) in decision making; methods and empirical research for establishing the cost-effectiveness threshold; and determining how data on the uncertainty of effectiveness and cost-effectiveness data should be taken into account in the decision-making process. Priorities emerging from the broadest group of respondents (through the Web exercise) included: methods for extrapolating beyond evidence observed in trials, methods for capturing benefits not included in the QALY and methods to assess when technologies should be recommended in the context of further evidence gathering.

Conclusions: Consideration needs to be given to the needs of those who use the outputs of research for decision making when determining priorities for future methods research.

Objectives: This study comprises a review of public involvement strategies across the five stages of research management in the UK's HTA program at the end of a 10-year period. These five stages are: identification of topics; prioritization of these topics as researchable questions; commissioning of research; monitoring of projects throughout their implementation; and publication and dissemination of findings.

Methods: Internal HTA documentation was analyzed alongside narrative analysis of semi-structured interviews of program staff, and a rapid review of published literature.

Results: Public involvement strategies have developed with the growth of the HTA program but were spread unevenly across the five stages of research management. Public involvement was present in identification, strongest in prioritization, present in commissioning but minimal in monitoring and absent in publication and dissemination.

Conclusions: The HTA program has developed public involvement strategies but mainly in prioritization. Further research is required to ascertain where public involvement can be most appropriately used and to evaluate its impact.

Objectives: The primary objective of this study was to determine the extent to which international standards on transparency and quality are met by the health technology assessment (HTA) process in Poland. A secondary objective is to describe the outcomes of the HTA process and their associated factors.

Methods: All published online HTA appraisal and meeting proceedings on pharmaceutical products in 2008 were reviewed using a score card developed from international checklists recommended by INAHTA and ECHTA.

Results: The sixty-nine reports reviewed showed that five of nine transparency standards and six of eight quality standards were usually met by the HTA reports. Areas for improvement for transparency include inputs from external stakeholders, availability of English summaries, conclusions, implications of results, and suggested program of action. Areas of improvement for quality include appropriateness of target population and comparator/s, sufficiency of evidence on efficacy and safety, methodological rigor, economic model assumptions, and adaptation to the Polish setting. A consideration of the ethical and social consequences to the healthcare system must also be strengthened.

Conclusions: The study demonstrates that the incorporation and implementation of the HTA appraisal process in Poland has been successful. HTA appraisal reports in Poland have considered most of the international standards of transparency and quality. Recommendations for both HTA users and doers are forwarded for the improvement of the HTA process in the Polish setting.

In their article, Niederstadt and Droste (1) suggest a clear representation of all search strategies for each database to be important. We agree but would like to suggest an even more robust representation of search strategies: a search strategy should be displayed as a string, rather than as a table. The advantage of this method of display is three-fold: it is clearer, it is less ambiguous, and it is easier to reproduce, adapt and apply in a database. Niederstadt and Droste also suggest to indicate the number of results per query. We deem this to be irrelevant. An indication of the number of results for each aspect would be more useful. Likewise, we consider the inclusion of a field code behind each term to be important as well. This will prevent ambiguity regarding the interpretation of the search by the search engine.

Background: The Institute for Quality and Efficiency in Health Care (IQWiG) was established in 2003 by the German parliament. Its legislative responsibilities are health technology assessment, mostly to support policy making and reimbursement decisions. It also has a mandate to serve patients’ interests directly, by assessing and communicating evidence for the general public.

Objectives: To develop a priority-setting framework based on the interests of patients and the general public.

Methods: A theoretical framework for priority setting from a patient/consumer perspective was developed. The process of development began with a poll to determine level of lay and health professional interest in the conclusions of 124 systematic reviews (194 responses). Data sources to identify patients’ and consumers’ information needs and interests were identified.

Results: IQWiG's theoretical framework encompasses criteria for quality of evidence and interest, as well as being explicit about editorial considerations, including potential for harm. Dimensions of “patient interest” were identified, such as patients’ concerns, information seeking, and use. Rather than being a single item capable of measurement by one means, the concept of “patients’ interests” requires consideration of data and opinions from various sources.

Conclusions: The best evidence to communicate to patients/consumers is right, relevant and likely to be considered interesting and/or important to the people affected. What is likely to be interesting for the community generally is sufficient evidence for a concrete conclusion, in a common condition. More research is needed on characteristics of information that interest patients and consumers, methods of evaluating the effectiveness of priority setting, and methods to determine priorities for disinvestment.

The article by O'Malley and Jordan (2009) contains fundamental errors and misunderstands the process it claims to describe (4).

The premise of the paper is that the Australian and New Zealand Horizon Scanning Network (ANZHSN) “was set up to scan the introduction of new and emerging medical technologies into the public sector, with consideration to the private sector.” However, the terms of reference of the policy body (HealthPACT) directing the ANZHSN's activities state that its role is to “provide a forum for monitoring, reviewing, and sharing information regarding health technology to inform service planning and policy development in the public sector” (1). HealthPACT is largely a jurisdictional committee, with representation from the Australian Government, each of its seven states and territories, the New Zealand Health Ministry and district health boards.

Thank you for the opportunity to reply to the letter by Mundy, Hiller, and Merlin “The true role of horizon scanning in Australia: Who it informs and why”. This letter states that our paper (1) “Horizon scanning of new and emerging medical technology in Australia: Its relevance to Medical Services Advisory Committee health technology assessments and public funding,” published in July 2009, contains fundamental errors and misunderstands the process it claims to describe.

Many thanks to van der Hoorn and colleagues for their interest in our article and the interesting discussion points. Their comments are very important and deserve to be answered.

Objectives: The aim of this study was to describe the process and challenges of developing a decision aid for the national public breast cancer screening program in Finland.

Methods: An expert team with stakeholder representation used European guidelines and other literature as basis for selecting relevant content and format for the decision aid for breast cancer screening. Feedback from women was sought for the draft documents.

Results: A decision aid attached to the invitation letter for screening was considered the best way to ensure access to information. In addition, tailored letter templates for all phases of the screening program, a poster, and a public website were developed. Initial feedback from users (women, professionals, and service providers), was mainly positive. Six months after publishing, the implementation of the decision aid was still incomplete.

Conclusions: Providing balanced information for women invited to breast cancer screening is demanding and requires careful planning. Professionals and service providers need to be engaged in the HTA process to ensure proper dissemination and implementation of the information. End user participation is essential in the formulation of information. There is a need to follow up the implementation of the decision aid.

Objectives: The aim of this study was to develop a breast cancer Patient Decision Aid (PDA), using a Health Technology Assessment (HTA) process, to assist patients in their choice of therapeutic options, and to promote shared decision making among patients, healthcare professionals, and other interested parties.

Methods: A systematic review (SR) was conducted of existing breast cancer patient Decision Aids encountered in the main scientific journal databases and on institutional Web sites that create PDAs, together with a Qualitative Research (QR) study, using semi-structured interviews and focus group with stakeholders (patients, family members, and health professionals), with the aim of developing a PDA for breast cancer.

Results: The SR shows that PDAs in breast cancer not only increase patient knowledge of the illness, leading to more realistic expectations of treatment outcomes, but also reduce passivity in the decision-making process and facilitate the appropriate choice of treatment options in accordance with patient medical and personal preferences. The analysis of QR shows that both breast cancer patients and healthcare professionals agree that surgery, adjuvant treatments, and breast reconstruction represent the most important decisions to be made. Worry, anxiety, optimism, and trust in healthcare professionals were determined as factors that most affected patients subjective experiences of the illness. This HTA was used as the basis for developing a PDA software program.

Conclusions: The SR and QR used in the development of this PDA for breast cancer allowed patients to access information, gain additional knowledge of their illness, make shared treatment decisions, and gave healthcare professionals a deeper insight into patient experiences of the disease.

Cleemput et al. make a point that the incremental cost-effectiveness ratio (ICER) alone is not a sufficient criterion to guide decision making in health care, and needs many other supplementary inputs. This is nothing new, it has been well known for years to researchers and decision makers alike. ICER serves as an important ingredient to guide decision making, at least in some healthcare systems.

Background: Patient involvement is widely acknowledged to be a valuable component in health technology assessment (HTA) and healthcare decision making. However, quantitative approaches to ascertain patients' preferences for treatment endpoints are not yet established. The objective of this study is to introduce the analytic hierarchy process (AHP) as a preference elicitation method in HTA. Based on a systematic literature review on the use of AHP in health care in 2009, the German Institute for Quality and Efficiency in Health Care (IQWiG) initiated an AHP study related to its HTA work in 2010.

Methods: The AHP study included two AHP workshops, one with twelve patients and one with seven healthcare professionals. In these workshops, both patients and professionals rated their preferences with respect to the importance of different endpoints of antidepressant treatment by a pairwise comparison of individual endpoints. These comparisons were performed and evaluated by the AHP method and relative weights were generated for each endpoint.

Results: The AHP study indicates that AHP is a well-structured technique whose cognitive demands were well handled by patients and professionals. The two groups rated some of the included endpoints of antidepressant treatment differently. For both groups, however, the same six of the eleven endpoints analyzed accounted for more than 80 percent of the total weight.

Conclusions: AHP can be used in HTA to give a quantitative dimension to patients' preferences for treatment endpoints. Preference elicitation could provide important information at various stages of HTA and challenge opinions on the importance of endpoints.