The preparation of HTA reports requires time, effort and financial resource. There is a need for different HTA agencies to share knowledge and information (Reference Turner, Chase and Milne8); also some agencies are under-resourced and consequently unable to produce the volume of outputs desirable (Reference Turner, Chase and Milne8). The adaptation of existing HTA reports may save time and resource, help increase output, avoid the duplication of effort, and facilitate the sharing of knowledge and information.
The process of adaptation is the systematic extraction of relevant information from an existing HTA report (or reports), and the use of this material for a new report for the desired new context. There is a spectrum of adaptation, with progressively more of the original text being used (Reference Turner, Chase and Milne8). A toolkit has been developed to aid in this process of adaptation. This instrument is part of the output from the European network for Health Technology Assessment (EUnetHTA) Project (3;Reference Kristensen, Lampe and Chase4) Work Package 5 (WP5) (2).
The toolkit is a collection of resources that helps the user assess whether data and information in existing HTA reports should and could be adapted for their own setting. These resources are in the form of checklists of questions on relevance, reliability, and transferability of data and information and links to useful Web sites. The domains covered by the toolkit are technology use, safety, effectiveness, economic evaluation, and organizational aspects. WP5 members agreed that legal, ethical, and social aspects would be beyond the scope of the toolkit.
HTA reports can contain primary research, or a synthesis of evidence. This toolkit supports the adaptation of syntheses of evidence. Examples are literature reviews, systematic reviews, meta-analyses, decision analyses, and consensus statements. This is research that does not generate primary data but involves the qualitative or quantitative synthesis of information from multiple primary studies. Adaptation of HTA reports that are primary research is not addressed in this toolkit.
Currently this toolkit may be accessed at: http://www.eunethta.net/upload/WP5/EUnetHTA_HTA_Adaptation_Toolkit_October08.pdf. It is also available in paper form and will be published within the journal series HTA Assessment (Reference Chase, Rosten and Turner1). Work currently under way aims to make this resource available in interactive form, as a Web-based tool.
This study aims to, first, describe the structure of the adaptation toolkit, and, second, to describe its use, guiding the user through the process of assessing the relevance to their current purpose of other existing HTA report(s), and the selection of possible relevant material from these other report(s) for adaptation for their own context.
METHODS
The toolkit was developed by a partnership of HTA agencies and networks from across Europe (known as EUnetHTA Work Package 5 [WP5]) (2). The methods used for its development have been described previously (Reference Chase, Rosten and Turner1;Reference Turner, Chase and Milne8).
The toolkit consists of a series of questions, checklists and resources. Where a local policy or research question has been posed, and literature searches have been performed to identify other existing HTA report(s) which may be of use in answering, or helping to answer the question(s); the role of the toolkit is to guide the user through the process of selecting possible relevant material from these report(s), assessing the relevance, reliability, and transferability of this information, and adapting it for the desired context. The toolkit achieves this by asking the user to progress systematically through the checklist of questions within the toolkit.
A glossary of HTA adaptation terms has been developed to accompany the toolkit. The aim was to give clarification of the terms used so that a common understanding could be established. Further details of the development, structure, and function of the glossary are given in Rosten et al. (Reference Rosten, Chase and Hicks7).
RESULTS
The toolkit contains two sections: (i) Speedy sifting—A screening tool to enable speedy sifting of HTA reports to assess the relevance of an HTA report for adaptation; and (ii) Main toolkit—A more comprehensive tool with questions on reliability and issues regarding the transferability of the information.
Speedy Sifting. The toolkit directs the user through a pathway of questions, beginning with the section called speedy sifting. This section enables the user to assess the relevance of an HTA report (or parts of that report) for adaptation, that is, is the policy and/or research question posed sufficiently similar to warrant adaptation of this report? The aim is that users could make a decision on each HTA report within 2 hours.
The first two questions in this section can result in either proceeding to the following question with a “yes” response, or ending the process with a “no” response. The subsequent six questions require judgments to be made by the user. The user must decide based on responses to these questions, whether to end the process, or proceed to the main part of the toolkit.
Main Toolkit. The main part of the toolkit is designed to help HTA agencies adapt HTA reports by questioning and helping to assess three aspects of information in a report: (i) Relevance of the report, that is, is the policy and/or research question posed sufficiently similar to warrant adaptation of this report?; (ii) Reliability, that is, an assessment of the quality of the report; and (iii) Transferability, that is, guidance on issues for consideration when applying information/data to a local context.
Currently, the adaptation toolkit includes five domains, as shown in Box 1. The main part of the toolkit can be used only to adapt information and/or data contained in an HTA report that includes one or more of these five domains. Domains not included are legal, social, or ethical aspects. These aspects were considered to be too context specific and not easily adapted between contexts.
Box 1. Domains Included in the EUnetHTA Adaptation Toolkit
An aim in developing the toolkit was that it should be possible to use it to adapt a report with no more than 5 person-days of work. The toolkit can be used to adapt an entire HTA report or parts of it. Hence, it may not be necessary for users to work through the whole of the main section of the toolkit. However, all users should undertake speedy sifting before using the more comprehensive tool.
Figure 1 gives an outline of the stages of adaptation and the role of the toolkit.
Figure 1. Stages of adaptation, from input to output and role of the toolkit.
Box 2 gives an explanation of the stages represented in Figure 1.
Box 2. Explanation of Stages of Adaptation as Illustrated in Figure 1
The main part of the toolkit can be used in its entirety, that is, as an aid to adapt information/data in all five domains, or can be used to adapt information/data in one or more domains. Hence, the user can use only the parts of the toolkit that are relevant to their needs.
The output from using the toolkit is adapted material from an HTA report that can be incorporated into a report for a local context. Further work by the user, for example, identifying local information and data, may be required before the local context of an HTA report can be completed.
DISCUSSION
This toolkit offers many potential benefits to the user and society. An aid to adapting HTA reports will help save time and resource, and increase output of HTA reports enabling agencies to evaluate more technologies than would otherwise be possible. It will also help avoid duplication of effort, where more than one report is written on the same technology at around the same time. The adaptation of HTA reports also helps facilitate the dissemination of knowledge and information.
A strength of the toolkit is the methodology used in its development (Reference Turner, Chase and Milne8). The process was iterative throughout, and drew on the combined knowledge and expertise of all the agencies involved, which were widely dispersed throughout Europe, each with their own experiences and knowledge of different systems.
The toolkit provides checklists and resources for just five HTA report domains: technology use and development, safety, efficacy and effectiveness, cost-effectiveness, and organizational aspects. It does not cover the adaptation of information on legal, social, or ethical aspects. This could be considered a weakness; however, this was a considered decision from within WP5. The Partners agreed that the toolkit should initially focus on those HTA report domains that are least context-dependent and therefore more amenable to adaptation.
Potentially the toolkit will be limited in its usefulness by the quality of the HTA report from which material is extracted for adaptation. Clearly, HTA reports with little information on how data was collected and analyzed would be more difficult to adapt than those with explicit details of the process undertaken. Furthermore, the toolkit has been developed for the adaptation of HTA reports that are systematic reviews. In some countries, reports of primary research studies are also considered to be HTA reports. The applicability testing of the toolkit, which has been previously described (Reference Turner, Chase and Milne8), also suggested that the instrument was more geared toward treatment reports (less so diagnostic and screening reports), which is another possible weakness.
This toolkit has been designed to support the adaptation of HTA reports from other contexts. It could equally be used by policy makers to help understand HTA reports written for other contexts. Clearly, the more information, data, and explanation provided in the HTA report for adaptation, the easier and more comprehensive the adaptation process. Hence, the toolkit would be best used as an aid to adapting more comprehensive HTA reports. However, it can also be used to adapt information and data from rapid reviews and mini HTAs, but the user will need to be aware of the purpose, and potential limitations, of the original report.
Work is currently under way to translate the toolkit into other languages which will make it usable by a wider community. It is currently available in English and Turkish.
The four main areas for future work on the toolkit are outlined as follows: (i) The toolkit is currently in a PDF version, and there is potential to develop an interactive, Web-based version; (ii) There is scope for further testing, review, and improvement both in EUnetHTA and beyond; (iii) The current version of the toolkit does not adequately support the adaptation of HTAs on diagnostic testing and screening technologies. There is scope to continue extending the toolkit to facilitate the adaptation of such HTA reports; and (iv) There is also work to be undertaken to incorporate closer integration with other EUnetHTA outputs, for example, the HTA Core Model (WP4) (Reference Lampe, Mäkelä and Velasco-Garrido6).
CONCLUSION
In conclusion, for those who wish to adapt HTA reports from other settings, this version of the toolkit produced by WP5 of the EUnetHTA Project offers a unique, time-saving, and up-to-date collection of resources.
CONTACT INFORMATION
Sheila Turner, BSc, PhD (s.turner@soton.ac.uk), Senior Researcher, Deborah L. Chase, BSc, PhD (dla1@soton.ac.uk), Senior Researcher, Ruairidh Milne, MB, BS, FFPH (rm2@soton.ac.uk), Senior Lecturer in Public Health, Andrew Cook, MBBS, BSc (Hons), MPH (andrewc@soton.ac.uk), Consultant in Public Health Medicine, Nicholas J. Hicks, MBBS, BA (Hons), FFPH (hicks@soton.ac.uk), Consultant in Public Health Medicine, National Coordinating Centre for Health Technology Assessment, University of Southampton, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK
Claire Rosten, BA, MA, PhD (c.e.rosten@brighton.ac.uk), Research Fellow, Research Design Service South East, University of Brighton, Mayfield House, Brighton, BN1 9PH, UK
Liz Payne, BA (Hons), PG Dip Lib, MCLIP (eapayne@f2s.com), Independent Information Specialist, Suzanne Coles, MBBS, BSc (Hons) (s.e.coles@soton.ac.uk), Specialist Registrar in Public Health Medicine, Eleanor Bell, MA (Hons) (ebell@soton.ac.uk), Project Manager, National Coordinating Centre for Health Technology Assessment, University of Southampton, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK
