The commentary by Culyer and Husereau (Reference Culyer and Husereau1) criticizes the development of the new definition of health technology assessment (HTA) with its accompanying notes, published in June of 2020 in the Journal (Reference O’Rourke, Oortwijn and Schuller2) and in Value in Health (Reference O’Rourke, Oortwijn and Schuller3), although they do not refer to the latter. The main arguments made by the commentators are that traditional conventions for developing definitions were not followed and the process to develop the definition was ill-described. As authors and participants in the process to develop the new definition, we do not support the arguments presented by the commentators. We provide the following information to respond to their critique, which centers on three main areas.
First, we feel they are basing their commentary on a narrow and outdated understanding of HTA methods, processes, and practices. For example, they describe the core disciplines of HTA as biostatistics, economics, and epidemiology. This thinking excludes all the other disciplines and contributors (including users of HTA) from the current and future practice of HTA. For example, Staniszewska and Söderholm Werkö recently showed that clinical and economic evidence are not enough for HTA, and that “for HTA to be complete, we need to consider all relevant aspects of the phenomena, including patient-based evidence” (Reference Staniszewska and Söderholm Werkö4). This also requires certain disciplines, such as psychology and ethics, and other skills or processes, such as coproduction. Furthermore, the commentators claim that “all HTA practitioners ought to possess a working knowledge of such entities as ‘pandemic’, ‘specificity’, ‘median’, and ‘opportunity cost’ is not substantiated.” We would like to refer to Mueller et al. (Reference Mueller, Gutierrez-Ibarluzea, Chiumente and Oortwijn5) who recently provided an overview of the core competencies necessary for HTA. They recommend HTA training covering all the required domains of HTA analysis as reflected in the new definition of HTA.
Second, we note that the commentators criticize the process of developing the new definition as “an ill-described process.” We disagree and reiterate some of the process issues that were articulated in our manuscripts announcing the new definition of HTA (Reference O’Rourke, Oortwijn and Schuller2;Reference O’Rourke, Oortwijn and Schuller3). The new definition with important clarifying information provided in four accompanying notes is not “a work in progress” as suggested by the commentators—it was deliberately developed as an aspirational definition using a structured, transparent, and inclusive process, and it has been well-received by colleagues and stakeholders across the HTA ecosystem. Much has been written about the history of HTA and there has been considerable effort to evaluate, improve, and harmonize the science, methods, and practice of HTA; however, there had never been a global consensus on the definition of HTA. With this as the background, leadership from HTAi and INAHTA asked relevant networks if there was a need to develop a new, internationally accepted definition. As their responses were affirmative, INAHTA and HTAi created a joint international Task Group, coled by INAHTA and HTAi. The goal of the Task Group was to develop an internationally accepted new definition of HTA that incorporated the central concepts of HTA in language that would be easily understood by anyone across different linguistic backgrounds and that was more memorable and aspirational than existing definitions. Membership on the Task Group included representatives appointed by the leadership of all relevant HTA networks and societies: INAHTA, HTAi, EUnetHTA, HTAsiaLink, RedETSA, the HTA Glossary Committee, ISPOR, while the WHO participated as an observer. The Task Group identified a set of guiding principles for developing the new definition and compiled a set of core concepts to include in the definition. They also designed an extensive, iterative consultation plan where draft versions of the definition were reviewed by the Boards of the organizations represented in the international joint Task Group, and an open consultation process to seek input from the broader HTA community. The Task Group relied heavily on input from the member representing the HTA Glossary Committee to ensure we followed generally accepted principles of lexicography, policies and procedures from the HTA Glossary Committee, and that we were congruent with ISO standards. The HTA Glossary Committee, while consisting of experts in HTA, also has representation from the Canadian Translation Bureau who ensures that definitions comply with the ISO standards required for a multilingual glossary. While the HTA definition is lengthier than some other terms in the Glossary, the additional explanation enabled semantic differentiation from other types of process (“Process” is the anchor word). Details on methods and types of disciplines were specifically not included in the definition because the methods and disciplines contributing to HTA vary across health systems and health technologies. A key characteristic for definitional purposes was that those methods are transparent to the decision maker. HTA’s purpose in informing decision making and its multidisciplinarity were also considered defining characteristics of HTA.
The open consultation provided an opportunity for all stakeholders such as patients, clinicians, industry representatives, and academics to review and comment on the new definition. The guiding principles and the process for drafting the definition were communicated to the consultees. Feedback from the open consultation, including a few comments on lexicography similar to the arguments presented by Culyer and Husereau, was carefully assessed by the Task Group and many of the comments were incorporated into the new definition. For further explanation on each component of the new definition, readers are encouraged to read the discussion in our original manuscripts (Reference O’Rourke, Oortwijn and Schuller2;Reference O’Rourke, Oortwijn and Schuller3).
Following the development of various versions of the definition, as well as an analysis of 172 responses to the open consultation, the final version of the new definition was approved by the Boards/leadership of all participating Task Group member organizations and networks. The new definition was also formally endorsed by the leadership of the EuroScan International Network.
Third, the new definition has been embraced by the HTA community. Since its publication, the new definition has been increasingly used as a reference, such as by Mukherjee (Reference Mukherjee6) on the relevance of the new definition in the case of COVID-19 and beyond; by Pollard et al. (Reference Pollard, Dunne, Costa and Regier7) to show that it reflects the need for continuous evidence generation informing both clinical and reimbursement decisions in precision oncology; and several papers (co) authored by Culyer as well (Reference Teerawattananon, Painter and Dabak8;Reference Culyer9). Indeed, the manuscripts announcing the new definition of HTA have been extensively cited. A recent Google search identified ninety-eight citations for the manuscript in IJTAHC (Reference O’Rourke, Oortwijn and Schuller2) and ten times for the ViH publication (Reference O’Rourke, Oortwijn and Schuller3). Altmetrics mentions sixty-six citations for the IJTAHC manuscript, and indicates it to be “in the top 5 percent of all research outputs scored by Altmetric”: https://cambridge.altmetric.com/details/81966065.
Even though the commentators claim “A newcomer to HTA, on reading this definition will have no indication of the true breadth of possible applications of HTA methods or of the critically important analytical building blocks that HTA necessarily involves,” we can prove otherwise. The definition is being taught in educational programs. For example, it is being taught in the Introduction to HTA course, which is part of the HTA track of the 2-year Masters Biomedical Sciences program at Radboud University. The new definition of HTA has clearly laid out the path for integrating empirical analysis and normative inquiry in HTA as taught in the e-learning course VALIDATE (VAlues In Doing Assessments of healthcare Technologies) and applied in internships at HTA organizations (validatehta.eu; (10)). As explained by one of the students: “I regard the VALIDATE course as a very valuable and challenging one. I was aware of the classic HTA approaches; however, I did not expect that value-related aspects like ethical and social aspects would be of such a great importance in HTAs. This course made me think on a whole new level, by offering examples that show you how important it is to take values into account from the start of the inquiry. I would highly recommend this course to other students” (https://validatehta.eu/students-about-validate/). As such, the new generation of HTA practitioners is indeed receiving an indication of the true breadth of possible applications of HTA methods and of the critically important analytical building blocks that HTA necessarily involves.
In conclusion, we referred to the global collaboration involved in developing the new definition of HTA as a milestone in international collaboration—and we stand by this claim. Developing an internationally accepted definition through a consensus building process was a new achievement and therefore a historic milestone. The new definition and the accompanying notes have been widely accepted and employed by HTA practitioners and users around the world. This is not a work in progress—it is the new definition of HTA.
Members of the International Joint Task Group
Brian O’Rourke, Wija Oortwijn, Tara Schuller, Tracy Merlin, Gert Jan van der Wilt, Karen Macpherson, Michelle Mujoomdar, Jasmine Pwu, Jeonghoon Ahn, Inaki Imaz-Iglesia, Juan Pablo Chalco-Orrego, Alicia Framarin, Veronica Gallegos, Finn Boerlum Kristensen, Kelly Lenahan, Adriana Velazquez-Berumen, Sue Hill, and Sarah Garner.
