Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2024-12-26T08:45:56.443Z Has data issue: false hasContentIssue false

Redefining Health Technology Assessment: A Comment on “The New Definition of Health Technology Assessment: A Milestone in International Collaboration”

Published online by Cambridge University Press:  05 July 2022

Anthony Culyer
Affiliation:
Centre for Health Economics, University of York, York, United Kingdom
Don Husereau*
Affiliation:
School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
*
*Author for correspondence: Don Husereau, E-mail: don.husereau@gmail.com
Rights & Permissions [Opens in a new window]

Abstract

A new definition of health technology assessment (HTA), developed by an International Joint Task Group claims to be a “milestone,” “an historic achievement,” and “a cornerstone reference”—claims that we think to be unjustified. We too favor clear definitions, especially when confusion abounds. However, the Task Group seems to have developed a definition without the help of usual conventions regarding definitions and, in our view, through an ill-described process. A definition ought to differentiate the entity defined from other entities. This one fails to do so. It states traits that are true of HTA (e.g., that is interdisciplinary) but HTA is not alone in this. There are other concerns: examples of HTA’s use are embodied in the definition, precluding other uses; the adjectives used, although generally true of HTA, are not differentiating features; and attributing to HTA specific purposes, thereby excluding other purposes. We have sympathy for these purposes but cannot consider them HTA’s only purposes or even, its main purpose. A newcomer to HTA, on reading this definition, will have no idea of HTA’s true potential. These numerous failings, we feel, send all the wrong signals, and could ultimately weaken, rather than strengthen perceptions of HTA’s legitimacy and objectivity. The production of a good definition remains, therefore, a work in progress.

Type
Article Commentary
Copyright
© The Author(s), 2022. Published by Cambridge University Press

An International Joint Task Group (“Task Group”) has recently published a new definition of health technology assessment (HTA), and has described their effort as “a milestone in international collaboration,” “an historic achievement,” and “a cornerstone reference” (Reference O’Rourke, Oortwijn and Schuller1). The Task Group members believe that a consensus achieved by the group brings the collective weight of the participating networks, societies, and organizations behind the new definition.

We agree that having a widely agreed definition of HTA is desirable but are concerned that the Task Group has crafted a poor one. It seems curmudgeonly to find fault, especially as we count all the authors of this enterprise amongst our friends, but that, alas, is what we do find. Let us explain.

The definition developed by the Task Group is accompanied by notes that define terms. It runs thus:

A multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its lifecycle. The purpose is to inform decision-making in order to promote an equitable, efficient, and high-quality health system. (Reference O’Rourke, Oortwijn and Schuller1)

The Task Group also provides a short history of a few past definitions as well as the previous INAHTA/HTAi definition. They provide no detailed analysis of the defects of existing definitions, from which it might have been possible to infer clear guidelines for the creation of a new definition. The chief characteristics of earlier definitions seem to have been these: shared common concepts (a promising characteristic but the specifics are unfortunately not identified), being too “technical,” too difficult to translate from English into some other languages, not memorable, and not aspirational.

If these claims are the reasons for creating a new definition, they are, on the face of things, ill defined themselves, and unlikely to yield useful insights into the definition’s redesign. We could not find any serious technical impediment in most of the existing definitions, in the form of, say, technical jargon from any of the common disciplines that furnish HTA with its analytical power. These disciplines have, of course, precise definitions of the clinical, statistical and economic concepts that they routinely deploy. These do not belong in the definition of HTA—although all HTA practitioners ought to possess a working knowledge of such entities as “pandemic,” “specificity,” “median,” and “opportunity cost.” However, neither did we find them used in past definitions.

A definition must surely define. To define is to differentiate. It must differentiate what is being defined from other similar entities. It must be comprehensible. These are basic rules of lexicography. However, as far as we can tell, the Task Group did not consider it necessary to become familiar with such principles. This failure probably lies at the root of our general disappointment with what has been produced.

The first specific comment we have is to note that the definition has two parts. The first sentence is, indeed, a definition. The second sentence is not a definition but a statement of the purposes for which HTA can be used. We consider first the definition itself.

Definitions and Adjectives

When defining things, adjectives are treacherous. Sometimes they are essential in stating a differentiating characteristic of the entity being defined (the definiendum). Adjectives can become key allies when lexicographers adopt a traditional model of definition that locates the item defined in a particular semantic category (the “genus expression,” “concept class,” or “superordinate concept”), in this case “assessment,” and then attempts to explain one or more of its differentiating features (the “differentiae”), in this case, related to assessment of investments in health technology. This approach is also outlined in the ISO 10241-5.25b and 704-6.31 standards, to which, presumably, the definition was intended to adhere (specific standards are not cited in the paper, so we assume those mentioned on the HTA glossary website—https://htaglossary.net/About-the-English-version—were used).

An example from economics illustrates the occasional importance of an adjective. The sentence “opportunity cost is the value attached to a forgone alternative use of a resource,” is a true statement but a wrong definition. Most resources have countless alternative uses, each of which may have a value. The definition as just stated is consequently empty of any useful meaning. To become a definition, in this case, we require an adjective: “opportunity cost is the highest value attached a forgone alternative use of a resource.”

In the Task Group’s proposed definition, we have two problematic adjectives: “multidisciplinary” and “explicit.” They are problematic because, although they are both usually true of HTA, they are not differentiating features. Many processes of assessment to inform decision making have these characteristics. For example, the classical process of haruspicy (the examination of the entrails of sacrificed animals in classical times) required multiple skills (and trained experts called haruspices). It had explicit rules and was widely used to forecast events (Reference Collins2). Astrology is another such process (Reference Lilly and Roell3). Both satisfy these elements of the definition but would scarcely, we conjecture, satisfy the authors as a definition of HTA. Brainstorming among experts on the effectiveness of a technology or taking the majority view from a national poll are similarly multidisciplinary and explicit, but neither is what is usually meant by HTA. Both may, of course, be used as tools in HTA, but may not be regarded highly in terms of reliability (Reference Schünemann, Zhang and Oxman4).

Removing these redundant adjectives (“multidisciplinary” and “explicit”) leaves us with this:

A process that uses methods to determine the value of a health technology at different points in its lifecycle. The purpose is to inform decision-making in order to promote an equitable, efficient, and high-quality health system.

Definitions and Purposes

This exposes a further problem: the definition includes specific purposes: “to determine value…” and “to promote an equitable, efficient, and high-quality health system.” These are not glosses on the definition. They are embodied in the definition itself. Consequently, any use of HTA for a purpose other than these must be something else than HTA. The Task Group did not (presumably) intend this.

Consider the everyday object, a spade. The OED definition is:

A tool for digging, paring, or cutting ground, turf, etc., now usually consisting of a flattish rectangular iron blade socketed on a wooden handle which has a grip or cross-piece at the upper end, the whole being adapted for grasping with both hands while the blade is pressed into the ground with the foot. (5)

The definition does not include the creation of beautiful gardens or the cultivation of prize-winning leeks. Like inessential adjectives, inessential purposes should not be a part of a definition, because they will always create an arbitrary limit on the applicability of the definiendum. Their value lies elsewhere, possibly in promoting good methodological practice and possibly in offering helpful examples of the uses of HTA. They have no role, however, in its definition.

HTA can, without doubt, be used to determine the relative value of a health technology at different points in a life cycle. It can also be used to assess overall value across the entire life cycle (Reference Sculpher, Drummond and Buxton6). It can also be usefully employed in addressing many other issues of value. Life-cycle estimates, equity, efficiency, and quality are examples of the varying purposes for which HTA may be useful. However, these are not offered as examples. They are part of the definition. Consequently, any purpose for conducting HTA that does not include all four purposes must be something other than HTA. Equity, which is one of three considerations that must be promoted by HTA according to the definition, has been, as a matter of fact, largely ignored in historic HTA production (Reference Panteli, Kreis and Busse7). We suspect the same for quality of care (Reference Fulop, Allen, Clarke and Black8).

The further problem is not just one of over-inclusion, but also of needless exclusion of other possible purposes. Consequently, studies designed to inform decision makers about how stigma has intersected with colonialism, racism, and migration in the context of tuberculosis policy and care (9), or on the need for reorganized supply chains in rural sub-Sahara Africa (10), or on the investment needed to develop HTA capacity in Southeast Asia (Reference Sharma, Teerawattananon, Dabak, Isaranuwatchai, Pearce and Pilasant11), or on the impact of a technology on the health ministry’s budget (Reference Jang, Simoens and Kwon12), could not count as HTA since their purposes may not have anything to do with life cycles, equity, efficiency, or quality. Instead, the studies may have been commissioned to help managers assess the level of cultural sensitivity required by their infectious disease programs, or their future workload for supplies, or to give universities advance warning to create new training programs, or to give plausible answers to the political opposition’s clamor for more/less public expenditure, or to provide evidence that health expenditures were properly controlled. (The possibilities are countless.)

Like adjectives, examples can be dangerous. They can be misleading, in much the same way that mistaking “i.e.” (id est: that is) with “e.g.” (exempli gratia: for example) is misleading. In fact, the potential uses of HTA are very numerous and its methods are ill served by arbitrarily restricting their scope and numbers. Moreover, like any tool, the underlying approaches to HTA can serve bad purposes as well as good ones. Spades can slice human heads and have done so (13). Cost-effectiveness analysis can be applied to eugenic instruments of oppression (Reference Price and Darity14), the effective organization of torture for suspected enemies of the state (Reference Yakovlev15), and to the creation and dissemination of fake news (Reference Kshetri and Voas16). The idea that HTA is inherently good, which the Task Group seems to espouse, is not so. It is a tool and, like all tools, can be put to both good and bad purposes.

Removing these ambiguities from the definition leaves us with

A process that uses methods.

Definitions and Differentiae

This reduced definition exposes the most fundamental weakness of the proposed definition: its tantalizing vagueness as to the nature of the process and of its methods. Unlike the authors, we think that stating any essential characteristics of the HTA process and its methods ought to form the meaningful content of the definition. Instead of the vagueness of “process” and “methods” we think the definition ought to contain explicit reference to the kinds of structure and interaction between participants in the process that are regarded as truly essential if a process is to be labeled “HTA.” These are the differentiae. As we have written, the traditional role of a definition is to explain what makes the entity under consideration (in this case, HTA) different from other processes, that is, “the characteristics that distinguish the concept from other members of the class” (17).

Some of what makes HTA different is reflected in the methods of its essential core disciplines, which we take to be biostatistics, economics and epidemiology (in alphabetical order); these are disciplines that make the interpretation of available data less tricky for decision makers and that also provide informative ways of evaluating evidence (such as its reliability, relevance, and completeness) and of thinking about trade-offs (such as consequences that occur at different dates, costs and benefits, acceptability of risks, greater opportunities to enjoy a healthy lifetime, or more equal opportunities for the same).

Mentioning these disciplines would convey useful information as to why HTA is different from other formal or less formal procedures. It is also an opportunity to illustrate the difference between multi- and interdisciplinary working. The authors use both terms without distinguishing them. We think that a notable characteristic of good quality HTA is that although analysts may have backgrounds in any one of these three core disciplines, they are barely distinguishable from one another in their working relationships. Having mastered the vocabulary and many of the techniques of each core discipline, intellectual integration through interdisciplinarity is virtually complete. Each analyst becomes more than merely a statistician, economist, or epidemiologist.

This does not imply that HTA is deterministic or something that can be entirely left to interdisciplinary “experts.” Instead, the use of these and any other contextually relevant disciplines is offered as a useful way of thinking about policy choices: identifying what considerations are relevant, what aspects may be quantifiable, where the principal trade-offs lie, where the main uncertainties are and the impact various assumptions and their consequences may have on ultimate outcomes, and which matters remain judgmental and/or political.

There are, of course, many disciplines sometimes useful in HTA—bibliometrics, decision science, demography, ergonomics, ethics, ethnography, management studies, medical anthropology, political science, and many others—but which are not “essential” for an activity to be fairly categorized as HTA. Rather than a note describing “dimensions of value” that may be assessed, there might have been a note focusing on the actual subtypes of research that HTA might employ, each having a varying purpose and more or less generality, and the variety of procedures available to support the decision-making process. Examples include economic evaluation, mathematical modeling of diseases, indirect treatment comparisons, stated preference research, multiple-criteria decision analysis, horizon scanning, deliberative processes, and citizens’ juries, to mention but a few. All are ways of doing HTA, although not entirely without controversy.

Concluding Remarks

We recognize from the spirit (and title) of the paper that collaboration among so many entities itself was of value and may have been at least as important as re-defining HTA. However, the description of its process and outcome is disappointing.

Regarding the process, and despite an increasing advocacy for more transparent and principled approaches to conducting and reporting deliberative approaches in the HTA community (Reference Bond, Stiffell and Ollendorf18;Reference Culyer19), the reader is provided with some details but not informed about the selection of the representatives of the organizations, the frequency of the “several” meetings, the roles played by members, the nature of any deliberation between members, the extent of agreement and disagreement, and the methods by which differences of opinion were settled. The final product was jointly agreed, which may or may not mean that it was unanimously accepted by all. The reader cannot tell. The lack of complete reporting is troubling and seems contrary to the core values of a community of HTA practitioners that highlight the importance of transparency (i.e., “explicitly describing, and making publicly available, information on the deliberative process and the basis for a recommendation or decision” (18)) when making recommendations.

Regarding the outcome, we are no lexicographers (although one of us has compiled a dictionary (19) and the other, standards for reporting (Reference Husereau, Drummond, Petrou, Carswell, Moher and Greenberg20)). However, we do recognize some of the basic ideas that are commonplace in all definition making, but which have been overlooked by our authors. This neglect has unfortunately resulted in a new definition of HTA that is empty of substantive content, that muddles examples of possible uses with uses that are held to define HTA but which also needlessly narrows the range of possible applications; that specifies specific value-laden purposes, but needlessly narrows the range of such purposes while also excluding the many purposes, value laden or not, that had not occurred to the authors. A newcomer to HTA, on reading this definition will have no indication of the true breadth of possible applications of HTA methods or of the critically important analytical building blocks that HTA necessarily involves.

The result is a definition clearly prejudiced by a love for HTA’s potential. It is the same love for HTA that has inspired this commentary. We share many of the values revealed by the authors but reject the idea that they are what defines HTA. The authors provide convincing examples of the kind of analysis needed in this increasingly important territory of public policy. The production of a good definition of HTA remains, however, a work in progress.

Author Contributions

D.H. is the submitting and corresponding author. Both authors (A.C. and D.H.) conceived this paper and were involved in revising the article critically for important intellectual content. Both authors approved the final version of the article. D.H. is the guarantor of this work. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding Statement

The authors declare no funding support for writing the article. Funding for open access publication was provided by 9363980 Canada Inc. This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.

Conflicts of Interest

The authors declare no potential conflicts of interest.

References

O’Rourke, B, Oortwijn, W, Schuller, T, IJT Group (2020) The new definition of health technology assessment: a milestone in international collaboration. Int J Technol Assess Health Care 36, 187–90.Google ScholarPubMed
Collins, D (2008) Mapping the entrails: the practice of Greek hepatoscopy. Am J Philol 129, 319–45.CrossRefGoogle Scholar
Lilly, W, Roell, DR. Christian astrology: modestly treated of in three books… Bel Air, MD: Astrology Classics; 2004.Google Scholar
Schünemann, HJ, Zhang, Y, Oxman, AD (2019) Distinguishing opinion from evidence in guidelines. BMJ 366, l4606.CrossRefGoogle ScholarPubMed
spade, n.1. OED online [Internet]. Oxford: Oxford University Press. Available at: http://www.oed.com/view/Entry/185449. Accessed 2021.Google Scholar
Sculpher, M, Drummond, M, Buxton, M (1997) The iterative use of economic evaluation as part of the process of health technology assessment. J Health Serv Res Policy 2, 2630.CrossRefGoogle ScholarPubMed
Panteli, D, Kreis, J, Busse, R (2015) Considering equity in health technology assessment: An exploratory analysis of agency practices. Int J Technol Assess Health Care 31, 314–23.Google ScholarPubMed
Fulop, N, Allen, P, Clarke, A, Black, N (2003) From health technology assessment to research on the organisation and delivery of health services: Addressing the balance. Health Policy 63, 155–65.CrossRefGoogle ScholarPubMed
View of tuberculosis stigma and racism, colonialism, and migration: A rapid qualitative review | Canadian Journal of Health Technologies [Internet]. Available at: https://canjhealthtechnol.ca/index.php/cjht/article/view/rc1350/91. Accessed 2021.Google Scholar
Disorders F on N and NS, policy B on HS, health B on G, medicine I of Challenge: Ineffective supply chains [Internet]. Improving access to essential medicines for mental, neurological, and substance use disorders in sub-Saharan Africa: Workshop summary. Washington, DC: National Academies Press; 2014. Available at: https://www.ncbi.nlm.nih.gov/books/NBK241504/. Accessed 2021.Google Scholar
Sharma, M, Teerawattananon, Y, Dabak, SV, Isaranuwatchai, W, Pearce, F, Pilasant, S, et al. (2021) A landscape analysis of health technology assessment capacity in the Association of South-East Asian Nations region. Health Res Policy Syst 19, 19.CrossRefGoogle ScholarPubMed
Jang, M, Simoens, S, Kwon, T (2021) Budget impact analysis of the introduction of rituximab and trastuzumab intravenous biosimilars to EU-5 markets. BioDrugs 35, 89101.CrossRefGoogle ScholarPubMed
Brit SAS soldier “decapitated ISIS fighter with a SPADE after running out of bullets during six-hour battle” [Internet]. Available at: https://www.thescottishsun.co.uk/news/2060324/brit-sas-hero-decapitated-isis-fighter-with-a-spade-after-running-out-of-bullets-during-six-hour-battle/. Accessed 2021.Google Scholar
Price, GN, Darity, WA (2010) The economics of race and eugenic sterilization in North Carolina: 1958-1968. Econ Hum Biol 8, 261–72.CrossRefGoogle ScholarPubMed
Yakovlev, P (2011) The economics of torture [Internet]. The handbook on the political economy of war. Cheltenham: Edward Elgar Publishing. Available at: https://www.elgaronline.com/view/9781848442481.00014.xml. Accessed 2021.Google Scholar
Kshetri, N, Voas, J (2017) The economics of “Fake News.” IT Prof 19, 812.CrossRefGoogle Scholar
HtaGlossary.net | Excerpts from the ISO standards 10241 and 704 [Internet]. HtaGlossary.net. Available at: http://htaglossary.net/Excerpts-from-the-ISO-standards-10241-and-704. Accessed 2021.Google Scholar
Bond, K, Stiffell, R, Ollendorf, DA (2020) Principles for deliberative processes in health technology assessment. Int J Technol Assess Health Care 4, 18.Google Scholar
Culyer, AJ (2005) The dictionary of health economics. Chelthenham; Northampton, MA: Edward Elgar, 390 p.Google Scholar
Husereau, D, Drummond, M, Petrou, S, Carswell, C, Moher, D, Greenberg, D, et al. (2013) Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Int J Technol Assess Health Care 29, 117–22.CrossRefGoogle ScholarPubMed