Knowledge in health technology assessment: Who, what, how?

Abstract

Health systems are placing more and more emphasis on designing and delivering services that are focused on the patient, and there is a growing interest in patient aspects of health policy research and health technology assessment (HTA). Only a few HTA agencies use and invest in scientific methods to generate knowledge and evidence about the patient aspects of a given technology. This raises questions about how knowledge is produced in HTA reports and what kind of knowledge is considered relevant. This article uses a Danish HTA on patient education from 2009 as empirical material for a critical examination and discussion of knowledge and knowledge production about the patient aspects of HTA.

In Denmark as well as in other European countries, patient education has been on the political agenda for several years and has been widely adopted by patient associations, regions and municipalities as part of their health promotion initiatives. The political interest in patient education is related to the ageing of the population and the concomitant increase in chronic diseases and public health expenditure (25). Patient education includes so-called “patient-oriented preventive efforts,” the purpose of which is to improve patients' (i.e., in particular people suffering from chronic diseases) competence and ability to care for themselves. The intention is to involve the patients in the management and treatment of their disease and its side-effects and to encourage them to take individual responsibility for their well-being. The emphasis on designing and delivering services that focus on the patient has been accompanied by a growing interest in patient aspects and patient's involvement in health policy research and health technology assessments' (HTA) (4;5;7;8;10;12;13;18;19).

Internationally, HTA has been defined as “a multidisciplinary activity that systematically examines the technical performance, safety, clinical efficacy and effectiveness, cost, cost-effectiveness, organizational implications, social consequences, legal and ethical considerations of the applications of a health technology” (9). In Denmark, HTAs are usually framed around four key elements: the technology, the patient, the organization, and the economy. This framework influences the ways in which HTAs are conceptualised and the assessment procedures organised, and accordingly, how knowledge in HTAs is produced and presented. The patient aspects are mostly investigated and presented in a separate chapter of the HTA. Presenting patient aspects in a separate chapter allows a more direct and explicit focus on the patient's problems, needs, experiences, and preferences in relation to a given technology. In the concluding parts of the Danish HTA, the patient aspects often influence the development of recommendations that are based on for instance economic modelling and/or clinical effectiveness outcomes. In general, patients are not actively involved in the Danish HTA process. Their views and perspectives are typically presented through secondary and/or primary research. Furthermore, a representative from a patient organization often takes part in a reference group.

The interest in eliciting patient aspects within HTAs in both Denmark and internationally deals not only with “what works” for patients, but also with the need of understanding issues related to the illness in question, the need for intervention and with factors influencing the implementation of particular health technologies, their appropriateness and acceptability (4;5;7;8;10;12;13;18;19). However, this is not an easy task. A review based on fifty HTA reports identified from the websites of members of the International Network of Agencies for Health Technology Assessments (INATHA) showed that only a minority of the assessments had provided research questions to be addressed about organizational or patient aspects. The review also argued that the reader was left unclear whether, for example, the relevant issues about patient aspects had been included and whether they had been assessed using a relevant methodology (8).

The present article is a result of the two authors' involvement in a Danish HTA think-tank and Hansen's involvement in writing chapters for the Handbook of Methods of HTA (13) as a member of the subgroup of Health Technology Assessment International (HTAi) and her participation in different Danish HTAs addressing patient aspects (7). One recurrent topic in our discussions of HTA reports has been the modes of knowing they represented, and in particular how the knowledge about patient aspects had come into existence. The objective of this article is therefore to inspire a critical examination, reflection and discussion of knowledge and knowledge production in HTAs on patients.

METHODOLOGY

Our analysis and discussion are based on a theoretically informed reading of one Danish HTA report entitled “Patient education: a health technology assessment” (7) (336 pages in total). The report is framed around four key elements: the technology, the patient, the organization, and the economics. The declared purpose of the report is “to critically assess patient education” with the aim of identifying opportunities and barriers for the future organisation of patient education in Denmark (7), building on the assumption that patient education is something we need in Denmark in the future. Patient education is, however, a broad concept covering diverse activities not always described in detail (25). The need for patient education is taken for granted and not questioned, even though the results show that the users of patient education are mostly women and people with higher education, who are independent and already have ample resources (7).

In the report on patient education, the technology element focuses primarily on the effects of patient education on people with Type 2 diabetes and chronic obstructive pulmonary disease (COPD) and the mechanisms of effect for patient programmes. The patient element explores who uses group-based patient education, user-based needs, and the benefits obtained. The organizational element assesses the current organization of patient education in the municipalities and regions, and identifies opportunities and barriers for future patient education. The economic element assesses the costs and benefits of patient education. The report states that it “provides scientific guidance for decision makers by broadly describing the evidence on patient education in Denmark and thereby contributing to a solid basis for determining the future use of patient education in the efforts to prevent chronic diseases and to improve the health of the people who have them” (7, p. 22). The report thus positions itself as providing “scientific guidance” for policy making. This positioning is in line with the “mission” of HTAi, which is: “to support and promote the development, communication, understanding and use of health technology assessment (HTA) around the world, as a scientifically based and multidisciplinary means of informing decision making regarding the introduction of effective innovations and the efficient use of resources in health care” (16, our italics). We use the report as an exemplar for exploring the dimensions of knowledge production in HTAs. We argue that, in presenting a form of authorised knowledge, the report encapsulates a particular form of knowledge that is the result of a specific process of knowledge production. An important aim of this article is thus to call attention to this process and to its implications for what can be known.

Analytical Perspectives

Our examination and discussion is theory driven. We draw on exponents of the anthropology of knowledge, which, stated briefly, takes an interest in different ways of knowing and in the social and cultural processes through which different forms of knowledge are generated (1;14;15;23;24). The Danish anthropologist Andreas Roepstorff proposes that we explore knowledge and knowledge production through three questions, addressing respectively the who, the what, and the how (24). The who prompts us to ask whose knowledge is being addressed, and what their interests and world views are. The what explores the differing criteria of validity in the traditions of knowledge and the kinds of knowledge that are produced by these criteria. In other words, the what investigates what counts as knowledge. Finally, the how examines how knowledge is constituted and produced. In the following, we present the results of our examination of the who, the what and the how in relation to the chosen report (7).

RESULTS

Whose Knowledge

DACEHTA initiated the report about patient education. The center is situated as a unit within the National Board of Health, who is also the contracting authority and the publisher of the report. As is the case with all national HTAs in Denmark, the “ownership” is announced clearly in the colophon to the report (7). HTAs are also always produced in collaboration with a multidisciplinary project group. The members of the project group generate the data, make the systematic reviews of existing literature and write the chapters in the report. However, the names of the researchers involved are not present either on the front page or in the colophon, but are listed inside the report, without any indication of their contribution. DACEHTA directs the assessment and carries out the final editing after the reports have been submitted to an external reference group and an external peer-review group.

Four researchers were engaged in the exploration of “the patient” element (two anthropologists, a philosopher, and a public health researcher). A reference group was invited to comment on a draft of the report and to join one meeting. Two of seventeen of the reference group came from patient organizations. The other members came from the Danish Regions, Danish medical and nursing societies, representatives from the Danish regions and municipalities, and the Ministry of Health and Prevention among others. This way of organizing the HTA work does not differ from the way other Danish HTAs are organized (6;21). However, it is interesting that the researchers and their individual contributions and responsibilities fade into the background. Contrary to research dissemination in general, they are kept opaque to the reader. Normally, such opaqueness conflicts with the principles of good scholarship. In this case, however, the idea seems to be that the mere list of names provides some of the scientific authority claimed by the report. Therefore, it is not readily detectable whose knowledge the report represents.

Health policy makers and administrative decision makers are described as the main target group for the report. Another target group is people who are somehow involved in patient education such as healthcare professionals, people with chronic diseases and researchers (7). Thus, the target group that is going to benefit from the knowledge represented in the HTA is rather broad including people with very different backgrounds and interests. Furthermore, as is often the case for Danish HTAs, the report is lengthy (almost 190 pages, 130 pages of appendixes, and a summary of 16 pages). This means that it will take considerable time and effort to read the report critically.

What Knowledge?

Another important question is what counts as knowledge? To answer this, we explored the HTA questions that the report seeks to answer about patient aspects and examined the ways in which the literature (the data) used in the assessment was sought and reviewed. The report asks three questions: “Who uses patient education programs?,” “What benefits do the participants experience in relation to patient education?,” and “Which needs of people with a chronic disease are significant for the future organization of patient education?” (7, pp. 7–8, 48, 81). While the first question requires the identification and mapping of the users, the questions addressing patients' “experiences” and “needs” call for studies using a qualitative research design. It is worth noting that the report only asks for “benefits,” the assumption apparently being that patient education cannot be of disadvantage to the patients. Also, whereas the report defines the concept of patient education (7, pp. 45–46), there is no clarification of the term patient. The HTA questions—like research questions in general—are decisive for knowledge production because they determine what to look for (and accordingly what to look past) and how. In addition, they carry with them assumptions and meanings that it is important to scrutinize critically. Also, an analysis of the meanings of “patients” and “needs” could provide a greater insight into the particular modes of knowing the patient; this is, however, beyond the scope of the present article.

The generation of data for patient aspects in this report was carried out through a “systematic” literature review using six databases and covering 121 studies. This decision of using secondary data instead of primary data is not discussed in the report. The search procedures, the chosen databases, the evaluations of the quality of the qualitative literature and of the evidence, and the characteristics of the excluded studies are all described in the appendix (7, p. 283ff). However, we found it particularly interesting that the report seems to be especially critical of the qualitative studies relating to the second HTA question about patient aspects (e.g., the experiences of the participants), both in a special section in the report (7, pp. 85–86) and in the appendices. The report gives no reason for the choice of the evaluation parameters or explains why the notions of “neutrality,” “representativity” (ibid.), “evidence” are mobilized in just this case. These terms originate from a positivist epistemological tradition and are not unambiguous when it comes to evaluating qualitative studies (22). For instance, the report states that, “One could have wished for research with a higher degree of neutrality. . .”; “there may easily occur biases with regard to the representativity of the participants. . .”, and “. . . the studies do not make us any wiser regarding the long-term effects of patient education” (7, p. 85, our translation from Danish). Furthermore, in using words such as “one” or “us,” it is not clear who the “sender” of the criticism is. We acknowledge that this is a common way of writing especially when analyzing and conveying quantitative data. However, this may be a way of signaling an objective and detached stance valued within the positivist tradition that has influenced health sciences.

The move from the HTA questions to the search criteria or the reason for the choice of particular search categories and words and their combination are not set out explicitly in the report. However, these search procedures and the exclusions do have implications for the literature that was uncovered and for what counts as knowledge. First, by limiting the assessment of patient education to two diseases (Type 2 diabetes and COPD), it becomes difficult to generate knowledge about patient education in more general terms. Second, the choice of data bases, search words, categories (e.g., “expert patients,” “chronic disease management,” and “qualitative methods”) leaves out “grey literature,” “berry-picking” (2, p. 21), books, book chapters and PhD dissertations that could be relevant (e.g., 11;17). Some of the reasons for exclusion are given in the appendix. Studies that do not address the “effect” of patient education are excluded. Also literature in, for instance, German and Spanish is excluded.

Third, words, terms, and categories are not “neutral,” but are always loaded with meaning. For example, how can the researchers of the patient education report be sure that the meaning of a search term as “expert patient” or the word “need” in the third research question is the same in different articles? Therefore, it should be asked whether the search words selected actually capture the literature that could have answered the HTA questions.

How? Constituting and Producing Knowledge

Questions of whose knowledge and what knowledge are obviously integral to the question of how knowledge is constituted and produced. However, compared with scholarly research in general, this HTA report does not explicate the premises for its knowledge production. Therefore, the answers to the how-question to a certain extent are rendered obscure. As one example, the HTA does not include a specific section on theory or analytical perspectives. However, the structuring of the report into the four key elements mentioned above can be seen as constituting a tacit theoretical approach because it is crucial for how knowledge is organized and presented.

While in relation to patient aspects the HTA report does to some extent describe its methods and procedures of exclusion and inclusion and its key concepts explicitly, it also takes certain notions and concepts for granted. As we have shown, the idea of patient education as a “benefit” or a “need” is never really questioned. The term patient is also taken for granted. But, who and what is the patient (and in that connection “expert patient”)? We have no space here to deal with this in depth, but we want to call attention to such tacit features of knowledge production in HTAs. In addition, we want to point out that words, concepts, and categories always carry with them meanings that may have a constituting impact on the knowledge product, and must, therefore, be critically scrutinized and clarified. Another crucial aspect of the how, is the relational dimensions of knowledge production. The Danish anthropologist Kirsten Hastrup states that knowledge is relational not only in the sense that it “attaches itself to relations among people,” but also because it emerges in a “dialogical field” (14, p. 456). It is impossible to infer this dialogical field (the discussions and negotiations between the different actors, and the forms of power that formed part of the HTA work) from the actual knowledge product (the HTA report) (ibid.). That would certainly require a qualitative enquiry into the process of knowledge production. But the micro-dynamics of the dialogical field and the ways in which the engaging researchers do their work and negotiate the knowledge outcome constitutes an important dimension of the social process of knowledge production. For instance, the researchers involved in HTA come from different disciplines and research areas and are, therefore, differently positioned with regard to university degrees, titles and academic interests and the forms of power—“symbolic capital” (3)—that come with a certain social and academic position.

Finally, it seems that the notion of technology in HTA deserves some attention. The report seems to operate with the view that “technology” (in this case patient education) “comes first” and thereafter is implemented to affect the patients. This view leaves out the possibility of producing knowledge about the dialogical relationship between technology and patients and the ways in which patients may also shape and co-produce the technology.

DISCUSSION

In her reflections on knowledge and evidence in anthropology (14;15), Hastrup makes several important points relevant not only for our analysis and discussion of this particular HTA, but also for primary research and knowledge production in more general terms. She states that “knowledge must be organized information” (14, p. 456). This suggests that knowledge is both reductive and selective: “It is reductive because it renders empirical complexity and messiness in clear, but, therefore also more limited, propositions about the world. It is selective, because for it to be knowledge it has to disregard some information” (ibid., our italics). A critical examination and discussion of any knowledge product—and in our case, specifically the patient aspects in the HTA report—must, therefore, look into and reflect on the reducing and selecting activities embedded in the knowledge production. This requires, as we have demonstrated, an attempt to examine the who, what, and how in the knowledge producing process. In this case, the HTA questions determine and delimit what is looked for, and the procedures for data generation are selective, because they exclude or disregard certain forms of data and information. It is also influenced by negotiations between different stakeholders and actors and their interests, agendas, decisions and compromises. In addition, assumptions (more or less hidden) and choice of words, categories, and concepts all influence knowledge production in HTAs. In scholarly research (especially social studies) reduction is normally theoretically informed, and there is a demand for transparency with regard to selection procedures and what is disregarded. Although the report positions itself as providing “scientific guidance” for policy making, it does not fully live up to such demands. This said, it is of course important to consider the extent to which the present report (as HTA reports more generally) are also influenced by socio-structural and economic conditions. Time, money, and limited institutional resources, therefore, place limits on what can be known.

Finally, Hastrup points out that knowledge is no simple “object” because “it bears all the marks of its institution,” which also includes a “particular style of reasoning” (14, p. 457). HTA reports (in Denmark and internationally) are no exception, as we have indicated above. The National Board is the supreme health-care authority in Denmark, assisting and advising ministries, other authorities and citizens (20). The Boards' publications therefore need to appear reliable and sound. As we have indicated, the comprehensive use of researchers, experts, and reviewers invest the reports with authority and credibility.

CONCLUDING REMARKS

It is important that HTAs nationally and internationally continue the integration of patient aspects in their reports. In this article, we have critically examined the Danish HTA report about patient education, but we also acknowledge the valuable contribution of both this and other Danish reports (13). They are unique in including thorough chapters about patient aspects that are based on secondary or primary research. However, we suggest that HTA organizations, researchers, and others (including patients) involved in producing HTA reports, begin to reflect on and discuss more explicitly and critically the process of knowledge production and the implications for the choices they make.

CONTACT INFORMATION

Tine Tjørnhøj-Thomsen, MA (extended master), PhD (tine.t.thomsen@post.tele.dk), Professor, National Institute of Public Health, University of Southern Denmark, Øster Farimagsgade 5A, DK-1353 Copenhagen K, Denmark

Helle Ploug Hansen, MA (extended master), PhD (rphansen@health.sdu.dk), Professor in Humanistic Rehabilitation Research, Institute of Public Health Research Unit; Health, Man & Society and National Research Centre for Cancer Rehabilitation, University of Southern Denmark, J.B. Winsløws Vej 9, DK-5000 Odense C, Denmark

CONFLICT OF INTEREST

Both authors report they have no potential conflicts of interest.