The International Journal of Technology Assessment in Health Care (IJTAHC) is part of the so-called Cambridge Core, the “home of academic content” (1). As such, not surprisingly, it is judged by standard scientific standards such as the Journal’s Impact Factor (IF). Although on the rise, there is still significant room for improvement (2). In this Editorial, I will discuss the relevance of the IF to the Journal and offer some suggestions as to how it might be given new impetus. I will conclude by briefly discussing some challenges associated with assessing impact in the context of health technology assessment (HTA).
What Does a High IF Mean?
So what does it mean when a journal has a high IF? Technically speaking, it means that on average, there is a large (say, more than eight) number of citations to papers that appeared in the journal, within a time frame of 2 years after its publication. In other words, there is a relatively large group of people (let's call them scientists) who, when writing about their own research, have reason to refer to papers that were published in this journal. What might those reasons be? Let us assume that the original publication reported results of a primary study of the clinical and cost-effectiveness of a specific healthcare technology. There may be several reasons to refer to this publication:
-
- Confirm: “We have done a similar study, and we found similar results”;
-
- Extend: “We found similar results, but we have reason to believe that they may be even extended to other indications, types of outcomes, time frames, jurisdictions, etc.”
-
- Incorporate: “We aimed to synthesize the available evidence, and we have incorporated the results of this study in our systematic review, modeling study, or meta-analysis.”
-
- Dispute: “We also investigated the clinical and cost-effectiveness of this healthcare technology, but were unable to replicate earlier findings.”
Of course, there may be still other reasons to refer to previous publications, for instance when authors do not wish to expand on certain issues (e.g., “chronic diseases are on the rise in countries with developing economies”) or methods (e.g., “costing of indirect nonhealthcare costs was conducted as described by . . .”). In general, then, the main reason for referring to the work of others is to support the plausibility of a specific interpretation of the evidence, to put research into perspective and to augment its relevance: “we are not the only ones who are investigating this subject, and our findings contribute to a better understanding, provide a novel perspective or a novel interpretation of the available evidence”.
The HTA Community
Generally speaking, HTA aims to support policy makers in reconciling multiple and conflicting objectives such as enabling healthcare professionals to provide compassionate care to patients, using a community's resources efficiently, and promoting equitable and timely access to high-quality care (Reference Banta and Luce3). This involves trade-offs, and these trade-offs should ideally be made in a transparent and accountable way. Hopefully, policy makers will find intimations in the IJTAHC that help them in achieving those objectives. If and when they do, however, this will usually not result in a reference to the relevant study and fail to contribute to the IF of the journal. In other words, the HTA community is quite heterogeneous, including a wide variety of researchers, policy makers, policy advisors, healthcare professionals, and representatives of patients and industry.
This is a unique asset, but it poses a problem to the Journal, too. Not all of those parties are equally engaged in conducting research and publishing their results. In addition, among those who are, only part of them work in academia, and publishing their results in academic journals may not be a top priority, if a priority at all. Their top priority is to reach out to policy makers, and journals may in fact refuse to publish findings that have been previously communicated through policy reports. Apart from these specific problems, there is, of course, the well-known and, in my view, valid criticism that has been leveled at IF-mania generally (Reference Casadevall and Fang4). These arguments could be taken to mean that the IF is an inappropriate metric to express the merits of our Journal.
I think, however, that such a conclusion would be premature. A high IF, in my view, testifies of the existence of a vibrant community, that uses the Journal as an important means to achieve its main objective. As voiced in its strategic plan, HTAi serves as the “global discussion platform for HTA thought leaders, researchers, agencies, policy makers, industry, academia, health service providers and patients” (5). For this heterogeneous community, the objective is to explore how healthcare technologies enable us to realize certain values (e.g., relief of suffering, restoring functioning, protecting human dignity, and promoting equity), and to discover and improve ways of conducting this type of inquiry. More than now, the Journal could act as an important forum where the results of such inquiries are presented and methods are critically discussed. In what follows I will suggest one possible way of achieving this.
How the Challenges of Healthcare Technology Present Opportunities for the Journal
The current development of healthcare technology is unprecedented, both in terms of pace and volume. Notably the combined use of genomics, robotics, information technology, and nanotechnology (“GRIN”) offers opportunities that were unthought-of only a decade ago. How can HTA be relevant to policy makers, in making up their minds as to how these technological opportunities might be used to realize community values? As a concrete example, take the developments in the area of retinal implants (Reference Greenemeier6).
Currently, this technology is at a stage where it would be appropriate to define what policy makers would need to know to decide in a reasonable and accountable way. Arguably, this would include information about safety, clinical effectiveness for possible indications, and short-term and long-term budget impact. Similarly, policy makers ought to demand for information on opportunity costs. Resources that are being spent on the development and use of this technology could be spent in other ways, producing other types of good, benefiting other groups in the community.
Also, policy makers may want to know how retinal implants affect the capability (Reference Sen, Fukuda-Parr, Shiva Kumar and Sen7) of people with degenerative retinal disease, and what other options exist to protect those people's capabilities. Scientifically and ethically, it would be interesting to know how technology can be used to interface with the human brain, possibly opening up entirely new opportunities for human enhancement. In other words, rather than simply waiting for what evidence is emerging from the field, now would be the time to define what we, as a community, would want to know about this technology to allow for reasonable and accountable decision making (Reference Daniels8).
It stands to reason that multiple stakeholders would have to be involved in this process of defining such desiderata, each reasoning from their own perspective. I am referring here, of course, to a process of scoping: defining relevant questions for research, options that need to be taken into account, and strategies that are likely to produce useful knowledge (Reference Brereton, Ingleton and Gardiner9). The Journal could invite submissions where the results of such scoping exercises are being presented. Would that increase the Journal's IF? Possibly. Future researchers who are reporting about specific pieces of the evidence could refer to this study, pointing out where and how their data enter into this wider framework. Researchers who are trying to synthesize the available evidence could refer to the study, indicating how comprehensive the current evidence base is, and highlighting any gaps.
In the short run, various stakeholders could respond to the list of desiderata, arguing that it is too demanding, or not demanding enough. In other words, this is how a vibrant community could operate, exchanging thoughts and ideas, using their journal as an important forum. Of course, the Journal’s policy in improving access to this wide audience through Open Access is critically important, too.
IF and Impact
However, even if successful, we still need to be aware that this might not fully capture the impact of HTA, generally. It might not reflect in what way and to what extent HTA helps policy makers in making up their minds about the opportunities that novel healthcare technologies offer to realize values that are constitutive of our societies. HTA might be more attentive to this issue, and the Journal might encourage submissions from researchers who have addressed this issue empirically. Those who would take an interest in pursuing along those lines would be well advised, however, to bear in mind the idiosyncratic pitfalls of such type of inquiry. As pointed out by the late Carol Weiss from Harvard, knowledge generally “creeps” into policy deliberations, and policy decisions tend to “accrete”, that is, take shape gradually over time (Reference Weiss10). This helps to explain the lack of direct use of research and analysis in policy decisions. More reports from research in this area, duly informed by findings from knowledge use in policy making, would be a welcome adjunct to the Journal.
