13.1 Introduction
This chapter explores the concept of risk-benefit analysis in health research regulation, as well as ethical and practical questions raised by identifying, quantifying, and weighing risks and benefits. It argues that the pursuit of objectivity in risk-benefit analysis is ultimately futile, as the very concepts of risk and benefit depend on attitudes and preferences about which reasonable people disagree. Building on the work of previous authors, the discussion draws on contemporary examples to show how entities reviewing proposed research can improve the process of risk-benefit assessment by incorporating diverse perspectives into their decision-making and engaging in a systematic analytical approach.
13.2 Identifying Risks
The term ‘risk’ refers to the possibility of experiencing a harm. The concept incorporates two different dimensions: (1) the magnitude or severity of the potential harm; and (2) the likelihood that this harm will occur. The significance of a risk depends on the interaction of these two considerations. Thus, a low chance of a serious harm, such as death, would be considered significant, as would a high chance of a lesser harm, such as temporary pain.
In the context of research, the assessment of risk focuses on the additional risks participants will experience as a result of participating in a study, which will often be less than the total level of risks to which participants are exposed. For example, a study might involve the administration of various standard-of-care procedures, such as biopsies or CT scans. If the participants would have received these same procedures even if they were not participating in the study, the risks of those interventions would not be taken into account in the risk-benefit analysis. As a result, it is possible that a study comparing two interventions that are routinely used in clinical practice could be considered low risk, even if the interventions themselves are associated with a significant potential for harm. This is the case with a significant proportion of research conducted in ‘learning health systems’, which seek to integrate research into the delivery of healthcare. Because many of the research activities in such systems involve the evaluation of interventions patients would be undergoing anyway, the risks of the research are often minimal, even when the risks of the interventions themselves may be high.Footnote 1
The risks associated with health-related research are not limited to potential physical injuries. For example, in some studies, participants may be asked to engage in discussions of emotionally sensitive topics, such as a history of previous trauma. Such discussions entail a risk of psychological distress. In other studies, a primary risk is the potential for unauthorised disclosure of sensitive personal information, such as information about criminal activity, or stigmatised conditions such as HIV, or mental disorders. If such disclosures occur, participants could suffer adverse social, legal, or economic consequences.
Research-related risks can extend beyond the individuals participating in a study. For example, studies of novel interventions for preventing or treating infectious diseases could affect the likelihood that participants will transmit the disease to third parties.Footnote 2 Similarly, studies in which psychiatric patients are taken off their medications could increase the risk that participants will engage in violent behaviour.Footnote 3 Third-party risks are an inherent feature of research on genetic characteristics, given that information about individuals’ genomes necessarily has implications for their blood relatives.Footnote 4 Thus, if a genetic study results in the discovery that a participant is genetically predisposed to a serious disease, other persons who did not consent to participate in the study might be confronted with distressing, and potentially stigmatising, information that they never wanted to know.
In some cases, third-party risks extend beyond individuals to broader social groups. As the Council for International Organizations of Medical Sciences (CIOMS) has recognised, research on particular racial or ethnic groups ‘could indicate – rightly or wrongly – that a group has a higher than average prevalence of alcoholism, mental illness or sexually transmitted disease, or that it is particularly susceptible to certain genetic disorders’,Footnote 5 thereby exposing the group to potential stigma or discrimination. One example was a study in which researchers took blood samples from members of the Havasupai tribe in an effort to identify a genetic link to type 2 diabetes. After the study was completed, the researchers used the blood samples for a variety of unrelated studies without the tribe members’ informed consent, including research related to schizophrenia, inbreeding and migration patterns. Tribe members claimed that the schizophrenia and inbreeding studies were stigmatising, and that they never would have agreed to participate in the migration research because it conflicted with the tribe’s origin story, which maintained that the tribe had originated in the Grand Canyon. The researcher institution reached a settlement with the tribe that included monetary compensation and a formal apology.Footnote 6
Despite the prevalence of third-party risks in research, most ethics codes and regulations do not mention risks to anyone other than research participants. This omission is striking given that some of these same sources explicitly state that benefits to non-participants should be factored into the risk-benefit analysis. A notable exception is the EU Clinical Trials Regulation, which states that the anticipated benefits of the study must be justified by ‘the foreseeable risks and inconveniences’,Footnote 7 without specifying that those risks and inconveniences must be experienced by the participants themselves.
In addition to omitting any reference to third-party risks, the US Federal Regulations on Research With Human Participants state that entities reviewing proposed research ‘should not consider possible long-range effects of applying knowledge gained in the research (e.g. the possible effects of the research on public policy) as among those research risks that fall within the purview of its responsibility’.Footnote 8 This provision is intended ‘to prevent scientifically valuable research from being stifled because of how sensitive or controversial findings might be used at a social level’.Footnote 9
13.3 Identifying Benefits
The primary potential benefit of research is the production of generalisable knowledge – i.e. knowledge that has relevance beyond the specific individuals participating in the study. For example, in a clinical trial of an investigational drug, data sufficient to establish the drug’s safety and efficacy would be a benefit of research. Data showing that an intervention is not safe or effective – or that it is inferior to the existing standard of care – would also count as a benefit of research, as such knowledge can protect future patients from potentially harmful and/or ineffective treatments they might otherwise undergo.
Whether a study has the potential to produce generalisable knowledge depends in part on how it is designed. The randomised controlled clinical trial (RCT) is often described as the ‘gold standard’ of research, as it includes methodological features designed to eliminate bias and control for potential confounding variables.Footnote 10 However, in some types of research, conducting an RCT may not be a realistic option. For example, if researchers want to understand the impact of different lifestyle factors on health, it might not be feasible to randomly assign participants to engage in different behaviours, particularly over a long period of time.Footnote 11 In addition, ethical considerations may sometimes preclude the use of RCTs. For example, researchers investigating the impact of smoking on health could not ethically conduct a study in which non-smokers are asked to take up smoking.Footnote 12 In these situations, alternative study designs may be used, such as cohort or case-control studies. These alternative designs can provide valuable scientific information, but the results may be prone to various biases, a factor that should be considered in assessing the potential benefits of the research.Footnote 13
A recent example of ethical challenges to RCTs arose during the Ebola outbreak of 2013–2016, when the international relief organisation Médicins Sans Frontières refused to participate in any RCTs of experimental Ebola treatments. The group argued that it would be unethical to withhold the experimental interventions from persons in a control group when ‘conventional care offers little benefit and mortality is extremely high’.Footnote 14 The difficulty with this argument was that, in the context of a rapidly evolving epidemic, the results of studies conducted without concurrent control groups would be difficult to interpret, meaning that an ineffective or even harmful intervention could erroneously be deemed effective. Some deviations from the ‘methodologically ideal approach’, such as the use of adaptive trial designs, could have been justified by the need ‘to accommodate the expectations of participants and to promote community trust’.Footnote 15 However, any alternative methodologies would need to offer a reasonable likelihood of producing scientifically valid information, or else it would not have been ethical to expose participants to any risk at all.
The potential benefit of scientific knowledge also depends on the size of a study, as studies with very small sample sizes may lack sufficient statistical power to produce reliable information. Some commentators maintain that underpowered studies lack any potential benefit, making them inherently unethical.Footnote 16 Others point out that small studies might be unavoidable in certain situations, such as research on rare diseases, and that their results can still be useful, particularly when they are aggregated using Bayesian techniques.Footnote 17
Often, choices about study design can require trade-offs between internal and external validity. While an RCT with tightly controlled inclusion and exclusion requirements is the most reliable way to establish whether an experimental intervention is causally linked to an observable result – thereby producing a high level of internal validity – if the study population does not reflect the diversity of patients in the real world, the results might have little relevance to clinical practice – thereby producing a low level of external validity.Footnote 18 In assessing the potential benefits of a study, decision-makers should take both of these considerations into account.
In addition to the potential benefit of generalisable knowledge, some research also offers potential benefits to the individuals participating in the study. Benefits to study participants can be divided into ‘direct’ and ‘indirect’ (or ‘collateral’) benefits.Footnote 19 Direct benefits refer to those that result directly from the interventions being studied, such as an improvement in symptoms that results from taking an investigational drug. In some studies, there is no realistic possibility that participants will directly benefit from the study interventions; this would be the case in a Phase I drug study involving healthy volunteers, where the purpose is simply to identify the highest dose humans can tolerate without serious side effects. Indirect benefits include those that result from ancillary features of the study, such as access to free health screenings, as well as the psychological benefits that some participants receive from engaging in altruistic activities. Study participants may also consider any payments or other remuneration they receive in exchange for their participation as a type of research-related benefit.
Most commentators take the position that only potential direct benefits to participants and potential contributions to generalisable knowledge should be factored into the risk-benefit analysis. The concern is that, otherwise, ‘simply increasing payment or adding more unrelated services could make the benefits outweigh even the riskiest research’.Footnote 20 Other commentators reject this position on the ground that it is not consistent with the ethical imperative to respect participants’ autonomy, and that it could preclude studies that would advance the interests of participants, investigators, and society.Footnote 21 The US Food and Drug Administration has stated that payments to participants should not be considered in the context of risk-benefit assessment,Footnote 22 but it has not taken a position on consideration of other indirect benefits, such as access to free health screenings.
13.4 Quantifying Risks and Benefits
Once the risks and benefits of a proposed study have been identified, the next step is to quantify them. Doing this is complicated by the fact that the significance of a particular risk or benefit is highly subjective. For example, a common risk in health-related research is the potential for unauthorised disclosure of participants’ medical records. This risk could be very troubling to individuals who place a high degree of value on personal privacy, but for persons who share intimate information freely, the risk of unauthorised disclosure might be a minor concern. In fact, in some studies, the same experience might be perceived by some participants as a harm and by others as a benefit. For example, in a study in which participants are asked to discuss prior traumatic experiences, some participants might experience psychological distress, while others might welcome the opportunity to process past experiences with a sympathetic listener.Footnote 23
In addition to differing attitudes about the potential outcomes of research, individuals differ in their perceptions about risk-taking itself. Many people are risk averse, meaning that they would prefer to forego a higher potential benefit if it enables them to reduce the potential for harm. Others are risk neutral, or even risk preferring. Similarly, individuals exhibit different levels of willingness to trade harmful outcomes for good ones.Footnote 24 For example, some people are willing to tolerate medical treatments with significant side effects, such as chemotherapy, because they place greater value on the potential therapeutic benefits. Others place greater weight on avoiding pain or discomfort and would be disinclined to accept high-risk interventions even when the potential benefits are substantial.
Another challenge in attempting to quantify risks and benefits is that the way that risks and benefits are perceived can be influenced by a variety of cognitive biases. For example, one study asked subjects to imagine that they had lung cancer and had to decide between surgery and radiation. One group was told that 68 per cent of surgical patients survived after one year, while a second group was told that 32 per cent of surgical patients died after one year. Even though the information being conveyed was identical, framing the information in terms of a risk of death increased the number of subjects who chose radiation from 18 per cent to 44 per cent.Footnote 25 Another common cognitive bias is the ‘availability heuristic’, which leads people to attach greater weight to information that is readily called to mind.Footnote 26 For example, if a well-known celebrity recently died after being implanted with a pacemaker, the risk of pacemaker-related deaths may be perceived as greater than it actually is.
Individuals’ perceptions of risks and benefits can also be influenced by their level of social trust, which has been defined as ‘the willingness to rely on those who have the responsibility for making decisions and taking actions related to the management of technology, the environment, medicine, or other realms of public health and safety’.Footnote 27 In particular, research suggests that, when individuals are considering the risks and benefits of new technologies, their level of social trust has ‘a positive influence on perceived benefits and a negative influence on perceived risks’.Footnote 28 This is not surprising: those who trust that decision-makers will act in their best interests are less likely to be fearful of changes, while those who lack such trust are more likely to be worried about the potential for harm (see Aitken and Cunningham-Burley, Chapter 11, in this volume).
Compounding these subjective variables is the fact that risk-benefit analysis typically takes place against a backdrop of scientific uncertainty. This is true for all risk-benefit assessments, but it is especially pronounced in research, as the very reason research is conducted is to fill an evidentiary gap. While evaluators can sometimes rely on prior research, including animal studies, to identify the potential harms and benefits of proposed studies, most health-related research takes place in highly controlled environments, over short periods of time. As a result, prior research results are unlikely to provide much information about rare safety risks, long-term dangers or harms and benefits that are limited to discrete population subgroups.
13.5 Weighing Risks and Benefits
Those responsible for reviewing proposed research must ultimately weigh the risks and benefits to determine whether the relationship between them is acceptable. This process is complicated by the fact that risks and benefits often cannot be measured on a uniform scale. First, ‘risks and benefits for subjects may affect different domains of health status’,Footnote 29 as when a risk of physical injury is incurred in an effort to achieve a potential psychological benefit. Second, ‘risks and benefits may affect different people’;Footnote 30 risks are typically borne by the participants in the research, but most of the benefits will be experienced by patients in the future.
Several approaches have been suggested for systematising the process of risk-benefit analysis in research. The first, and most influential, approach is known as ‘component analysis’. This approach calls on decision-makers to independently assess the risks and potential benefits of each intervention or procedure to be used in a study, distinguishing those that have the potential to provide direct benefits to participants (‘therapeutic’) from those that are administered solely for the purpose of developing generalisable knowledge (‘non-therapeutic’). For therapeutic interventions, there must be genuine uncertainty regarding the relative therapeutic benefits of the intervention as compared to those of the standard of care for treating the participants’ condition or disorder (a standard known as ‘clinical equipoise’Footnote 31). For non-therapeutic interventions, the risks must be minimised to the extent consistent with sound scientific design, and the remaining risks must be reasonable in relation to the knowledge that is expected to result. In addition, when a study involves a vulnerable population, such as children or adults who lack decision-making capacity, the risks posed by nontherapeutic procedures may not exceed a ‘minor increase above minimal risk’.Footnote 32
Component analysis has been influential, but it is not universally supported. Some critics maintain that the distinction between therapeutic and non-therapeutic procedures is inherently ambiguous, as ‘all interventions offer at least some very low chance of clinical benefit’.Footnote 33 Others argue that the approach’s reliance on clinical equipoise rests on the mistaken assumption that researchers have a duty to promote each participant’s medical best interests, which conflates the ethics of research with those of clinical care.Footnote 34
One alternative to component analysis is known as the ‘net risk test’, which is based on the principle that the fundamental ethical requirement of research is ‘to protect research participants from being exposed to excessive risks of harm for the benefit of others’.Footnote 35 The approach has four elements. First, for each procedure involved in a study, the risks to participants should be minimised and the potential clinical benefits to participants enhanced, to the extent doing so is consistent with the study’s scientific design. Second, instead of clinical equipoise, the approach requires that, ‘when compared to the available alternatives, a research procedure must not present an excessive increase in risk, or an excessive decrease in potential benefit, for the participant’.Footnote 36 Third, to the extent particular procedures involve greater risks than benefits, those net risks ‘must be justified by the expected knowledge gained from using that procedure in the study’.Footnote 37 Finally, the cumulative net risks of all of the procedures in a study must not be excessive.Footnote 38
Both component analysis and the net risk test can add structure to the process of risk-benefit analysis by focusing attention on the risks and potential benefits of each intervention in a study. The advantage of this approach is that it reduces the likelihood that potential direct benefits from one intervention will be used as a justification for exposing participants to risks from unrelated interventions that offer no direct benefits. However, neither approach eliminates the need for subjective determinations. Under component analysis, the principle of clinical equipoise offers a benchmark for judging the risks and potential benefits of therapeutic procedures, but for non-therapeutic procedures, the only guidance offered is that the risks must be ‘reasonable’ in relation to the knowledge expected to result. The net benefit test dispenses with clinical equipoise entirely, instead relying on a general principle of avoiding ‘excessive risk’. Whether a particular mix of risks and potential benefits is ‘reasonable’ or ‘excessive’ is ultimately left to the judgment of those charged with reviewing the study.
Most regulations and ethics codes provide little guidance on the process of weighing the risks and potential benefits of research. The primary exception is the CIOMS guidelines, which adopts what it describes as a ‘middle ground’ between component analysis and the net risk test. In most respects, the CIOMS approach reflects component analysis, including its reliance on clinical equipoise as a standard for evaluating interventions or procedures that have the potential to provide direct benefits to participants. However, the guidelines also call for a judgment that ‘the aggregate risks of all research interventions or procedures … must be considered appropriate in light of the potential individual benefits to participants and the scientific social value of the research’,Footnote 39 a requirement that mirrors the final step of the net risk test.
Neither component analysis nor the net risk test explicitly sets an upper limit on permissible risk, at least in studies involving competent adults. However, one of the developers of component analysis has stated that ‘the notion of excessive net risks, and the underlying ethical principle of non-exploitation, clearly impose a cap on the risks that individuals are allowed to assume for the benefit of others’.Footnote 40 The notion of an upper limit on risk also appears in several ethical guidelines. For example, the CIOMS guidelines state that ‘some risks cannot be justified, even when the research has great social and scientific value and adults who are capable of giving informed consent would give their voluntary, informed consent to participate in the study’.Footnote 41 Similarly, the European Commission has suggested that certain ‘threats to human dignity and shared values’ should never be traded against the potential scientific benefits of research, including ‘commonly shared values like privacy or free movement … certain perceptions of the integrity of a person (e.g. cloning, technological modifications) … [and] widely shared view[s] of our place in the world (e.g. inhumane treatment of animals or threat to biodiversity)’.Footnote 42
In light of the inherent ambiguities involved in weighing the risks and benefits of research, the results of risk-benefit assessments can be heavily influenced by the type of decision-making process used. The next section looks at these procedural issues more closely.
13.6 Procedural Issues in Risk-Benefit Analysis
In most health-related research, the process of risk-benefit assessment is undertaken by interdisciplinary bodies known as research ethics committees (RECs), research ethics boards (REBs), or institutional review boards (IRBs). These committees make judgments based on predictions about the preferences and attitudes of typical research participants, which do not necessarily reflect how the actual participants would react to particular risk-benefit trade-offs.Footnote 43 In addition, because few committees rely on formal methods of risk-benefit analysis, decisions are likely to be influenced by individual members’ personal attitudes and cognitive biases.Footnote 44 For this reason, it is not surprising that different committees’ assessments of the risks and potential benefits of identical situations exhibit widespread variation.Footnote 45
Some commentators have proposed techniques to promote greater consistency in risk-benefit assessments. For example, it has been suggested that committees issue written assessments that could be entered into searchable databases.Footnote 46 Others have called on committees to engage in a formal process of ‘evidence-based research ethics review’, in which judgments about risks and potential benefits would be informed by a systematic retrieval and critical appraisal of the best available evidence.Footnote 47
Outside of research ethics, a variety of techniques have been developed to systematise the process of risk-benefit analysis. For example, several quantitative approaches to risk-benefit assessment exist, such as the Quality-Adjusted Time Without Symptoms and Toxicity (Q-TWIST) test, which ‘compares therapies in terms of achieved survival and quality-of-life outcomes’,Footnote 48 or the ‘standard gamble’, which assigns utility values to health outcomes based on individuals’ stated choice between hypothetical health risks.Footnote 49 Committees reviewing proposed studies can draw on these quantitative analyses when relevant ones exist.
In some cases, formal consultation with the community from which participants will be drawn can be an important component of assessing risks and benefits. For example, in the study of Havasupai tribe members discussed above, prior consultation with the community could have alerted researchers to the fact that research on migration patterns was threatening to the tribe’s cultural beliefs. In cancer research, consultation with patient advocacy groups may help identify concerns about potential adverse effects that might not have been sufficiently considered by the researchers.Footnote 50 Further lessons might be learned from the the analysis by Chuong and O'Doherty, Chapter 12, this volume.
13.7 Conclusion
Risk-benefit analysis is a critical part of the process of evaluating the ethical acceptability of health-related research. The primary challenge in risk-benefit assessment arises from the fact that perceptions about risks and potential benefits are inherently subjective. Those charged with assessing the ethical acceptability of research should make efforts to incorporate as many different perspectives into the process as possible, to ensure that their decisions do not simply reflect their own idiosyncratic views.
Open access