Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-26T16:47:31.300Z Has data issue: false hasContentIssue false
  • English
  • Français

Communicating clinical trial outcomes: Effects of presentation methodon physicians’ evaluations of new treatments

Published online by Cambridge University Press:  01 January 2023

Francesco Marcatto ,
Jonathan J. Rolison  and
Donatella Ferrante
Francesco Marcatto*
Affiliation:
Department of Life Sciences, University of Trieste, Via Weiss 21, 34138, Trieste, Italy
Jonathan J. Rolison
Affiliation:
Department of Psychology, Queen’s University Belfast, UK
Donatella Ferrante
Affiliation:
Department of Life Sciences, University of Trieste, Italy
*
*Email: fmarcatto@units.it.
Rights & Permissions [Opens in a new window]

Abstract

Physicians expect a treatment to be more effective when its clinical outcomes aredescribed as relative rather than as absolute risk reductions. We examinedwhether effects of presentation method (relative vs. absolute risk reduction)remain when physicians are provided the baseline risk information, a vital pieceof statistical information omitted in previous studies. Using a between-subjectsdesign, ninety five physicians were presented the risk reduction associated witha fictitious treatment for hypertension either as an absolute risk reduction oras a relative risk reduction, with or without including baseline riskinformation. Physicians reported that the treatment would be more effective andthat they would be more willing to prescribe it when its risk reduction waspresented to them in relative rather than in absolute terms. The relative riskreduction was perceived as more effective than absolute risk reduction even whenthe baseline risk information was explicitly reported. We recommend thatinformation about absolute risk reduction be made available to physicians in thereporting of clinical outcomes. Moreover, health professionals should becognizant of the potential biasing effects of risk information presented inrelative risk terms.

Keywords

relative riskabsolute riskbaseline riskrisk communicationpresentation format
Type
Research Article
Information
Judgment and Decision Making , Volume 8 , Issue 1 , January 2013 , pp. 29 - 33
Creative Commons
Creative Common License - CCCreative Common License - BY
The authors license this article under the terms of the Creative Commons Attribution 3.0 License.
Copyright
Copyright © The Authors [2013] This is an Open Access article, distributed under the terms of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

1 Introduction

Most, if not all patients, consult with their physician regarding medical treatment, and the physician plays a major role in the treatment decisions of patients (Reference AuerbachAuerbach, 2001; Reference Degner and SloanDegner & Sloan, 1992). An important finding of recent years is that patients and the general public poorly interpret risk reductions associated with treatment options (see Reference Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz and WoloshinGigerenzer, Gaissmaier, Kurz-Milcke, Schwartz & Woloshin, 2007, for a review). With patients relying on the recommendations of their physicians, we must ask whether physicians understand the reports of risk reduction behind the treatment options they recommend.

Risk reductions associated with medical treatment are typically reported either in relative or absolute terms. Relative risk reduction presents risks associated with a treatment relative to some baseline (e.g., a control group). A risk reduction from 10% to 5% for individuals administered a treatment represents a relative risk reduction of 50%, compared to an overall reduction of 5% in absolute terms. Perhaps not surprisingly, patients and people in general with no formal training in statistics anticipate treatments to be more effective when risk reductions are presented to them in relative than in absolute terms (Reference Akl, Oxman, Herrin, Vist, Terrenato, Sperati, Costiniuk, Blank and SchünemannAkl et al., 2011; Reference Chao, Studts, Abell, Hadley, Roetzer and DineenChao et al., 2003; Reference CoveyCovey, 2007; Reference FeinsteinFeinstein, 1992; Reference Rolison, Hanoch and Miron-ShatzRolison, Hanoch & Miron-Shatz, 2012; Reference Wegwarth, Gaissmaier and GigerenzerWegwarth, Gaissmaier & Gigerenzer, 2010). Of perhaps greater concern, physicians—who typically receive instruction in statistics as part of their basic medical training Footnote 1—also succumb to the lure of relative risk reports, and are found to recommend treatment options more on basis of relative than of absolute risk reduction (Reference Akl, Oxman, Herrin, Vist, Terrenato, Sperati, Costiniuk, Blank and SchünemannAkl et al., 2011; Reference Bobbio, Demichelis and GiustettoBobbio, Demichelis & Giustetto, 1994; Reference Forrow, Taylor and ArnoldForrow, Taylor & Arnold, 1992; Reference Moxey, O’Connell, McGettigan and HenryMoxey, O’Connell, McGettigan & Henry, 2003; Reference Naylor, Chen and StraussNaylor, Chen & Strauss, 1992; Reference Nexøe, Gyrd-Hansen, Kragstrup, Kristiansen and NielsenNexøe, Gyrd-Hansen, Kragstrup, Kristiansen & Nielsen, 2002).

Since physicians, patients, and the general public alike are unduly influenced by relative risk information, it is surprising that relative risk reduction is the most commonly used method of delivering statements about the efficacy of clinical trials in media reports (Reference Moynihan, Bero, Ross-Degnan, Henry, Lee and WatkinsMoynihan et al., 2000), and in the technical reports on which physicians base their recommendations (Reference Bucher, Weinbacher and GyrBucher, Weinbacher & Gyr, 1994; Reference Fahey, Griffiths and PetersFahey, Griffiths & Peters, 1995). In a survey of media reports, Reference Moynihan, Bero, Ross-Degnan, Henry, Lee and WatkinsMoynihan et al. (2000) found that 83% of reports described risk reductions only in relative terms, compared to just 17% that provided the absolute risks.

Some authors suggest either discouraging the use of relative risk information, or combining it with absolute risk information (Reference Laupacis, Sackett and RobertsLaupacis, Sackett & Roberts, 1988; Reference Rolison, Hanoch and Miron-ShatzRolison et al., 2012; see also Reference Schwartz and MeslinSchwartz & Meslin, 2008), while others advocate policies for improving health literacy in the general public (Reference Schwartz, Woloshin, Black and WelchSchwartz, Woloshin, Black & Welch, 1997). Despite the merits of these approaches, there still remains the possibility that physicians’ misunderstanding of relative risk reduction has been overestimated due to the default assumption, common to the previous studies, that physicians are aware of the baseline risks. Without the baseline risk, the efficacy of a treatment cannot be assessed, and any recommendations based on it may be misled. Depending on whether a baseline risk is 10%, 5%, or 1%, a relative risk reduction of 50% could mean that 50, 25, or 5 lives in every 1,000 will be saved by a treatment. Can we assume that physicians always know the baseline risks, especially when the baseline risks refer to domains outside the physician’s everyday practice? And when physicians know the baseline risk, is their understanding of the relative risk reduction accurate? Literature has showed that patients and the general public overestimate the relative risk reduction even when the baseline risk is explicitly provided (Reference Chao, Studts, Abell, Hadley, Roetzer and DineenChao et al., 2003; Reference Wolf and SchorlingWolf & Schorling, 2000). To date, however, no previous study has examined the influence of baseline risk information on physicians’ interpretations of relative risk information.

In the current article, we present physicians with reports of a fictitious clinical trial of a new treatment for hypertension—a chronic but treatable cardiovascular disease (Reference Ong, Cheung, Man, Lau and LamOng, Cheung, Man, Lau & Man, 2007). The report provides the risk reduction of the treatment either in terms of its absolute risk reduction, or in terms of its relative risk reduction with or without including the baseline risk information. Our aims were (a) to examine whether physicians’ willingness to prescribe a treatment and their ratings of its effectiveness would be influenced by the method in which risk reduction is provided (relative or absolute risk reduction), and (b) whether explicitly reporting the baseline risk in combination with information about relative risk reduction would influence the physicians’ willingness to prescribe and ratings of effectiveness.

2 Method

2.1 Participants

A total of 95 physicians (55 men, 40 women; mean age=37.27, SD=10.64), of which 41 were interns and 54 resident physicians at a public hospital in Northeast Italy were contacted at their workplace. All physicians agreed to participate in the study on a voluntary basis.

2.2 Materials and Procedure

Physicians were presented with one of three vignettes, adapted from a previous study by Forrow et al. (1992). The vignette reported the results of a clinical trial of a new drug treatment for hypertension. Physicians were randomly assigned to receive the vignette reporting the clinical trial outcome either as an absolute risk reduction, a relative risk reduction, or a relative risk reduction that included the baseline risk. The vignette that referred to an absolute risk reduction followed (translated from Italian; N=32):

A randomized controlled study of over 6,000 men with a moderate form of hypertension (DBP 90-104) conducted to evaluate the effectiveness of an experimental drug treatment, showed that the experimental drug treatment reduced the overall mortality rate over the 5 year period from 7.8% in the “routine treatment” control group to 6.3%. The 1.5% reduction in total mortality observed over the 5 year period was statistically significant.

The vignette that referred to a relative risk reduction followed (N=30):

A randomized controlled study of over 6,000 men with a moderate form of hypertension (DBP 90-104) conducted to evaluate the effectiveness of an experimental drug treatment, showed that the group treated with the experimental drug registered a reduction in the overall mortality rate over the 5 year period by 20.3% as compared with rate in the “routine treatment” control group. This reduction was statistically significant.

The vignette that referred to a relative risk reduction including the baseline risk followed (N=33):

A randomized controlled study of over 6,000 men with a moderate form of hypertension (DBP 90-104) conducted to evaluate the effectiveness of an experimental drug treatment, showed that the group treated with the experimental drug registered a reduction in the overall mortality rate over the 5 year period by 20.3% as compared with rate in the “routine treatment” control group (the “routine treatment” control group’s overall mortality rate over the 5 year period was equal to 7.8%). This reduction was statistically significant.

Regarding the vignette, physicians were asked to rate the effectiveness of the drug treatment on a 7 point scale ranging from −3 (“Not effective at all”) to 3 (“Very effective”), and their willingness to prescribe the drug treatment on the same 7 point scale from −3 (“Certainly not inclined to prescribe”) to 3 (“Certainly inclined to prescribe”).

3 Results

Physicians’ mean group ratings of treatment effectiveness and their willingness to prescribe are provided in Figure 1. A one way independent analysis of variance (ANOVA), including presentation method (absolute risk reduction, relative risk reduction, relative risk reduction including baseline risk) as an independent factor, and including physicians’ experience (residency vs. internship) and gender as covariates, was conducted separately on physicians’ ratings of effectiveness and their willingness to prescribe. The analysis revealed significant effects of presentation method both on physicians’ ratings of effectiveness (without covariates: F (2, 92)=13.53, MS e=0.71, p<.001, η2=.23; with covariates: F (2, 90)=13.02, MS e=0.71, p<.001, η2=.22) and their willingness to prescribe (without covariates: F (2, 90)=9.56, MS e=0.66, p<.001, η2=.17; with covariates: F (2, 90)=9.17, MS e=0.65, p<.001, η2=.16). There were no significant effects of gender (ratings of effectiveness, F (1, 90)=0.08, MS e=0.71, p=.773; willingness to prescribe, F (1, 90)=0.53, MS e=0.65, p=.470) or experience (ratings of effectiveness, F (1, 90)=1.78, MS e=0.71, p=.185; willingness to prescribe, F (1, 90)=2.86, MS e=0.71, p=.094).

Figure 1: Physicians’ ratings of the effectiveness of a new treatment (Panel A) and their willingness to prescribe it (Panel B) by method of risk presentation. Bars represent 1 standard error below and above mean ratings. ARR = Absolute Risk Reduction, RRR = Relative Risk Reduction.

Follow-up t-test comparisons (with Bonferroni correction) revealed significant mean group differences on ratings of effectiveness between the absolute risk reduction method and the relative risk reduction method, regardless of whether (p<.001) or not (p<.001) the baseline risk was included, such that physicians rated the treatment as more effective when information was provided as a relative (including baseline risk, M=2.24, SD=0.75; not including baseline risk, M=2.07, SD=0.64) rather than as an absolute risk reduction (M=1.22; SD=1.10). Regarding physicians’ willingness to prescribe, compared to the absolute risk reduction method (M=1.19, SD=0.90), physicians were significantly more willing to prescribe the treatment when presented as a relative risk reduction (including baseline risk, M=2.06, SD=0.70, p<.001; not including baseline risk, M=1.73, SD=0.83, p=.029). Including the baseline risk in combination with the relative risk reduction did not affect physicians’ ratings of effectiveness (p=.213) nor their willingness to prescribe (p=.205) compared with when the baseline risk was not included with the relative risk reduction. Footnote 2

4 Discussion

The physician plays an important role in the treatment decisions of patients (Reference AuerbachAuerbach, 2001; Reference Degner and SloanDegner & Sloan, 1992). It is thus crucial that we examine how physicians evaluate reports of new clinical trial outcomes, and how their evaluations influence their willingness to prescribe new treatments to patients and their expectations about the effectiveness of new treatments.

In this study, we provided physicians the risk reduction associated with a new (but fictitious) treatment. Physicians were presented the treatment’s benefits as an absolute risk reduction, as a relative risk reduction, or as a relative risk reduction including the baseline risk. In this respect our study is unique. In fact previous studies (e.g., Forrow et al., 1992), as well as media reports of clinical outcomes (Reference Moynihan, Bero, Ross-Degnan, Henry, Lee and WatkinsMoynihan et al., 2000) typically do not report the baseline risk, a crucial aspect of relative risk information. If the baseline risk is unknown, the relative risk reduction cannot be interpreted. We reasoned that participants of previous studies could have overestimated the effectiveness of treatments because they did not know the baseline risk information.

As expected, physicians claimed that the new treatment would be more effective and that they would be more willing to prescribe it when its risk reduction was presented to them in relative rather than in absolute terms. To our surprise, however, physicians appeared entirely unaffected by the addition of the baseline risk in combination with the relative risk reduction.

Why do physicians’ not take into account baseline risk information when evaluating clinical outcomes? We do not know whether physicians are not aware that the baseline risk information is relevant, or whether they do not have the statistical or numerical skills to integrate baseline risk information with relative risk reduction (see, for example, Reference Barbey and SlomanBarbey & Sloman, 2007; Reference Kahneman and TverskyKahneman & Tversky, 1973). It is clear, however, that physicians, like others, are unduly influenced by the “big” numbers that relative risk reductions present. Indeed, this may also explain the popular use of relative risk information in media reports of clinical outcomes (Reference Moynihan, Bero, Ross-Degnan, Henry, Lee and WatkinsMoynihan et al., 2000).

It would seem that, even when all relevant statistical information is made available, the format in which risks are reported to physicians influences strongly their expectations of a treatment’s effectiveness and their willingness to prescribe a treatment to patients. Our current results echo the strong recommendations made by others that information about absolute risk reduction should be made available to physicians (as well as to the lay public) when information about relative risk reduction is used (Reference Laupacis, Sackett and RobertsLaupacis et al., 1988; Reference Rolison, Hanoch and Miron-ShatzRolison et al., 2012). The lure of relative risk information may be especially strong among the general public who typically do not have direct training in interpreting health risk statistics (Reference Galesic and Garcia-RetameroGalesic & Garcia-Retamero, 2010; Reference Schwartz, Woloshin, Black and WelchSchwartz et al., 1997). Although better than the general public, physicians also show difficulties with numerical skills necessary for interpreting health risk statistics (Reference Estrada, Barnes, Collins and ByrdEstrada, Barnes, Collins & Byrd, 1999). Thus, we suggest also that more extensive statistical and numerical training be made available to physicians.

In conclusion, based on our current results we strongly recommend that health professionals be cognizant of the potential biasing effects of risk information presented in relative risk terms. We urge that absolute risk information also be made available when clinical outcomes are reported to physicians.

Footnotes

We thank Fabio Del Missier and Yaniv Hanoch for helpful comments on anearlier version of this article and Manuela Slavec for her assistance withdata collection.

1 According to the WFME Global Standards for Quality Improvement, basic medical education curricula must incorporate behavioral and social sciences, including biostatistics and epidemiology, with the aim to enable effective communication and clinical decision making (World Federation for Medical Education, 2012).

2 A power analysis using G*Power 3 software (Reference Bucher, Weinbacher and GyrFaul, Erdfelder, Lang & Buchner, 2007) revealed that, in order to detect a medium effect size (d = .50) between two groups, with α =.05 and power =.95, each group required at least 27 participants. Thus, the sample sizes used in this study should have been adequate to detect a medium sized effect.

References

Akl, E. A., Oxman, A. D., Herrin, J., Vist, G. E., Terrenato, I., Sperati, F., Costiniuk, C., Blank, D., & Schünemann, H. (2011) Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database of Systematic Reviews, Issue 3, Art. No.: CD006776.CrossRefGoogle Scholar
Auerbach, S. M. (2001). Do patinets want control over their own health care? A review of measures, findings, and research issues. Journal of Health Psychology, 6, 191–203.CrossRefGoogle Scholar
Barbey, A. K., & Sloman, S. A. (2007). Base-rate respect: From ecological rationality to dual processes. Behavioral and Brain Sciences, 30, 241–254.CrossRefGoogle ScholarPubMed
Bobbio, M., Demichelis, B., & Giustetto, G. (1994). Completeness of reporting trial results: effect on physicians’ willingness to prescribe. The Lancet, 8907, 1209–1211.CrossRefGoogle Scholar
Bucher, H. C., Weinbacher, M., & Gyr, K. (1994). Influence of method of reporting study results on decision of physicians to prescribe drugs to lower cholesterol concentration. British Medical Journal, 309, 761–764.CrossRefGoogle ScholarPubMed
Chao, C., Studts, J. L., Abell, T., Hadley, T., Roetzer, L., Dineen, S., et al. (2003). Adjuvant chemotherapy for breast cancer: How presentation of recurrence risk influences decision-making. Journal of Clinical Oncology, 21, 4299-4305.CrossRefGoogle ScholarPubMed
Covey, J. (2007). A meta-analysis of the effects of presenting treatment benefits in different formats. Medical Decision Making, 27, 638–654.CrossRefGoogle ScholarPubMed
Degner, L. F., & Sloan, J. A. (1992). Decision making during serious illness: What role do patients really want to play? Journal of Clinical Epidemiology, 45, 941–950.CrossRefGoogle ScholarPubMed
Estrada, C., Barnes, V., Collins, C., & Byrd, J.C. (1999). Health literacy and numeracy. Journal of the American Medical Association, 282, 527.CrossRefGoogle ScholarPubMed
Fahey, T., Griffiths, S., & Peters, T. J. (1995). Evidence based purchasing: Understanding results of clinical trials and systematic reviews. British Medical Journal, 311, 1056–1059.CrossRefGoogle ScholarPubMed
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.CrossRefGoogle ScholarPubMed
Feinstein, A. R. (1992). Invidious comparisons and unmet clinical challenges. The American Journal of Medicine, 92, 117–120.CrossRefGoogle ScholarPubMed
Forrow, L., Taylor, W. C., & Arnold, R. M. (1992). Absolutely relative: How research results are summarized can affect treatment decisions. The American Journal of Medicine, 92, 121–124.CrossRefGoogle ScholarPubMed
Galesic, M., Garcia-Retamero, R. (2010). Statistical numeracy for health: A cross-culural comparison with probabilistic national samples. Archives of Internal Medicine, 170, 462–468.CrossRefGoogle Scholar
Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz, L. M., & Woloshin, S. (2007). Helping doctors and patients to make sense of health statistics. Psychological Science in the Public Interest, 8, 53–96.CrossRefGoogle ScholarPubMed
Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80, 237–251.CrossRefGoogle Scholar
Laupacis, A., Sackett, D. L., & Roberts, R. (1988). An assessment of clinically useful measures of the consequences of treatment. New England Jounral of Medicine, 318, 1728–1733.CrossRefGoogle ScholarPubMed
Moynihan, R., Bero, L., Ross-Degnan, D., Henry, D., Lee, K., Watkins, J., et al. (2000). Coverage by the news media of the benefits and risks of medications. New England Journal of Medicine, 342, 1645–1650.CrossRefGoogle ScholarPubMed
Moxey, A., O’Connell, D., McGettigan, P., & Henry, D. (2003). Describing treatment effects to patients. Journal of General Internal Medicine, 18, 948–959.CrossRefGoogle ScholarPubMed
Naylor, C. D., Chen, E., & Strauss, B. (1992). Measured Enthusiasm: Does the Method of Reporting Trial Results Alter Perceptions of Therapeutic Effectiveness? Annals of Internal Medicine, 117, 916–21.CrossRefGoogle ScholarPubMed
Nexøe, J., Gyrd-Hansen, D., Kragstrup, J., Kristiansen, I. S., & Nielsen, J. B. (2002). Danish GPs’ perception of disease risk and benefit of prevention. Family Practice, 19, 3–6.CrossRefGoogle ScholarPubMed
Ong, K. L., Cheung, B. M. Y., Man, Y. B., Lau, C. P., & Lam, K. S. L. (2007). Prevalence, awareness, treatment, and control of hypertension among United States adults 1999–2004. Hypertension, 49, 69–75.CrossRefGoogle ScholarPubMed
Rolison, J. J., Hanoch, Y., & Miron-Shatz, T. (2012). What do men understand about lifetime risk following genetic testing? The effect of context and numeracy. Health Psychology, 31, 530–533.CrossRefGoogle ScholarPubMed
Schwartz, L. M., Woloshin, S., Black, W. C., & Welch, H. G. (1997). The role of numeracy in understanding the benefit of screening mammography. Annals of Internal Medicine, 127, 966–972.CrossRefGoogle ScholarPubMed
Schwartz, P. H., & Meslin, E. M. (2008). The Ethics of Information: Absolute Risk Reduction and Patient Understanding of Screening. Journal of General Internal Medicine, 23, 867–70.CrossRefGoogle ScholarPubMed
Wegwarth, O., Gaissmaier, W., & Gigerenzer, G. (2010). Deceiving numbers: Survival rates and their impact on doctors’ risk communication. Medical Decision Making, 31, 386–394.CrossRefGoogle ScholarPubMed
Wolf, A. M. D., & Schorling, J. B. (2000). Does Informed Consent Alter Elderly Patients’ Preferences for Colorectal Cancer Screening? Journal of General Internal Medicine, 15, 24–30.CrossRefGoogle ScholarPubMed
World Federation for Medical Education. (2012) Basic Medical Education. WFME Global Standards for Quality Improvement. The 2012 Revision. Retrieved December 11, 2012, from the World Federation for Medical Education Web site: http://www.wfme.org/standards/bmeGoogle Scholar
Figure 0

Figure 1: Physicians’ ratings of the effectiveness of a new treatment (Panel A) and their willingness to prescribe it (Panel B) by method of risk presentation. Bars represent 1 standard error below and above mean ratings. ARR = Absolute Risk Reduction, RRR = Relative Risk Reduction.

Supplementary material: File

Marcatto et al. supplementary material

Marcatto et al. supplementary material
Download Marcatto et al. supplementary material(File)
File 1.4 KB