Hostname: page-component-78c5997874-dh8gc Total loading time: 0 Render date: 2024-11-13T12:19:04.721Z Has data issue: false hasContentIssue false

CJEM Debate Series: #ChoosingWisely – The Choosing Wisely campaign will not impact physician behaviour and choices

Published online by Cambridge University Press:  14 March 2018

Paul Atkinson*
Affiliation:
Department of Emergency Medicine, Dalhousie University, Saint John Regional Hospital, Saint John, NB
Eddy Lang
Affiliation:
Department of Emergency Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB
Meaghan Mackenzie
Affiliation:
Cumming School of Medicine, University of Calgary, Calgary, AB
Rashi Hirandani
Affiliation:
School of Medicine, University of Ottawa, Ottawa, ON
Rebecca Lys
Affiliation:
School of Medicine, Queen’s University, Kingston, ON
Megan Laupacis
Affiliation:
School of Medicine, Queen’s University, Kingston, ON
Heather Murray
Affiliation:
Department of Emergency Medicine, Queen’s University, Kingston, ON Department of Public Health Sciences, Queen’s University, Kingston, ON.
*
Correspondence to: Dr. Paul Atkinson, Emergency Medicine, Dalhousie University, Saint John Regional Hospital, 400 University Ave., Saint John, NB E2L 4L4; Email: paul.atkinson@dal.ca

Abstract

Type
Commentary
Copyright
Copyright © Canadian Association of Emergency Physicians 2018 

INTRODUCTION

Paul Atkinson

This series of editorials will provide CJEM readers with the opportunity to hear differing perspectives on topics pertinent to the practice of emergency medicine. The debaters have allocated opposing arguments on topics where there is some controversy or perhaps scientific equipoise.

We continue the series with the topic of Choosing Wisely (CW), a high profile campaign backed by several medical associations, including the Canadian Association of Emergency Physicians (CAEP).

Is the CW campaign simply a re-branding of common sense and the currently followed best practice in consumer organization style? Or is it a novel evidence-based program that will both save money and improve the quality of care that we deliver? Is it an attempt to remove clinical judgment from patient care, replacing it with simplistic rules that do not recognize variability in populations? Or is the aim of the campaign to empower patients to facilitate improved communication with care providers and ultimately better choices? Does the campaign seek to cut cost as its bottom line? Or will CW help prompt clinicians to challenge old habits and follow best evidence?

Dr. Eddy Lang and his team make the argument that the CW campaign is well intentioned; however, due to weak methodology, it will not impact physician behaviour and choices as they argue for the motion with Dr. Heather Murray and her team responding to back the campaign, arguing that it is okay not to order all of those tests, and that ultimately CW will change practice.

Join the CJEM debate: Follow @CJEMOnline or go to www.facebook.com/CJEMonline to participate in the online poll and to see the results!

THE CHOOSING WISELY CAMPAIGN WILL NOT IMPACT PHYSICIAN BEHAVIOUR AND CHOICES

For: Eddy Lang (@eddylang1), Meaghan Mackenzie, and Rashi Hirandani

Well-intentioned but weak methods: Top eight Choosing Wisely fails

The CW campaign was first piloted 8 years ago as a collaboration between the American Board of Internal Medicine and Consumer Reports, the latter being the organization that advises on what brand of dishwasher that we may wish to purchase. The genesis of this initially U.S.-based movement recognized that, in much the same way, some dishwasher makers were selling sub-standard product to unwitting consumers, and patients were getting the raw end of the stick as a result of many of the tests, treatments, and procedures that were being prescribed for them by their physicians. Hence, the basic premise of CW is to “promote conversations about unnecessary tests, treatments, and procedures, and to help physicians and patients make smart and effective choices to ensure high-quality care.” 1

Overuse of low-value interventions is an incontrovertible reality in modern medicine and is the source of tremendous waste and harm inextricably tied to what we do on every shift.Reference Sahota and Lang 2 The problem needs urgent fixing but promoting conversations between patients and emergency physicians has failed to deliver results using the current approach. The basic modus operandi of CW is to have medical specialty societies select one or more top five lists of tests, treatments, or procedures that physicians should reconsider using in caring for their patients. The rapid dissemination and uptake of CW across Canada and the world have been nothing short of remarkable and are well described by my debating partner. Sadly, these efforts have been largely ineffective in demonstrating any meaningful change. Without a significant re-engineering of CW implementation strategies, I fear that this well-intentioned movement will continue to disappoint.

Three years of post-CW claimed data from 25 million Anthem-affiliated Blue Cross and Blue Shield Insurance recipients in the United States were analysed for improvements in seven widely publicized low-value services from 2010 to 2013.Reference Rosenberg, Agiro and Gottlieb 3 Only two services (computed tomography [CT] for headaches and cardiac imaging for low-risk chest pain) demonstrated declines. However, both amounted to anemic improvements of less than 2%. Two services actually increased over the course of the study: nonsteroidal anti-inflammatory drug (NSAID) use for selective conditions and screening of very young women for the human papilloma virus. The remaining measures were unchanged. We need only look at our own backyard to that for some of Canadian emergency medicine’s most robust CW recommendations, we have yet to see improvement. 4

The reasons for CW’s disappointing performance in Canadian emergency medicine context are both myriad and complex, but, from our vantage point, they might be considered in decreasing order of importance from the following list.

  1. 1) Our patients are often affected by distress from multiple sources, including pain, anxiety, and prolonged wait times. If their expectations include tests and treatments, it might be unrealistic to expect that this CW-promoted conversation related to low-value tests and treatments can proceed effectively in the chaotic and access-blocked environments in which we work. The CanMEDS framework suggests that we be patient advocates (advocate role) and judicious stewards of resources (leader role). 5 Physicians likely recognize that, not infrequently, these objectives are at loggerheads.

  2. 2) Most Canadian emergency physicians work in a fee for service model of care that, in some instances, can serve as a perverse incentive to not choose that wisely, for example, more tests can justify more lucrative billing codes. Many CW recommendations take time to weave into a shared decision-making conversation that may detract from other on-shift priorities. Of note, these crucial efforts and important conversations are generally not remunerated through physician services contracts. Fee-for-service payment models incentivize emergency department (ED) throughput but generally do not recognize the investment in time and effort required to engage patients and families in a CW conversation.

  3. 3) Although the methods underlying CW recommendations are generally robust, they have been formulated largely by academic physician leaders from what are often perceived as ivory tower institutions. As such, they risk being viewed as top-down directives with little or no ownership by front-line providers having been developed without specific efforts to involve and engage local physicians. Perhaps there is nothing that sacred about CW recommendations, and physician groups should choose the kinds of low value that they would like to see reduced in their own settings, establishing their own targets.

  4. 4) Although the original intent of CW was largely oriented towards curbing unnecessary and low-value services so as to spare the patient from downstream harms associated with the risk of false positive findings and other iatrogenic harms, the cost-saving aspect of CW often seems to take centre-stage. For better or worse, costing data are underemphasized at all levels of medical education, making the notion of saving money for a universal and socialized health care system somewhat alien to many medical professionals in this country. In other words, with no skin in the game and limited knowledge of costs, emergency physicians may feel hard-pressed to save money for the system.

  5. 5) Achieving system-level improvement that reflects CW priorities requires robust and reliable measurement, preferably through an electronic health record. The required health system analytics are generally unavailable, and few EDs have sufficiently well-developed quality improvement programs that can meet suggested requirements for reporting performance data back to providers.Reference Brehaut, Colquhoun and Eva 6 Most physicians will agree that we have a problem with overuse of low-value interventions, but, without robust and reliable audit and feedback, most do not appreciate the extent to which they may be contributing.

  6. 6) Some CW recommendations are solidly founded upon a robust evidence base of scientific literature and include CAEP-endorsed examples related to the use of CTs in suspected pulmonary embolism and minor head trauma. There are others, however, that lack the backing of a strong recommendation from a high-quality clinical practice guideline or the shifting sands of medical knowledge. CAEP CW identified antibiotics for drained abscesses as a practices to avoid. However, the evidence on these questions, some of which emerged after the CW recommendations were published, actually brought that recommendation into question.Reference Talan, Mower and Krishnadasan 7

  7. 7) To move the needle on low-value interventions, we need reliable data on the current degree of overuse across multiple jurisdictions. Perhaps we are already choosing wisely and are not worthy of the campaign’s focus. Work in Calgary suggests that the use of radiographic imaging for low-risk back pain patients in the ED sits at only 15%. Do we need to improve on that? The current state of overuse is not well-known and could not inform all of the CW recommendations arising from the CAEP effort. In the same vein, reasonable benchmarks and targets remain elusive. Without specific goals to aim for, it can be difficult to move from general guidance to a more formal quality improvement (QI) strategy.

  8. 8) Living largely in a virtual space, CW recommendations are not commonly delivered at the point of care and decision-making. In a U.S. survey, emergency physicians were largely unable to distinguish a false CW recommendation from a real one, whereas no more than 36% of respondents could recall more than three of the five CW recommendations from the list developed by the American College of Emergency Medicine.Reference Lin, Nguyen and Probst 8

I could not agree more with my highly esteemed debating partner that overuse and wasteful practices are problems in urgent need of fixing. I have outlined eight reasons why CW, in its current formulation, is not delivering as hoped. All is not lost; however, re-reading the arguments that we have cited to explain CW’s lackluster impact of 8 years in, and imagining them as a “to-do list” for how to fix what is broken in the current movement, we may be able to deliver more equitable and appropriate emergency care in Canadian EDs.

THE CHOOSING WISELY CAMPAIGN WILL NOT IMPACT PHYSICIAN BEHAVIOUR AND CHOICES

Against: Rebecca Lys, Megan Laupacis, and Heather Murray (@heatherm211)

The first duty of the physician is to educate the masses not to take medicine.

– Sir William Osler

Don’t just do something, stand there

In September 2016, an American died from a bacterial infection resistant to every available antibiotic, and an old threat became a new reality.Reference Brink 9 Overtreatment of self-limiting infections with antibiotics plays a major role in the development of these lethal “superbugs,” which now kill more Americans than emphysema, HIV/AIDS, Parkinson’s disease, and homicide combined.Reference Laxminarayan, Duse and Wattal 10 - Reference Llor and Bjerrum 12 Beyond resistance, there are other consequences of antibiotic overtreatment. As emergency physicians and medical trainees, we see them daily – epidemics of Clostridium difficile infections, adverse drug events, and the medicalization of self-limiting illness.Reference Llor and Bjerrum 12

CW is an international campaign to address the epidemic of unnecessary care. Created by and for physicians and endorsed by over 50 medical societies, including CAEP, it is more than a series of lists. It engages physician, trainee, and patient stakeholders in every section of our health care system. 13 The goal of cutting overtesting and overtreatment is ambitious, but it will succeed because it promotes accelerated knowledge translation, win-win solutions for physicians and patients, and much needed culture change.

CW will succeed because it is our best solution to address health care resource shortages. Health care spending as a share of Gross Domestic Product in Canada has been decreasing since 2010. 14 Meanwhile, the Canadian Institute for Health Information (CIHI) reports that up to 30% of health care in Canada is potentially unnecessary. 15 Unnecessary imaging is estimated to cost Canadians $220 million annually and increases lifetime cancer incidence.Reference Emery, Shojania and Forster 16 , Reference Mathews, Forsythe and Brady 17 Despite this, 30% of patients presenting to Ontario and Alberta EDs for low-risk minor head trauma still received a CT scan. 15 As physicians, we are presiding over a rising sea of unnecessary tests and medications. With growing demands and finite resources, it is untenable to order tests and treatments that do not improve patient outcomes and worse, cause harm. We cannot stand by and be complacent – overtesting and overtreatment are the greenhouse gases of medicine, and patient harms in the face of unsustainable spending are the consequences.

CW will succeed because physicians designed it to overcome the barriers to reducing unnecessary care: habit, malpractice worries, safety concerns, and patient preference.Reference Bhatia, Levinson and Shortt 18 , Reference Sedrak, Patel and Ziemba 19 Fundamentally, physicians want to provide patients with safe and effective care. Yet, they often focus on the upfront dangers of missing an unlikely problem and fail to see the downstream dangers of unnecessary radiation, antibiotic exposure, prescription inertia, or false positives. A systematic review found that physicians frequently overestimated benefit and underestimated harm when evaluating tests and treatments.Reference Hoffmann and Del Mar 20 Evidence-based interventions to reduce low-value care, such as the Canadian ankle, knee, and C-spine rules, have been shown to reduce diagnostic imaging.Reference Stiell, Clement and Grimshaw 21 - Reference Stiell, Wells and Hoag 23 CW has compiled lists of low-value tests and treatments in collaboration with over 50 medical societies, including CAEP. As the interventions gain recognition, the pressure to “do something” will be reduced by normalizing watching and waiting (one of the most difficult “treatments” for physicians to administer). Malpractice concerns are allayed because the CW guidelines demonstrate that conservative approaches are within the standard of care.Reference Lenzer 24

CW will succeed because it has a plan for action. Knowledge translation for new advances has been estimated to take an average of 17 years,Reference Morris, Wooding and Grant 25 but effective programs can reduce this lag by engaging with decision-makers, designing comprehensive intervention plans, and regularly evaluating progress.Reference Needham 26 In addition to providing education about unnecessary care, the program has created toolkits to make standardized nationwide implementation easier for providers. 27 It has also published a framework based on physician attitudes, behaviours, and patient engagement to measure its effectiveness going forward.Reference Bhatia, Levinson and Shortt 18

How soon will CW show effectiveness on a larger scale? It will take time to change a culture of rewarding batteries of tests that find “incidentalomas,” but there are already early positive results.Reference Rosenberg, Agiro and Gottlieb 28 , Reference Lin, Cserti‐Gazdewich and Lieberman 29 A recent U.S. analysis of database compensation claims examined the impact of the CW recommendations published in 2012. There were clinically significant reductions in two tests, statistically significant but clinically insignificant decreases in four recommendations, and a small increase in one recommendation.Reference Rosenberg, Agiro and Gottlieb 28 There is a growing literature of successful local, “bottom-up” CW interventions across many disciplines. These include initiatives that have reduced red blood cell transfusions,Reference Lin, Cserti‐Gazdewich and Lieberman 29 urinary catheter usage on general medical wards,Reference Leis, Corpus and Rahmani 30 low-risk ankle and back imaging,Reference Min, Chan and Aristizabal 31 , Reference Al-Sani, Ben-Yakov and Harvey 32 and unnecessary bloodwork.Reference Fralick, Hicks and Chaudhry 33 - Reference Naugler, Hemmelgarn and Quan 35 Small interventions can create large changes. The introduction of a special requisition needed for vitamin D testing in Alberta decreased the rate of testing by 91.4%, with an estimated annual cost saving of $1 million.Reference Naugler, Hemmelgarn and Quan 35 Rather than relying on an overhaul of our entire health care funding structure, the CW organization has addressed culture change within our current system. It has identified easy high yield behaviours that can make a large difference in the problem of overtesting and overtreatment, and do not depend on a major political culture change. These initiatives highlight the buy-in and engagement of physicians and institutions on the front line, and the positive findings show the campaign progress.

CW will succeed because it engages learners at the earliest stages of training. Recognizing that starting with a fresh slate is significantly easier than unlearning bad habits, CW Canada launched Students and Trainees Advocating for Resource Stewardship (STARS) in 2015. 36 Delegates from all 17 Canadian medical schools learn about resource stewardship, how to think critically before ordering tests and treatment, and how to approach difficult conversations with supervisors and patients about unnecessary care.Reference Naugler, Hemmelgarn and Quan 35 They have organized events and interest groups to spread their knowledge of resource stewardship with other students, as well as influencing formal curricular changes.Reference Lakhani, Lass and Silverstein 37 Today’s trainees will engage in conversations with patients before ordering a battery of tests or writing a mindless prescription because they know that shared decision-making and accurate risk/benefit assessments are the key behaviours of excellent doctors.

CW will succeed because it prioritizes patient-centred outcomes and shared decision-making with patients. Physician surveys cite patient preference as a significant driver of unnecessary care.Reference Friedman, Liu and Wakeling 38 However, when patients are aware about the harms of unnecessary care, they are more likely to choose wisely.Reference Abu Taha, Abu-Zaydeh and Ardah 39 - Reference Vallin, Polyzoi and Marrone 41 CW is addressing demand for unnecessary care with patient-focused posters and educational pamphlets, explaining in simple terms why “more is not always better” when it comes to medical care. The materials emphasize the individual harms or opportunity costs of unnecessary care (such as missing work), which is more persuasive than materials emphasizing escalating health care costs.Reference Schlesinger and Grob 42 , Reference Santa 43 Early studies have shown that this messaging is effective. When patients and caregivers in family practice waiting rooms were given educational materials on antibiotics for sinusitis and other CW guidelines, their knowledge improved significantly; 70% reported that they intended to discuss this new information with their health care provider, family, or friends, and almost 85% reported that they were ready to adopt or had already adopted the CW philosophy.Reference Silverstein, Lass and Born 44

Overtesting and overtreatment are harmful to patients and threaten Canada’s health care system. CW is well-positioned to reduce unnecessary care with its multi-pronged approach. Specialty-specific recommendations and toolkits have been used to implement interventions that have cut unnecessary testing by as much as 91%.Reference Naugler, Hemmelgarn and Quan 35 Patient-directed educational materials help patients understand why more treatment is not necessarily better and will reduce the pressure that physicians face from patients to provide unnecessary care. CW works with medical trainees, emphasizing therapies with proven benefits to patients. As with most developments in medicine, reducing unnecessary care in clinical practice will take time. By assisting physicians, targeting trainees, and reframing the messaging to the public, CW is poised to accelerate the much-needed culture shift towards value-based care. In this era of increasing health care costs and antibiotic-resistant superbugs, we cannot afford to let it fail.

Competing interests: None declared.

References

REFERENCES

1. Choosing Wisely Canada. www.choosingwisely.ca.Google Scholar
2. Sahota, IS, Lang, E. Reducing low-value interventions in the emergency department: you may be part of the problem. CJEM 2017;19(2):143-146, 10.1017/cem.2016.349.Google Scholar
3. Rosenberg, A, Agiro, A, Gottlieb, M, et al. JAMA early trends among seven recommendations from the Choosing Wisely campaign. Intern Med 2015;175(12):1913-1920, 10.1001/jamainternmed.2015.5441.Google Scholar
6. Brehaut, JC, Colquhoun, HL, Eva, KW, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med 2016;164(6):435-441, 10.7326/M15-2248.Google Scholar
7. Talan, DA, Mower, WR, Krishnadasan, A, et al. Trimethoprim-sulfamethoxazole versus placebo for uncomplicated skin abscess. N Engl J Med 2016;374(9):823-832, 10.1056/NEJMoa1507476.Google Scholar
8. Lin, MP, Nguyen, T, Probst, MA, et al. Emergency physician knowledge, attitudes, and behavior regarding ACEP’s Choosing Wisely recommendations: a survey study. Acad Emerg Med 2017;24(6):668-675, 10.1111/acem.13167.Google Scholar
9. Brink, S. A superbug that resisted 26 antibiotics. National Public Radio; 2017. Available at: http://www.npr.org/sections/goatsandsoda/2017/01/17/510227493/a-superbug-that-resisted-26-antibiotics (accessed 1 May 2017).Google Scholar
10. Laxminarayan, R, Duse, A, Wattal, C, et al. Antibiotic resistance – the need for global solutions. Lancet Infect Dis 2013;13(12):1057-1098.10.1016/S1473-3099(13)70318-9Google Scholar
11. Datta, S, Wattal, C, Goel, N, et al. A ten year analysis of multi-drug resistant blood stream infections caused by Escherichia coli & Klebsiella pneumoniae in a tertiary care hospital. Indian J Med Res 2012;135(6):907-912.Google Scholar
12. Llor, C, Bjerrum, L. Antimicrobial resistance: risk associated with antibiotic overuse and initiatives to reduce the problem. Ther Adv Drug Saf 2014;5(6):229-241.Google Scholar
13. Choosing Wisely Canada. The lists; (n.d.). Available at: http://choosingwiselycanada.org/recommendations/ (accessed 18 May 2017).Google Scholar
14. OECD. Health at a glance 2015: OECD indicators; 2015. Available at: http://apps.who.int/medicinedocs/documents/s22177en/s22177en.pdf (accessed 1 June 2017).Google Scholar
15. CIHI. Unnecessary care in Canada; 2017. Available at: https://www.cihi.ca/sites/default/files/document/choosing-wisely-baseline-report-en-web.pdf (accessed 1 June 2017).Google Scholar
16. Emery, DJ, Shojania, KG, Forster, AJ, et al. Overuse of magnetic resonance imaging. JAMA Intern Med 2013;173(9):823-825.10.1001/jamainternmed.2013.3804Google Scholar
17. Mathews, JD, Forsythe, AV, Brady, Z, et al. Cancer risk in 680,000 people exposed to computed tomography scans in childhood or adolescence: data linkage study of 11 million Australians. BMJ 2013;346:f2360.Google Scholar
18. Bhatia, RS, Levinson, W, Shortt, S, et al. Measuring the effect of Choosing Wisely: an integrated framework to assess campaign impact on low-value care. BMJ Qual Saf 2015;24(8):523-531.Google Scholar
19. Sedrak, MS, Patel, MS, Ziemba, JB, et al. Residents’ self-report on why they order perceived unnecessary inpatient laboratory tests: why residents order unnecessary inpatient laboratory tests. J Hosp Med 2016;11(12):869-872.Google Scholar
20. Hoffmann, TC, Del Mar, C. Clinicians’ expectations of the benefits and harms of treatments, screening, and tests: a systematic review. JAMA Intern Med 2017;177(3):407-419.Google Scholar
21. Stiell, IG, Clement, CM, Grimshaw, J, et al. Implementation of the Canadian C-spine rule: prospective 12 centre cluster randomised trial. BMJ 2009;339:b4146.Google Scholar
22. Stiell, IG, Wells, G, Laupacis, A, et al. A multicentre trial to introduce clinical decision rules for the use of radiography in acute ankle injuries. BMJ 1995;311(7005):594-597.Google Scholar
23. Stiell, IG, Wells, GA, Hoag, RA, et al. Implementation of the Ottawa knee rule for the use of radiography in acute knee injuries. JAMA 1997;278(23):2075-2079.10.1001/jama.1997.03550230051036Google Scholar
24. Lenzer, J. Choosing Wisely: setbacks and progress. BMJ 2015;351:h6760.10.1136/bmj.h6760Google Scholar
25. Morris, ZS, Wooding, S, Grant, J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011;104(2):510-520.Google Scholar
26. Needham, DM. Translating evidence into practice: a model for large scale knowledge translation. BMJ 2008;337:a1714.Google Scholar
27. Choosing Wisely Canada. Toolkits. Available at: http://www.choosingwiselycanada.org/in-action/toolkits/ (accessed 18 May 2017).Google Scholar
28. Rosenberg, A, Agiro, A, Gottlieb, M, et al. Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Intern Med 2015;175(12):1913-1920.Google Scholar
29. Lin, Y, Cserti‐Gazdewich, C, Lieberman, L, et al. Improving transfusion practice with guidelines and prospective auditing by medical laboratory technologists. Transfusion 2016;56(11):2903-2905.Google Scholar
30. Leis, JA, Corpus, C, Rahmani, A, et al. Medical directive for urinary catheter removal by nurses on general medical wards. JAMA Intern Med 2016;176(1):113-115.Google Scholar
31. Min, A, Chan, VWY, Aristizabal, R, et al. Clinical decision support decreases volume of imaging for low back pain in an urban emergency department. J Am Coll Radiol 2017;14(7):889-899.Google Scholar
32. Al-Sani, F, Ben-Yakov, M, Harvey, G, et al. Low risk ankle rule, high reward – a quality improvement initiative to reduce ankle x-rays in the pediatric emergency department [abstract]. CJEM 2017;19(S1):S83.Google Scholar
33. Fralick, M, Hicks, LK, Chaudhry, H, et al. REDucing unnecessary coagulation testing in the emergency department (REDUCED). BMJ Qual Improv Rep 2017;6(1):1-4.Google Scholar
34. Venkatesh, AK, Hajdasz, D, Rothenberg, C, et al. Reducing unnecessary blood chemistry testing in the emergency department. Am J Med Qual 2017; epub, 10.1177/1062860617691842.Google Scholar
35. Naugler, C, Hemmelgarn, B, Quan, H, et al. Implementation of an intervention to reduce population-based screening for vitamin D deficiency: a cross-sectional study. CMAJ Open 2017;5(1):E36-E39.Google Scholar
36. Choosing Wisely Canada. Medical education. Available at: http://www.choosingwiselycanada.org/medical-education/ (accessed 1 May 2017).Google Scholar
37. Lakhani, A, Lass, E, Silverstein, WK, et al. Choosing wisely for medical education: six things medical students and trainees should question. Acad Med 2016;91(10):1374-1378.Google Scholar
38. Friedman, CL, Liu, J, Wakeling, S. Integrating resource stewardship into the medical curriculum. Poster presented at: McMaster Medical Student Research Day; 2017.Google Scholar
39. Abu Taha, A, Abu-Zaydeh, AH, Ardah, RA, et al. Public knowledge and attitudes regarding the use of antibiotics and resistance: findings from a cross‐sectional study among Palestinian adults. Zoonoses Public Health 2017;63(6):449-457.Google Scholar
40. Awad, IA, Aboud, ES. Knowledge, attitude and practice towards antibiotic use among the public in Kuwait. PLoS One 2015;10(2):e0117910.Google Scholar
41. Vallin, M, Polyzoi, M, Marrone, G, et al. Knowledge and attitudes towards antibiotic use and resistance – a latent class analysis of a Swedish population-based sample. PLoS One 2016;11(4):e0152160.Google Scholar
42. Schlesinger, M, Grob, R, , Treating, fast and slow: Americans’ understanding of and responses to low‐value care. Milbank Q 2017;95(1):70-116.Google Scholar
43. Santa, JS. Communicating information about “what not to do” to consumers. BMC Med Inform Decis Mak 2013;13(Suppl 3):S2.Google Scholar
44. Silverstein, W, Lass, E, Born, K, et al. A survey of primary care patients’ readiness to engage in the de-adoption practices recommended by Choosing Wisely Canada. BMC Res Notes 2016;9(1):301.Google Scholar