Introduction: Despite studies highlighting the inaccuracies of self-assessment, practicing physicians continue to rely on self-perception to maintain clinical competence. Many approaches have been proposed to augment physician performance. In the realm of Quality Improvement (QI), Audit and Feedback (A&F) has a modest effect. Educators have proposed coaching interventions and academic constructs have invoked training for early-career clinicians. Very few of these are driven by the perceptions and the needs of the end-user - the physicians. We currently lack a model to understand physicians’ perceptions of their own practice data and an understanding of the factors which would enable practice change. In this study, we sought to develop a model for data feedback which may best help physicians change practice. Methods: In a previous study, we conducted a needs analysis of 105 physicians in the Hamilton-Niagara area in order to understand which data metrics were most valuable to physicians. Using the survey results, we designed an interview guide that was used as a qualitative study of physicians’ perspectives on A&F. By intentional sampling, we recruited 15 physicians amongst gender groups, types of practice (academic vs community) and durations of practice. We conducted this interview with all 15 participants which were then transcribed. We then performed thematic analysis and extraction of all interviews using a realist framework. These were then translated into broader themes and, by using a grounded theory framework, created a model to understand how physicians relate practice data to their own sense of self. Interviews were anonymized and no identifying data was shared as part of the interview. All interviewees consented to participation at the outset and could withdraw at any time. Results: Via stakeholder interviews from 15 key informants, we developed a model for the understanding of how a physician's sense of self and the nature of the data (quantity and quality) may be combined to understand the likelihood of practice change and the adoption of the change strategy. Using this model, it is possible to understand the conditions under which A&F would provide the greatest opportunity for practice change. Conclusion: Physician identity intersects with A&F data to shed insights on practice improvement. Understanding the core identity constructs of different physician groups may allow for increased uptake in A&F processes.

Introduction: As part of our audit and feedback process, Emergency Physicians (EP) are provided feedback on flow metrics and resource utilization. We analysed the relationship between two specific metrics (adjusted workload measurement (AWM), with the number of patients seen per hour adjusted according to CTAS, and percentage of revisits within 72 hours and diagnostic imaging use. Unfortunately, we are unable to evaluate quality of care, nor appropriateness of DI indication at this stage. Methods: We used data from 86 physicians at an academic ED, from June 1, 2015 to May31, 2017. The Data Envelope Analysis (DEA) model incorporated performance quality measures as outputs and efficiency measures as inputs. DEA is a method widely used in physician performance analysis. The method provides a score (optimal performance efficiency-OPE) for each EP based on maximization of the performance (AWM) in proportion to the combination of efficient use of resources, diagnostic imaging (DI). The score was used to regress against demographic characteristics and training. Results: The median AWM was 6.8 (quartiles Q1-Q3 = 6.4-7.4) with the median diagnostic imaging use of percentages of CT (median = 10.1, 8.6-11.9), US (median = 4.7, 3.6-5.6) and x-ray (80, 74-84). The EPs who had highest AWM combined with least use of DI (OPE = 100%), provided median AWM of 9.1 (range 8.9-9.7) with percentage CT, US and x-ray medians at 5.8% (range 5.8-6.2), 2.7% (range 2.4-3.6) and 59% (range 59-72). These provided benchmarks for optimal performance indicators. We found statistically significant differences of OPE scores based on gender (men 4.1 times higher, p < 0.001) and degree (RCPS < CCFPEM, Other < CCFPEM, p < 0.001). Overall AWM diminishes at the rate of 14% (95%CI: 9-20%) for a combination of 100 DI tests ordered. In order to reach the optimal level of performance, to reach an OPE of 100%, the median CT use percentage needs to be reduced by 6% (quartile range 3.9- 7.7%), US by 2.2% (quartile range 1.5-3.4%) and x-rays by 37.2% (quartile range: 26.8-44.3%). Return visit rates were not associated with DI use, possibly due to homogeneity in the percentage of return visits. Conclusion: We found significant performance variations in terms of average workload measurement in proportion to the weighted average of diagnostic imaging use, with increased use of DI being associated with decreasing AWM. Percentage of return visits does not appear to be useful as a performance indicator.

Background: Despite strong evidence recommending supportive care as the mainstay of management for most infants with bronchiolitis, prior studies suggest that many of these patients receive low-value interventions. Providing clinicians with their practice reports and peer comparator data or an achievable benchmark of care (audit and feedback) has been shown to be an effective strategy to improve adherence to guidelines. Aim Statement: To decrease low-value care (use of any or all of chest radiographs, viral testing and salbutamol) in infants with bronchiolitis by delivering individual physician reports in addition to Group Facilitated Feedback Sessions (GFFS) to pediatric emergency physicians (PEPs). Measures & Design: Our cohort included 3,883 patients ≤12 months old that presented to two emergency departments with a diagnosis of bronchiolitis from April 1, 2013 to April 30, 2018. Using administrative data we captured baseline characteristics and interventions. Consenting PEPs received two audit and feedback (A&F) reports which included their individual and peer comparator data. Two multi-disciplinary GFFS (including inpatient pediatricians, nurse, learners and respiratory therapists) presented data and identified barriers and enablers of reducing low-value care. The primary outcome was the proportion of patients who received any low-value intervention, and was analyzed using statistical process control charts. Process measures (consent to obtain report, attendance and evaluations from the feedback session) and balancing measures were also captured. Evaluation/Results: 78% of PEPs consented to receive their A&F reports. Patient baseline characteristics were similar in the baseline (n = 3109) and intervention period (n = 774). Following the baseline physician reports and the GFFS, low-value care decreased from 42.6% to 27.1% (absolute difference: -15.5%; 95% confidence interval (CI): -19.8% to -11.2%) and 78.9% to 64.4% (absolute difference: -14.5%; 95% CI: -21.9% to -7.2%) in patients who were not admitted and admitted, respectively. Balancing measures such as ICU admission (absolute difference: -0.6%; 95%CI: -5.7% to 4.4%) and ED revisit within 72 hours (absolute difference: -0.1%; 95% CI: -3.1% to 3.0% non-admitted patients, 1.0%; 95% CI: -1.2% to 3.2% admitted patients) were unchanged. Discussion/Impact: The combination of audit and feedback and a GFFS significantly reduced low-value care for pediatric patients with bronchiolitis by PEP's.

Introduction: Bronchiolitis is a viral respiratory infection and the most common reason for hospitalization of infants. Despite evidence that few interventions are beneficial in patients with bronchiolitis, other studies would have shown that a significant proportion of patients undergo various forms of low value care. This objective of this project was to 1. establish baseline management of bronchiolitis in the Calgary Zone, and 2. deliver audit and feedback (A&F) reports to pediatric emergency physicians (PEP) to identify opportunities and strategies for practice improvement. Methods: This retrospective cohort study included all patients 12 months old that presented to a Calgary emergency department or urgent care center with a diagnosis of bronchiolitis from April 1, 2013 to March 31, 2017. Using data from various electronic health data sources, we captured age, vital signs, CTAS, common therapeutic interventions (bronchodilators, steroids, antibiotics) and investigations (chest x-ray (CXR), viral studies, antibiotics). Results were stratified by site and by admission status. Descriptive statistics were used to report baseline characteristics and interventions. Interhospital ranges (IHR) were provided to compare different hospitals in the zone. For the A&F component of the project, consenting PEP received a report of both their individual and peer comparator data and an in-person multi-disciplinary facilitated feedback session. Results: We included 4023 patients from all 6 sites (range from 28 to 3316 patients). Admission rates were 21.7% (IHR 0-29%). Mean age was 5.4 months old. Bronchodilator use was 27.0% (IHR 21-41%). 22.0% of patients received a CXR (IHR 0-57%) and 30.3% had viral studies done (IHR range 0.8-33%). PEP had higher usage of viral studies (30% vs 5.7%), whereas non-PEP had higher CXR usage (46.2% vs 23.4%). 41 of 66 PEP consented to receive their individual A&F reports (62%). In the facilitated feedback session PEP 1. identified two areas (bronchodilators and viral studies) where improvements could be made and 2. discussed specific strategies to decrease practice variation and minimize low value care including development of a multi-disciplinary care pathway, alignment with in-patient management, education and repeated A&F reports. Conclusion: Significant variability exists in management of patients with bronchiolitis across different hospitals in our zone. A facilitated feedback session identified areas for improvement and multi-disciplinary strategies to reduced low value care for patients with bronchiolitis. Future phases of this project include repeated data in 6 months and implementation of a provincial care pathway for the management of bronchiolitis.

Introduction: Intravenous (IV) therapy in the emergency department (ED) is associated with risk of harm from IV complications, higher ED monitoring requirements and increased ED length of stay (LOS), the latter a measure most cumbersome in lower-acuity patients that are eventually discharged from the ED. The aim of this quality improvement project was to evaluate the effectiveness of educational and audit-and-feedback interventions, with a goal of relative reduction of ED IV therapy by 20% over eight week periods, in lower-acuity patients in the high-turnover intake area of the ED who were discharged from the ED. Methods: The first cycle of the project was education about IV therapy use and alternatives in lower-acuity, ED patients (Canadian Triage Acuity Scale (CTAS) 3 and 4) from July 2 to August 31, 2017. Education was delivered through email information, posters, education sessions with nurse educators, and working groups sharing information. The second cycle of the project, from October 16 to December 15, 2017, also integrated an audit-and-feedback tool whereby physicians received their own pooled ordering data of IVs from the same period the previous year and then trial period as well pooled comparison averages for the physician group in the population of interest. Measures were the percentage of IVs ordered by physicians and administered by nurses in the population of interest in each time period. Results: From July 2 to August 31, 2017, when the intervention was education only, the rate of IV therapy changed from 31% to 37%, which reflects a 19% relative increase in IV use. In the beginning of the second cycle utilizing both education and audit-and-feedback interventions, from October 16 to December 15, 2017, 35% of patients had IV therapy. At the end of the second cycle, 25% of patients had IV therapy, a 28% relative decrease in IV therapy rates. When both cycles are reviewed sequentially, IV therapy rates decreased from 31% to 25%, a relative reduction of IV usage of 19%. Conclusion: In this quality improvement project, an educational initiative for the interdisciplinary team alone did not reduce IV use in lower-acuity patients. Concurrent education and audit-and-feedback interventions were more effective than education alone in decreasing IV therapy in appropriately selected patients in a tertiary ED.

Introduction: Prior research has shown that audit and feedback (A &F) can be an effective tool for practice change. However, questions remain about how to optimize A&F. The objectives of this project were to determine if: 1) there are differences in practice between physicians who do, and do not, consent to receive a confidential report on their practice and; 2) if there is a relationship between consenting physicians self-predicted and actual practice. Methods: This was a prospective, cross-sectional study embedded in a larger quality improvement (QI) initiative to align physician practice with best evidence in the emergency department (ED) care of infants with bronchiolitis. All physicians practicing in the ED of a tertiary care pediatric hospital were offered the opportunity to consent to receive an individual, confidential data report on their practice. Prior to receiving their data, consenting physicians completed a survey which asked them to predict the proportion of bronchiolitic patients for whom they ordered diagnostic tests or treatments. We used chi-squared testing to compare the proportion of consenting and non-consenting physicians whose diagnostic test (Chest X-ray (CXR), viral study) and treatment (steroid, Ventolin) ordering was above the median for all ED physicians. We used Pearsons correlation to assess the relationship between consenting physicians self -predicted and actual practice. Results: 56% (37/66) of physicians consented to receive a data report. The median proportion of patients with an x-ray ordered was 20%, 63% of non-consenters were above the median, compared to 36% of consenters (X2 (1, N=66)=4.91 p=0.03). For viral testing, 31% of patients had a test ordered, with 50% of non-consenters and 50% consenters above the median (( X2 (1, N=66) =0 p=1); 11% of patients had steroids ordered, with 53% of non-consenters and 47% of consenters above the median ( X2 (1, N=66)=0.24 p-0.621); and 18% of patients had Ventolin ordered, with 60% of non-consenters and 42% of consenters above the median ( X2 (1, N=66) =2.2 p=0.138). There was a moderate correlation between physicians predicted and actual practice with respect to viral testing (r=0.67), but minimal correlation for CXR (0.05), steroids (r=0.17) or Ventolin (r=0.33) ordering. Conclusion: The finding that physicians have a limited ability to accurately predict their own performance emphasizes the importance of providing physicians with feedback. However, our results suggest that the consent process may be a potential barrier to effective A &F.

Introduction: There is a growing interest in providing clinicians with performance reports via audit and feedback (A&F). Despite significant evidence exists to support A&F as a tool for self-reflection and identifying unperceived learning needs, there are many questions that remain such as the optimal content of the A&F reports, the method of dissemination for emergency physicians (EP) and the perceived benefit. The goal of the project was to 1. evaluate EP perceptions regarding satisfaction with A&F reports and its’ ability to stimulate physicians to identify opportunities for practice change and 2. identify areas for optimization of the A&F reports. Methods: EP practicing at any of the four adult hospital sites in Calgary were eligible. We conducted a web survey using a modified Dillman technique eliciting EP perspectives regarding satisfaction, usefulness and suggestions for improvement regarding the A&F reports. Quantitative data were analyzed descriptively and free-text were subjected to thematic analysis. Results: From 2015 onwards, EP could access their clinical performance data via an online dashboard. Despite the online reports being available, few physicians reviewed their reports stating access and perceived lack of utility as a barrier. In October 2016, we began disseminated static performance reports to all EP containing a subset of 10 clinical and operational performance metrics via encrypted e-mail. These static reports provided clinician with their performance with peer comparator data (anonymized), rationale and evidence for A&F, information on how to use the report and how to obtain continuing medical education credits for reviewing the report. Conclusion: Of 177 EP in Calgary, we received 49 completed surveys (response rate 28%). 86% of the respondents were very/satisfied with the report. 88% of EP stated they would take action based on the report including self-reflection (91%) and modifying specific aspects of their practice (63%). Respondents indicated that by receiving static reports, 77% were equally or more likely to visit the online version of the eA&F tool. The vast majority of EP felt that receiving the A&F reports on a semi-annual basis was preferred. Three improvements were made to the eA&F based on survey results: 1) addition of trend over time data, 2) new clinical metrics, and 3) optimization of report layout. We also initiated a separate, real-time 72-hour bounceback electronic notification system based on the feedback. EP value the dissemination of clinical performance indicators both in static report and dashboard format. Eliciting feedback from clinicians allows iterative optimization of eA&F. Based on these results, we plan to continue to provide physicians with A&F reports on a semi-annual basis.

There is increasing interest in collecting morbidity data from general practice. We describe our experience from Northumberland MEDICS, one of the first morbidity data collection projects in the UK. All Northumberland practices were invited to participate. Data were initially collected every 3–6 months and included the prevalence of chronic diseases, disability in the over 75s, and recording of health markers, such as smoking status. Thirty-three out of 52 practices participated. There was marked variation in prevalences and recording of health markers between practices. Recorded prevalence of hypertension and diabetes increased steadily from 1994 to 1998. Outcomes, judged by the original objectives, were mixed. However, as the project evolved, evidence emerged that MEDICS was contributing to a culture in which the use of data from practice clinical systems to improve patient care has become a core objective. Key lessons from our experience include appreciating: the importance of data quality and minimising workload for practices; the difficulties practices face in recording morbidity data consistently and systematically; the limitations of GP morbidity data for health needs assessment and commissioning at district level; and the need to focus on providing useful and relevant data for individual practices. MEDICS now covers all 53 Northumberland practices. The project focuses on recording and analysing data to help practices improve structured patient care. Increasingly data collection in general practice in Northumberland is seen as a core activity with the current dataset linking closely with local priorities and reflecting national initiatives such as Clinical Governance and National Service Frameworks.