Infectious disease (ID) physicians are important to the implementation of antibiotic stewardship processes. Reference Lesprit, Landelle and Brun-Buisson1–Reference Barlam, Cosgrove and Abbo6 However, ID physicians are absent from ∼25% of US hospitals, and 80% of US counties lack an ID physician. Reference Doron, Nadkarni and Lyn Price7–Reference Walensky, McQuillen, Shahbazi and Goodson9 The absence of ID support is a barrier to the effective implementation of stewardship processes. Reference Livorsi, Nair and Lund10,Reference Livorsi, Stewart Steffensmeier, Perencevich, Goetz and Schacht Reisinger11
Telehealth can provide ID physician expertise to resource-limited settings. Reference Young, Abdel-Massih and Herchline12 Telehealth can be used to assist local stewardship strategies even when direct patient care is not provided. Studies have reported that this approach can reduce unnecessary antibiotic use and minimize antibiotic-related adverse events. Reference Bartlett and Siola13–Reference Wilson, Banks and Crnich17
The most effective and efficient way of deploying telehealth for stewardship activities is undefined, and the barriers to using telehealth in this manner are not well described. In this project, we assessed the implementation of telehealth-supported stewardship activities across 3 Veterans’ Affairs medical centers (VAMCs).
Methods
Study design
We performed a before-and-after, quasi-experimental mixed-methods study across 3 VAMCs with acute care and long-term care (LTC) units to evaluate the implementation of telehealth-supported stewardship activities. We defined a baseline (2019–2020) and an intervention period (2021). The reach–effectiveness–adoption–implementation–maintenance (RE-AIM) framework was used to guide the evaluation (Table 1). Reference Glasgow, Vogt and Boles18
Table 1. RE-AIM Framework
Site selection
Eligible VAMCs were (1) designated as rural, based on rural–urban commuting-area codes; (2) lacked onsite ID physician consultation, and (3) lacked any ID physician or ID pharmacist support for their antibiotic stewardship processes. We used data from a mandatory antibiotic stewardship survey (2016) to identify the small number of VAMCs meeting these criteria. Reference Roselle, Kelly, Neuhauser and Crump19 Once an eligible site was identified, we used internal VHA resources to contact each site’s designated stewardship pharmacist. In total, 3 geographically diverse VAMCs were invited to participate in this program, and all 3 sites agreed to participate.
Site characteristics
Intervention sites were each located in a different state. Each intervention site had both acute-care beds and LTC beds. During the 12-month intervention, average daily census ranged from 3–8 acute-care patients per VAMC, with wider variations in the average daily census on their LTC units (site 1, n = 64 patients; site 2, n = 93 patients: site 3, n = 8 patients). Two sites had an onsite microbiology laboratory, and all 3 sites lacked access to rapid molecular diagnostics. Delays in reporting microbiologic results were common. All sites had a stewardship pharmacist who had completed a 1-year clinical pharmacy residency program and a stewardship certification course but not a formal postgraduate training program in ID. These pharmacists, who had each served in their stewardship champion role for at least 4.5 years, also had several other clinical responsibilities, which varied by site. Some of these stewardship pharmacists were performing prospective audit and feedback (PAF) before the intervention, but no physician was helping them with the process. Each site had access to offsite ID consultations, which were completed by an outside ID physician through either an e-consultation or an offsite outpatient visit. Sites were able to continue to place these outside consultations throughout the project. All sites used the same electronic medical record (EMR) as the remote ID physician’s home site. Each site requested the ID physician’s credentialing information from his own VA facility. Because all sites were VHA facilities, the ID physician did not need to apply for a separate medical license from each site’s state.
Intervention
Our intervention primarily used the following implementation strategies to improve antibiotic-prescribing.
Develop stakeholder relationships
During the fall of 2020, the remote ID physician met virtually (via Microsoft Teams software, Redmond, WA) with the local stewardship pharmacist, local inpatient prescribers, and hospital leadership at each of the 3 participating sites. The purpose of these meetings was to highlight the rationale for the project and to begin to build rapport.
Prospective audit and feedback (PAF)
During the 12-month intervention period, the remote ID physician met with the stewardship pharmacist at each site via Microsoft Teams on Monday, Wednesday, and Friday mornings to conduct telehealth-mediated PAF, or tele-PAF. Prior to each meeting, the stewardship pharmacist generated a list of all patients in acute-care and LTCs who were actively receiving antibiotics. The pharmacist was encouraged to review each patient’s medical record prior to the meeting. During the video call, the remote ID physician and the site pharmacist discussed each patient on antibiotics and looked for opportunities to prescribe antibiotics in a more evidence-based manner (eg, changing the selection, route of administration, planned duration, etc). Any recommendations were relayed in real time to the primary prescriber. Recommendations were usually communicated by telephone or in person. The remote ID physician frequently virtually joined the pharmacist in providing feedback, especially for more complicated issues or for prescribers who had shown a hesitancy to accept recommendations directly from the pharmacist. Documentation of recommendations was left to the pharmacist’s discretion. The stewardship pharmacist at sites 1 and 2 consistently documented tele-PAF recommendations in the EMR. At site 2, the remote ID physician would cosign the stewardship pharmacist’s note.
Education
At the end of every month, all inpatient providers at each site were e-mailed a 2-page educational sheet about a topic related to antibiotic prescribing. Topics were selected based on local needs and included the following: asymptomatic bacteriuria, Clostridioides difficile infections, COVID-19 management, duration of therapy for common infections, penicillin allergies, and skin and soft-tissue infections.
Quality monitoring
On a quarterly basis, each stewardship pharmacist was sent data on the number of tele-PAF recommendations that had been made, the type of recommendations made, and the percentage of recommendations that had been accepted. Data for these quarterly assessments were drawn from a weekly REDCap electronic survey completed by the stewardship pharmacists.
RE-AIM outcomes
Reach. Reach was measured by tracking, on a weekly basis, the number of recommendations that were made and accepted using the aforementioned electronic survey as well as the names of different prescribers who received feedback.
Effectiveness
The primary outcome of effectiveness was inpatient days of antibiotic therapy (DOT) per 1,000 days present. Reference Barlam, Cosgrove and Abbo6 Secondary outcomes were days of inpatient antibiotic-spectrum coverage (DASC) per 1,000 days present and postdischarge DOT per 100 acute-care discharges. Reference Kakiuchi, Livorsi and Perencevich20,Reference Feller, Lund and Perencevich21 DASC is a novel metric that combines both antibiotic consumption and spectrum together. Both inpatient DOT and inpatient DASC were aggregated across all 3 acute-care units and separately across all 3 LTCs monthly. Data on antibiotic use, days present, and discharges were extracted from the Corporate Data Warehouse (CDW) using the VA Informatics and Computing Infrastructure (VINCI). Inpatient antibiotic use was collected from the barcoding medication administration system, and postdischarge antibiotics were collected from the outpatient medication files. Antibiotics included all antibacterial agents administered via the following routes as defined by the National Healthcare Safety Network (NHSN): intravenous, intramuscular, digestive tract (eg, oral), or respiratory tract. 22
Safety outcomes included acute-care, risk-adjusted, length of stay and all-cause, hospitalwide, 30-day readmission rate. These patient-centered outcomes were extracted from the Strategic Analytics for Improvement and Learning (SAIL) quarterly reports for each site. 23
Adoption
Adoption was primarily measured during the preintervention virtual site visits.
Implementation
Implementation was measured in several ways. First, the weekly surveys were used to monitor fidelity and cost (ie, time commitment) to the project. One stewardship call per month at each site was devoted to checking on the implementation process through an iterative technique called periodic reflections. Reference Finley, Huynh and Farmer24 Our team’s qualitative analysts (S.H.S. and C.C.G.) attended these regularly scheduled calls to identify and discuss implementation barriers and facilitators. All calls were recorded and transcribed, and systematic notes were documented. Acceptability and adaptation as well as other implementation measures were collected through semistructured qualitative interviews with 20 key stakeholders after the 12-month tele-PAF intervention ended. Interviewees include 7 pharmacists, 5 nurse practitioners, 5 hospitalists, 2 medical directors, and the remote ID physician (Supplementary Material online).
Maintenance
Maintenance was primarily evaluated by sending each stewardship pharmacist or their supervisor a short e-mail with targeted questions six months after the tele-PAF intervention ended.
Data analysis
To assess our primary and secondary effectiveness outcomes, we performed an interrupted times-series analysis using autoregressive integrated moving average (ARIMA) models. Reference Schaffer, Dobbins and Pearson25 To identify the ARIMA model, a stepwise automatic forecasting algorithm was implemented in the forecast package for R version 4.2.1 statistical software. Reference Hyndman and Khandakar26
Interview data were analyzed using MAXQDA, a qualitative data program (VERBI Software, Berlin, Germany). We performed thematic content analysis via consensus-based inductive and deductive coding. Reference Pope and Mays27
The study consent process and procedures were approved by the University of Iowa Institutional Review Board (no. 202108047) and Research and Development Committee at the Iowa City VAMC.
Results
Reach
During the 12-month intervention period, the tele-PAF process reviewed 502 unique patients and made 681 recommendations to 24 providers across the 3 sites. These 24 providers accounted for nearly all inpatient providers prescribing antibiotics during daytime, weekday hours. Off-hour providers and nurses were generally not reached and were typically entirely unaware of the project.
Overall, 190 recommendations were made at site 1 to 8 unique providers, 380 recommendations were made at site 2 to 12 different providers, and 111 recommendations were made at site 3 to 4 providers. Seventy-seven percent of recommendations were accepted, but the overall frequency of acceptance varied across sites (site 1, 78%; site 2, 81%; and site 3, 60%). As shown in Figure 1, the most common recommendations were to stop antibiotics (28%) and change antibiotic duration (19%). Antibiotic escalation was recommended in 14% of cases.
Figure 1. Types of recommendations made during the prospective audit-and-feedback process across the 3 participating VA medical centers, 2021 (n = 681).
Postintervention interviews indicated that the LTC providers were generally more receptive to feedback than acute-care providers. The LTCs had a more stable group of providers, typically nurse practitioners, who followed a cohort of patients longitudinally and, as a result, had a greater familiarity with their patients’ history. In contrast, “with the rotating schedule of the hospitalists [in acute care], it was a little bit more challenging when it was someone else who started it [the antibiotic] and recommending someone other than the lead attending to stop it” (pharmacist, site 1).
Effectiveness
Time-series data on antibiotic use are shown in Table 2 and Figures 2, 3 and 4. After the start of the program, antibiotic DOT immediately decreased in LTC units (−26 DOT per 1,000 days present, or 30% decrease; 95% CI, 13%– 47%; P = .01) and then remained stable during the rest of the intervention period. There was no immediate change in antibiotic DOT in acute-care units (+82 DOT, or 16% increase; 95% CI, −10% to 42%; P = .22) and no change in slope during the 12-month intervention period. Different trends in antibiotic use were seen among the different sites (Supplementary Tables 1–3 and Supplementary Figs. 1–3 online).
Table 2. Changes in Antibiotic Use During the Baseline and Intervention Periods Across the 3 Participating Veterans’ Affairs Medical Centers
Note. DOT, days of therapy; DASC, days of antibiotic spectrum coverage; LTC, long-term care; SE, standard error.
Figure 2. Comparison of antibiotic days of therapy (DOT) per 1,000 days-present between the baseline and intervention periods across the 3 participating VA medical centers, 2019–2021.
Figure 3. Comparison of days of antibiotic spectrum coverage (DASC) per 1,000 days present between the baseline and intervention periods across the 3 participating VA medical centers, 2019–2021.
Figure 4. Comparison of post-discharge days of therapy (DOT) per 100 acute-care discharges between the baseline and intervention periods across the 3 participating VA medical centers, 2019–2021.
Antibiotic spectrum, as measured by DASC, also immediately decreased in LTC units (−304 DASC per 1,000 days present, or 37% decrease; P < .001) and then remained stable during the rest of the intervention period (Table 2 and Fig. 3). There were no significant changes in DASC in acute-care units. In addition, postdischarge DOT did not significantly decrease in acute-care units (Table 2 and Fig. 4).
Adjusted length of stay and adjusted 30-day readmission rates were stable or declined at each site during the intervention period (Supplementary Figs. 4 and 5 online). The volume of consultations with outside ID physicians also did not change (Supplementary Table 4 online).
Adoption
The 3 initial sites that were eligible and invited all agreed to participate. All 3 sites requested the intervention to be implemented in both their acute-care and LTC units. The primary motivation for adoption was the lack of on-site ID support (Table 3). According to the chief of medicine at site 3, “We’re looking at this as an opportunity to get better personally, as well as benefit the veterans by preventing unnecessary use of antibiotics for prolonged durations.”
Table 3. Sample Quotations from Semistructured Interviews With Hospital Leaders, Pharmacists and Providers Involved in the Telehealth-Supported Antibiotic Stewardship Program Organized by the 5 Key Outcomes of the RE-AIM Framework
Implementation
Fidelity. Intervention audits were consistently performed 3 times per week at each site, except when tele-PAF team members were out of the office. However, the method for providing feedback at times varied based on the intended recipient. According to the stewardship pharmacist at site 1, “When I was speaking with [the ID physician] about a case, knowing who the provider was would often shape how we decided to approach it.” Because some physicians did not want to accept recommendations from the pharmacist, a 3-way telephone call was sometimes used to incorporate the remote ID physician into the conversation as well (Table 3). Certain providers were more difficult to reach, so the mode of communicating feedback had to be tailored to their work style: “It was kind of a challenge to get a hold of some of our providers. I think towards the end of the project we had a decent understanding… of more effective ways to get in touch with people” (pharmacist, site 3).
Acceptability
Providers generally appreciated feedback and responded favorably to collaborative discussions. According to a hospitalist at site 1, “I think it [the project] was very helpful and well received. If we could continue with that [the project], we would…. Having an ID consultant give some recommendations on the length of therapy and scaling up or down therapy is very helpful.”
The acceptability of the project was facilitated by the collaborative communication style of the ID physician (Table 3). Providers identified 2 important perceived barriers to acceptability: (1) the project’s inability to perform bedside assessments and (2) some physicians’ reluctance to act on recommendations delivered by a clinical pharmacist. Both barriers were alleviated by good communication between the providers and the ID physician (Table 3).
Costs
The remote ID physician devoted an average of 3.9 hours per week to tele-PAF. The mean stewardship pharmacist time commitment per week varied by site (site 1: 3.4 hours; site 2: 2.6 hours; site 3: 1.5 hours). Pharmacists at sites 1 and 2 indicated that this project was easy to incorporate into their other work responsibilities. At site 3, the pharmacist was “really involved with covering other areas due to staff leaving and us being short.” This pharmacist acknowledged that “pre-COVID, it [the project] absolutely would have been more sustainable.”
Maintenance
At 6 months after the project ended, none of the sites had established a contractual relationship with a remote ID physician to continue supporting stewardship activities. Sites 1 and 2 had enrolled in a VA-funded research project to further evaluate the implementation of telehealth to provide remote ID physician support. It is unclear how much their experience with tele-PAF influenced these two sites’ decision to enroll in this other project.
Although tele-PAF ended in December 2021, some benefits of the project may have been sustained (Table 3).
Discussion
The implementation of our telehealth stewardship project was associated with reductions in antibiotic use across 3 LTC units but not in the smaller acute-care units. Overall, providers perceived the project as acceptable, and project activities were workload appropriate for stewardship pharmacists at 2 of the 3 sites.
Our telehealth project had a different effect on antibiotic use in acute-care units versus LTC units. The immediate decrease in antibiotic use in LTC units could reflect the success of engaging LTC unit stakeholders in the preimplementation phase and the ease at developing rapport with these providers once tele-PAF started. The providers in the LTC units worked every weekday and followed their patients longitudinally. In contrast, there was more frequent rotation of providers in acute-care units, which may have been a barrier to building trust. In addition, because antibiotic use was already decreasing in the acute-care units during the baseline period, there may have been fewer opportunities for improved prescribing in these settings during the intervention.
One novel aspect of our project was the use of the RE-AIM framework, which helped identify relevant outcomes to measure for program evaluation. In general, research on implementing antibiotic stewardship processes should make use of an implementation science framework. Reference Livorsi, Drainoni and Reisinger28 These frameworks share a common language and help more efficiently contribute to generalized knowledge about how to change behavior. The RE-AIM framework is specifically designed to evaluate health interventions, such as our telehealth project. Our use of the RE-AIM framework prompted our team to evaluate outcomes we may not have otherwise considered, such as maintenance and certain aspects of implementation. Through our implementation assessments, we gained a better understanding of how the project was adapted to meet local needs and why it was acceptable to some providers but not others. Interestingly, the barriers and facilitators to acceptability we identified have also been described in studies of stewardship performed without telehealth. Reference Barlam, Childs and Zieminski29–Reference Perozziello, Routelous and Charani31
Our project was not without limitations. First, our intervention only lasted a year, and it is unclear whether a longer intervention would have achieved larger reductions in antibiotic use. In 2 published studies, telehealth-supported antibiotic stewardship programs that lasted >2 years were associated with reductions in antibiotic use. Reference Moehring, Yarrington and Davis32,Reference Khadem, Nguyen, Mellors and Bariola33 The effectiveness of these programs requires developing trust and rapport between remote ID specialists and local team members, and it is possible that 1 year was insufficient for accomplishing this. Second, while all the sites were generally positive about the project, it is unclear whether the perceived value was large enough for each hospital to make a long-term funding commitment to ID telehealth. This is because sites were invited to participate without cost in a very similar telehealth project that was set to begin after the current project ended. Third, we were only able to interview a proportion of providers who received feedback as part of our tele-PAF process, and it is possible that certain perspectives about acceptability were not captured. Fourth, our analysis did not include an assessment of interfacility transfers or cost savings. Fifth, it is unclear how the COVID-19 pandemic affected our findings. Patterns of hospital staffing, antibiotic prescribing and bed use during the pandemic may have differed from the prepandemic period, which could have affected the findings of our interrupted time-series analysis. To mitigate this concern, our baseline period was 2 years in duration, roughly half of which fell during the pandemic period. Finally, our telehealth project occurred within a unique integrated healthcare system and included a remote ID physician with protected time to dedicate to this project. Although telehealth-supported stewardship can be successful under different circumstances, the implementation may be more difficult. Reference Livorsi, Abdel-Massih and Crnich34
In conclusion, using telehealth, a single, remote, ID physician was able to work effectively with 3 geographically diverse VAMCs to provide stewardship support for patients in both acute-care beds and LTC beds. Wider implementation of telehealth-supported stewardship activities may achieve further reductions in antibiotic use.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ice.2023.81
Acknowledgments
We thank Tammy Walkner, PhD, for her support in the preparation of this manuscript. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans’ Affairs or the United States government.
Financial support
This work was supported by the Veterans’ Health Administration, Office of Rural Health, Veterans’ Rural Health Resource Center Iowa City (grant no. 03611) and by a Career Development Award (D.J.L.) from the VA Health Services Research and Development Service (grant no. CDA 16-204).
Competing interests
All other authors report no conflicts of interest.
Open access
