Development and implementation of the Antimicrobial Stewardship Intervention Scoring Tool at a single pediatric institution

Abstract

Objective:

The primary objective was to grade the potential impact of antimicrobial stewardship program (ASP) interventions on patient safety at a single center using a newly developed scoring tool, the Antimicrobial Stewardship Impact Scoring Tool (ASIST).

Design:

Retrospective descriptive study.

Setting:

A 367-bed free-standing, pediatric academic medical center.

Methods:

The ASP team developed the ASIST which scored each intervention on an impact level (low, moderate, high) based on patient harm avoidance and degree of antibiotic optimization. Intervention frequency and characteristics were collected between May 1, 2022 and October 31, 2023. Intervention rates per impact level were calculated monthly.

Results:

The ASP team made 1024 interventions further classified as low (45.1%), moderate (47%), and high impact (7.9%). The interventions for general pediatrics (53.9%) and those to modify formulation (62.2%), dose/frequency (58.1%), and duration (57.5%) were frequently low impact. Hematology/oncology (12.5%), sub-specialty (11.7%), and surgical services (11.3%) had the greatest rate of high-impact interventions. Interventions to broaden antibiotics (40.8%) and those associated with antibiotics used to treat bacteremia (20.6%) were frequently classified as high-impact.

Conclusion:

The ASIST is an effective tool to link ASP interventions to prevention of antimicrobial-associated patient harm. For our ASP team, it provided meaningful data to present to hospital leadership and identified opportunities to prevent future harm and reduce ASP team workload.

Background

Antimicrobials are one of the most frequent medications resulting in adverse drug reactions (ADR), and use of antimicrobials, both appropriately and inappropriately, has been associated with an increased risk of patient harm. ^1–4 In a single-center retrospective study of hospitalized pediatric patients receiving antimicrobials, 21% experienced an ADR, and each additional day of antimicrobials was associated with a 7% increased risk. ⁵ Antimicrobial stewardship programs (ASPs) are closely tied to patient safety as the overall goal is to “to optimize clinical outcomes while minimizing unintended consequences of antimicrobial use.” ⁶ ASP interventions in the inpatient setting have decreased inappropriate antimicrobial use, Clostridioides difficile infection rates, and other variables. ^7–10

A key component of ASPs has been ongoing monitoring of various metrics. ^11–13 In the “The Core Elements of Hospital Antibiotic Stewardship Programs, 2019,” the Centers for Disease Control and Prevention (CDC) recommends following process metrics (eg, antibiotic utilization) and outcomes (eg, antibiotic resistance rates). ¹¹ While effective at displaying program progress, these metrics do not adequately depict the direct impact of the ASP on patient safety, which is an important metric for both hospital leadership who are charged with resource allocation and frontline teams who are responsible for implementing change. There is a need for an objective, quantifiable measure to assess antimicrobial-associated patient harm avoided due to ASP intervention. This study describes the development of a novel metric—the Antimicrobial Stewardship Impact Scoring Tool (ASIST)—which associates each inpatient ASP prospective audit and feedback (PAF) intervention with a level (high, moderate, low impact) of patient harm prevention and antimicrobial optimization. The primary objective was to grade the potential impact of ASP PAF interventions on patient safety at a single center using the ASIST.

Methods

Study setting

Children’s Mercy Kansas City (CMKC) is a 367-bed free-standing, pediatric academic medical center that provides comprehensive primary and tertiary care for a 5-state, 100-county region. During the study period, the ASP team consisted of two infectious diseases (ID) clinical pharmacy specialists, five ID physicians, one advanced practice provider (APP), one nurse with infection prevention and clinical safety experience, and one data analyst. A hospital wide PAF ASP was initiated on March 3, 2008 targeting inpatients receiving antibiotics without an active ID consult. An ID pharmacist, physician, or the APP performed a daily review of inpatients receiving antibiotics for two consecutive calendar days. During the weekdays, when an ID pharmacist was responsible for PAF review, an ID physician assisted in identifying and providing interventions. The ASP team provided interventions to the primary medical team when indicated and utilized standardized intervention categories: ID consult, narrow antibiotics, broaden antibiotics, modify formulation (ex. intravenous (IV) to oral (PO) transition), optimize duration, modify dose/frequency, consolidate antibiotics, and stop antibiotics. The ASP team also collected whether interventions were accepted or rejected within 24 hours of intervention. This information was documented by the ASP team member performing the review in an ASP intervention form within the electronic health record (EHR). The ASP team tracked various program metrics (days of therapy (DOT) per 1,000 patient days, intervention characteristics, etc) monthly and reported regularly to various leadership groups within the institution.

Impact score design and implementation

In September 2021, the ASP team began developing the ASIST to quantify patient harm avoidance and degree of antibiotic optimization with PAF interventions. The ASP team created a multidisciplinary subgroup including ID/ASP pharmacists, an ID/ASP physician, and a nurse to develop the ASIST. We modeled the ASIST loosely after the American Society of Healthcare Risk Management Patient Safety Event Classification which includes three safety event classes depending on whether the error reaches the patient (Serious Safety Event, Safety Event, Pre-Patient Event) and the level of harm occurring. ¹⁴ Similarly, the group defined three levels of impact: low, moderate, and high for the ASIST. With low-impact interventions, antimicrobial-associated patient harm was unlikely to occur, but opportunities existed for minor antibiotic optimization. Moderate interventions had substantial room for antibiotic optimization but still were thought to have low risk for antimicrobial-associated patient harm. High-impact interventions carried substantial risk of antimicrobial-associated patient harm due to high probability of a severe adverse effect or due to poor outcomes from an inappropriate regimen as documented in evidence-based guidelines or high-quality literature. A severe adverse effect included situations that could contribute to patient death or temporary/permanent harm such as organ dysfunction, prolonged hospitalization, or medical/surgical intervention. ¹⁴ Using these principles and examples of interventions made during daily ASP PAF rounds, definitions for each level of impact were created for all the standardized ASP intervention categories (Table 1). For the “stop antibiotics” intervention category, the ASP team classified specific antibiotics as narrow- or broad-spectrum utilizing a previously described rank system created by the Duke Antimicrobial Stewardship Outreach Network. ¹⁵ (Supplemental Table 1) Examples of interventions within each impact category is found in the supplemental information (Supplemental Table 2).

Table 1. Antimicrobial Stewardship Impact Scoring Tool (ASIST)

Note. IV, intravenous; PO, oral; PIV, peripheral intravenous line; CVL, central venous line.

In November 2021, the ASP team began piloting the ASIST during daily PAF rounds. ASP team members were trained on the tool, and each intervention was independently scored by the ID pharmacist or APP and ID physician conducting ASP PAF that day. Discrepancies and interventions which did not cleanly fall into a distinct category were identified and reviewed monthly by the ASP subgroup to inform modifications. The ASP subgroup met monthly to evaluate these instances and modified the tool as necessary. The tool was finalized in April 2022.

The impact level for each intervention was recorded in the EHR within our existing ASP intervention documentation. High-impact interventions were discussed at the ASP multidisciplinary huddle to identify any systematic interventions to implement to prevent possible antimicrobial-associated patient harm. Numbers of low-, moderate-, and high-impact interventions were included in ASP data sharing with various hospital committees.

Study design and end points

The institutional review board reviewed and approved this study. The primary objective was to grade the potential impact of ASP PAF interventions on patient safety at a single center using the ASIST. Data were extracted from the EHR between May 1, 2022 and October 31, 2023 on a monthly basis. The absolute number and frequency of all ASP PAF interventions as well as those classified as high, moderate, or low impact were collected for each month. Details such as patient age, patient sex, primary medical service, antibiotic indication, intervention category, and intervention acceptance were obtained from ASP documentation in the EHR.

Results

Between May 1, 2022 and October 31, 2023, the ASP team made 1024 interventions with an average of 57 interventions provided each month (range: 36–84). The proportion of all interventions classified as low-, moderate-, or high-impact differed by month (Figure 1). On average, five high-impact interventions were provided each month (range 2–10).

Figure 1. Monthly frequency of low-, moderate-, and high-impact antimicrobial stewardship interventions.

The vast majority of all ASP interventions were classified as low (45.1%) or moderate impact (47%) (Table 2). Intervention categories with the largest proportion of low-impact interventions included modify formulation (62.2%), modify dose/frequency (58.1%), and optimize duration (57.5%). The general pediatrics service had the greatest proportion of low-impact interventions (53.9%). The intervention categories of consolidate antibiotics (61.1%), stop antibiotics (56.9%), and ID consult (55.4%) were frequently classified as moderate impact. A smaller proportion of interventions (7.9%) were classified as high-impact. While the general pediatrics, PICU, and NICU services had the greatest absolute number of high-impact interventions, the greatest rate was observed with the hematology/oncology (12.5%), sub-specialty (11.7%), and surgical services (11.3%). Interventions to broaden antibiotics (40.8%) and those associated with antibiotics used to treat bacteremia (20.6%) were frequently classified as high-impact. The antibiotics with the highest rates of high-impact interventions included aminoglycosides (12.5%), vancomycin (10.3%), and fluoroquinolones (9.7%). Primary care teams accepted 91.4% of high-impact interventions.

Table 2. Characteristics of antimicrobial stewardship program interventions by impact level

^a Proportions reflect the percentage of overall interventions with a denominator of 1024 total interventions

^b Proportions are the percentage of low-, moderate-, and high-impact interventions composing the total number of interventions in each row

^c Multiple antibiotics could be associated with a single intervention

Discussion

This report describes the successful development and implementation of a novel scoring tool to associate each ASP intervention with a level (low-, moderate-, high-impact) of patient harm prevention and antimicrobial optimization. While the majority of the ASP interventions were considered low- or moderate-impact, high-impact interventions occurred consistently. Traditional metrics fail to capture ASP impact on patient safety; a measure such as the ASIST is needed to better describe impact and inform program agenda. Enumerating high-impact interventions quantifies the prevention of significant harm by the ASP and can help to prevent future antimicrobial-related safety events. At the same time, low-impact interventions may identify opportunities for automation, protocol development, and prioritization of program resources.

Traditional ASP measures have focused on four primary areas: antimicrobial utilization, program process, clinical or microbiological outcomes, and financial impact. ¹⁶ These metrics focus on cost and consumption of antimicrobials and fail to capture instances where ASP interventions impacted patient safety, especially in the pediatric setting. ¹⁷ In the literature, various ASPs have developed subjective systems to recognize “great catches” where the ASP intervention changed the trajectory of patient care. ⁷ The ASIST was designed as an objective measure to be utilized by our ASP team members and eventually expanded across multiple healthcare settings. The tool allows ASPs to capture not only the quantity of interventions but also the qualitative aspect. Our ASP team identifies that the ASIST is easy to use and adds limited time to daily ASP documentation. As the ASIST was loosely modeled after the American Society of Healthcare Risk Management Patient Safety Event Classification, its terminology is something familiar to hospital leadership. Similar to event reporting systems, as a program we have used the ASIST to identify trends and put forth interventions to prevent future harm from antimicrobials. ¹⁴ We share the distribution of high-, moderate-, and low-impact interventions with hospital leadership allowing us to have further conversations about the value of ASP involvement in patient safety.

High-impact interventions occurred almost weekly. These were almost always followed by the primary medical team, further justifying the need for ASP PAF reviews. For example, high-impact interventions occurred frequently with vancomycin, aminoglycosides, and fluoroquinolones which have high rates of ADRs. ^18,19 A large proportion of ASP interventions for hematology/oncology, sub-specialty, and surgical services were considered high-impact. This is likely because these patients are often more complex, have higher risk for error or ADRs, or utilize broad-spectrum antimicrobials more frequently making treatment more challenging. ^20–23 The increased risk of high-impact interventions for these services highlights the impact the ASP may have in complex patients. Additionally, ASP team members made many high-impact interventions related to broadening antimicrobials. Upon review, many of these interventions were related to inappropriate use of third generation cephalosporins to treat Enterobacterales at moderate to high risk for clinically significant AmpC production, which was a deviation from national guideline recommendations. ²⁴ Therefore, we worked with our microbiology laboratory to suppress third generation cephalosporin susceptibilities on specific isolate types (eg, blood cultures) for these organisms and provided education to specific units on appropriate treatment.

In contrast, low-impact interventions occurred consistently and in certain months accounted for majority of the ASP interventions. With increasing demand for ASPs, recent publications highlight the need to shift the responsibility of appropriate antimicrobial use away from the ASP and to the frontline clinician thus limiting the need for ASP oversight. ²⁵ Low-impact interventions may provide opportunity to do so. At our institution, updating antimicrobial order sentences to include preset durations for specific indications (eg, 5 days for pneumonia) or implementing automatic IV to PO transitions for certain antimicrobials may reduce number of low-impact interventions. As a program, we could leverage primary team pharmacist involvement to make low-impact interventions, team pharmacist participation in ASPs has been successful in expanding stewardship reach. ^26,27

A limitation of this study is that the ASIST was developed by and utilized only at our institution. It is unknown whether the tool would be applicable to other ASPs at other centers and whether similar results would be obtained. However, a multi-center collaborative is currently underway to gain consensus on the scoring tool. While we aimed for the tool to be as objective as possible, the tool was not formally validated. Our program has multiple ASP team members, and it is possible that there would be differences in coding of interventions.

ASPs are a vital component to prevent antimicrobial-associated patient harm and new metrics are needed to better depict the association of ASPs and patient safety. ⁶ We report the successful implementation of a novel tool, the ASIST, to score ASP interventions on the degree of antimicrobial optimization and impact on harm prevention. The ASIST tool was helpful as a method to report ASP safety data to hospital leadership. As a program, we continue to review high-impact interventions to identify processes to implement to prevent future harm as well as review trends in low-impact interventions to potentially reduce ASP team workload. Future directions include approval and expansion of the ASIST outside of our organization.