Operationalizing an adverse event detection surveillance system to support antimicrobial stewardship activities: perceptions and insights from the SHEA research network

Abstract

A surveillance system for measuring patient-level antimicrobial adverse drug events (ADE) may support stewardship activities, however, design and implementation questions remain. In this national survey, stewardship experts favored simple, laboratory-based ADE definitions although there were tensions between feasibility, ability to identify attribution without chart review, and importance of specific ADE.

Introduction

Antimicrobial-attributed adverse drug events (ADE) are estimated to impact an estimated 20% of hospitalized patients receiving these medications. The overwhelming majority (97%) of these ADE are considered clinically significant, leading to either additional testing or extended time in healthcare system encounters. ¹ However, despite their frequency and recognized clinical importance, stewardship teams tend to rely on other metrics, such as total use and cost, for surveillance and monitoring activities. ²

Metrics that reflect patient-level ADE may be useful for advancing antimicrobial stewardship program (ASP) goals. However, despite their potential for facilitating practice improvement, there are limited antibiotic ADE metrics widely in use other than hospital-acquired Clostridioides difficile infection (CDI). Limited uptake of this strategy may be due to challenges with defining and operationalizing antimicrobial harm surveillance tools. Each drug has different toxicity profiles, hospitals have patient populations with varying complexities and diagnoses (e.g. transplant, oncology), with different rates of underlying antimicrobial resistance (AMR). Therefore, attribution of specific ADEs is challenging, and it is unclear how different levels of toxicity should be weighted and reported. Varied resources and analytics infrastructure across institutions also create implementation and dissemination challenges. ³ Thus, the aim of this study is to explore the perceptions of ASP leaders about the utility and feasibility of creating an antimicrobial ADE surveillance system and to identify strategies to inform development and future implementation.

Methods

We conducted a survey of antimicrobial stewards using the Society for Healthcare Epidemiology of America Research Network (SRN). ⁴ All SRN sites with an ASP were invited to participate. Instructions were provided to emphasize the preference for the medical or pharmacy director of stewardship or the facility stewardship champion as the designated survey respondent.

The 30-question survey collected demographic information and perceptions about the utility and feasibility of constructing an antimicrobial harm surveillance system designed for inpatient settings. This included input about potential for electronic data pulls and reporting and about challenges associated with attributing specific ADEs to antimicrobial exposures as well as candidate ADE definitions. See Supplementary Materials for full details of the survey questions including candidate ADE definitions.

The survey was primarily comprised of multiple-choice questions, ranking grid questions, and Likert-scale-based queries. Throughout the survey, there were free-text sections where participants could provide qualitative responses to elaborate on the topic or their answer choices. To optimize the comprehensibility and effectiveness of the survey, a trial administration was conducted with two local ASP physicians who did not contribute to the initial survey’s design, and their feedback was incorporated into an updated data collection instrument. The survey was also reviewed and approved by the SRN prior to data collection and analysis. It was conducted online over a 6-week period from 6/2/2023 to 7/14/23 and was distributed using the Alchemer platform.

Statistical testing and analysis

Simple descriptive and univariate statistics were used for the majority of analysis. For ranking grid questions, a score was calculated to assign overall rank; the score is a weighted calculation that accounts for number of times an item was selected and the ranking level, with items ranked first given a larger “weight.” The final ranked score, computed for each answer option, is the sum of all the weighted values. All data were analyzed using Statistical Analysis System. Free-text responses were organized into categories and were presented qualitatively.

This project received an exempt designation from the VA Boston Research and Development Committee.

Results

The survey was distributed to 114 eligible SRN sites including 94 US-based institutions. The overall response rate was 32.7%. Respondents were from geographically diverse US institutions (See Supplementary Materials for full survey response). Most respondents represented academic hospitals (69%), followed by community hospitals with academic affiliations (13.8%). A variety of facility sizes were represented, ranging from <100 beds to >500 beds, with the majority falling into the >500-bed category (48.6%). Types of IT support varied with a majority utilizing local Electronic Medical Record linked systems (69.4%). Half of the respondents reported having an analytical surveillance software (50%) (Supplementary Table 1).

ASP respondents identified that an antimicrobial ADE surveillance system could be used to support communication with clinicians to de-prescribe or de-escalate antibiotics, to monitor trends of ADEs hospital-wide or system-wide, and to monitor the impact of local stewardship interventions (Supplementary Figure 1). Drug-specific ADE (e.g. linezolid associated thrombocytopenia) and future AMR in the individual patient were ranked as “most often discussed” in ASP daily practice by 40% and 30% of respondents, respectively (Supplementary Figure 2).

ASP respondents favored organizing ADE “by antibiotic type” (e.g. cefepime-related ADEs) as this reporting system had the highest overall ranking and was ranked first (most useful and feasible) 15/28 times (Supplementary Figure 3). Structuring harms “by organ system affected” (e.g. hematology ADEs) ranked second, and “one all-encompassing metric” (includes the rate of all types of antibiotic ADEs from all antibiotics) ranked third. Participant-preferred strategies for integrating severe and rare ADEs, such as Stevens Johnson Syndrome (SJS), are summarized in Figure 1. Approximately half of the respondents (50%) expressed a preference for using a differential weighting system utilizing Quality-Adjusted Life Years, where the more severe the ADE, the more it contributes to the metric.

Figure 1. Respondents’ preferred approach towards addressing severe and rare adverse events (ADE) in a metric. Panel (A) Severe ADE. Panel (B) Rare ADE. Using two separate questions, respondents were asked to consider an approach to addressing severe ADE and rare ADE in the context of an antibiotic harm metric. They were able to select only one option out of those presented. Rare ADE were defined as occurring at a rate of <0.1%. *The full answer choice: “assign differential weighting based on severity (using Quality-Adjusted Life-Year “QALY”) and increase the weighting of a more severe consequence (e.g., a case of mild hives would score 1 whereas a case of SJS would score 4).”

Participant feedback about four proposed ADE surveillance definitions (dermatological, hematological, drug-associated liver injury, and AMR rate, see Supplementary Materials for full details) reflect that AMR was scored as most useful, feasible, and most impactful on patient outcomes; responses about the feasibility of AMR reporting was correlated with facility size (p-value, 0.0179). After AMR, liver injury had the highest score in feasibility on a 1–9 Likert scale (median score of 6). Key themes that emerged in the qualitative responses about the four candidates ADE definitions included concerns about subjectivity and reporting of some of the proposed harms, such as dermatologic ADEs, which cannot be easily captured using simple electronic surveillance tools (Table 1). Respondents suggested other potential ADE that could be considered for inclusion, such as Acute Kidney Injury, CDI, diarrhea, and QTc prolongation.

Table 1. Summary of qualitative responses on selected adverse drug events (ADE) in a proposed harm surveillance system

Respondents Suggestions for Additional ADE to be considered for a Harm Surveillance System: Nephrotoxicity, Clostridioides difficile infections, diarrhea, QTc prolongation, phlebitis, dysbiosis, central nervous system toxicities, tendinopathies, drug-specific harms (e.g., daptomycin and eosinophilic pneumonia).

Discussion

Although 20% of hospitalized patients experience an antibiotic-associated ADE, there is a gap in tracking and reporting the direct harms of antibiotics to ASPs and clinical providers. ¹ In this study, we obtained antibiotic stewards’ perceptions on the utility, feasibility, and possible structure of an antibiotic ADE surveillance system to fill this information gap. Prior work shows that prescribers prioritized immediate patient risks over long-term population risks such as AMR, which were perceived by frontline clinicians as more abstract and a less imminent threat. ^5,6 Focusing antimicrobial discontinuation efforts on patient-level harms may support stewardship efforts by aligning the long-term goals of ASP with those of frontline clinicians.

Antibiotic ADEs constitute a wide spectrum of toxicities with varying incidence and severity and therefore could be measured and structured in several ways. For example, ADEs could be grouped by antibiotic type or by organ system affected or both. Respondents’ preference for organization of ADE by specific drug likely reflects a desire for simplicity over complexity. Major challenges in operationalizing a drug-based surveillance system include the diversity of drugs involved, the sheer number of potential toxicities, challenges with electronic measurement, and difficulties attributing specific events to specific drugs, particularly in the setting of comorbidities and polypharmacy. Despite these barriers, a drug-based system may be perceived to be more practical as it could utilize existing data infrastructure, which frequently tracks use by type.

The respondents’ emphasis on incorporating ADE severity was also reflected in the free-text results, although there were clear tensions between implementation feasibility and clinical importance and significance. Among the three proposed definitions, two of which were laboratory-based, the dermatological reactions definition (which included SJS) had the lowest median feasibility score, despite other responses indicating the importance of identifying severe and life-threatening ADE. How information will be used is an important consideration in designing surveillance tools. ICD-10 codes could be combined with laboratory or other measures to create standardized variables. However, these systems are inherently delayed and thus would be unlikely to be available for supporting point-of-care decisions. Additionally using a general adverse event identification system such as the Institute for Healthcare Improvement Trigger tool would reduce the need to develop large numbers of individual definitions, however, the trade-off of the general ADE surveillance approach is that manual review would be necessary to determine the type and severity of the ADE.

While not statistically significant, the median score for local AMR was higher, compared to the three other ADEs when it came to utility, feasibility, and patient-level impact. This may be because rates of AMR are felt to be more directly attributable to antimicrobial exposure than drug toxicities. However, AMR is primarily a population-level risk, rather than a patient-level risk. The perceived greater importance of local AMR rates in a harm metric conflicts with the implied emphasis of the ADE metric on direct patient outcomes. Interestingly, it also contrasts with how the majority of respondents chose “direct communication with clinician” in daily stewardship activities as the objective of the surveillance tool when prior work indicated that AMR is not a considerable priority in day-to-day clinician decision-making. ⁷ This preference for local AMR rates may reflect the pervasiveness of population-level AMR as the traditional focus of ASPs.

To our knowledge, this is the first study to capture the perceptions of antimicrobial stewardship practitioners on establishing an antibiotic ADE surveillance system. The respondents represent an expert panel that would be the expected to adapt and implement ADE surveillance tools. Limitations include small sample size and a survey-based format that might lack the depth of focus group discussions and introduce the possibility of response bias. The respondents were limited to U.S. institutions, predominantly those with an academic background and some data analytics infrastructure, and thus perceptions may not be reflective of hospitals with less IT support and more limited access to Electronic Health Records. An additional limitation is that the scope of the study did not encompass surveying frontline prescribers who are key stakeholders in stewardship initiatives.

Additional research is needed to develop and implement an antibiotic ADE metric. Some of the key challenges include formulating standardized definitions for ADEs that acknowledge differences in event timing. Furthermore, it is important to devise a practical system for identifying ADE and for appropriately attributing ADEs to antibiotic use. Next steps could involve using the valuable insights provided by the respondents in our study to develop a model of an antimicrobial ADE surveillance system and solicit feedback from both antimicrobial stewards and frontline prescribers.

In conclusion, this study provides important pre-implementation data to inform the development and testing of surveillance tools within antimicrobial stewardship that directly measure patient-level ADE. A successful ADE measurement system should feature standardized definitions that allow for accurate attribution of ADEs to antimicrobials while considering the severity of the events. Additionally, it is essential to balance this accuracy with feasibility and equity, ensuring operationalization of the tool minimizes the need for additional resources and manual chart review that may not be universally available or accessible.