Characteristics and framework analysis of Department of Defense hospital Antibiotic Stewardship Programs guided by the Centers for Disease Control and Prevention Core Elements

Abstract

Objective:

Characterization and assessment of Department of Defense’s (DoD’s) Antibiotic Stewardship Programs (ASPs) to determine adherence to Centers for Disease Control and Prevention (CDC) Core Elements and compare to national adherence

Design:

Retrospective, observational with supplemental survey

Methods:

Facility characteristics and CDC Core Elements (CE) adherence data for 2017–2021 were retrieved from the National Healthcare Safety Network’s (NHSN) annual hospital survey with DoD data from the Defense Health Agency and national data from the Antibiotic Resistance and Patient Safety Portal. An online supplemental survey was administered to DoD hospitals. Descriptive statistics and bivariate analyses were completed for facility characteristics and supplemental survey questions to determine correlations between variables. A framework analysis compared DoD ASPs to CEs and Priority Elements.

Results:

Supplemental surveys were completed for 85.1% of DoD’s hospitals. DoD’s hospitals were smaller on average than national hospitals. ASP leaders were assigned more often than volunteer and typically served in the role for less than four years. Staffing mix differed, with more equivalent proportions of civilian/contractor to military at larger hospitals in the U.S. Most DoD ASPs consisted of ≤ 25% pharmacists. ASP leaders were largely available on a daily basis; pharmacist leaders spent more time on ASP activities than physicians. CE adherence was high, but in 2021 DoD lagged national adherence in the structural CEs of Leadership, Accountability, and Pharmacy Expertise.

Conclusions:

DoD hospitals lagged in national adherence to the structural CEs, presenting opportunities for ASP improvement. Refinement of CE adherence measurements, coupled with impact on health outcomes, could aid in better-identifying areas for improvement.

Introduction

The Centers for Disease Control and Prevention (CDC) released its first guide for the Core Elements (CEs) of Hospital Antibiotic Stewardship Programs (ASPs) in 2014, recommending that all U.S. hospitals implement ASPs following a flexible framework of seven structural and procedural CEs.¹ The 2015 National Action Plan for Combating Antibiotic-Resistant Bacteria set a goal to have ASPs instituted in all hospitals that receive federal funding,² and the 2017 Department of Defense (DoD) Instruction echoed this with a requirement for DoD hospital ASPs to include the CEs.³ The CDC released Priority CEs in 2022 to further direct activities to the most effective approaches.⁴

DoD ASPs are monitored at the facility level, evaluating individual interventions,^5,6 and occasionally multicenter studies are conducted.⁷ DoD components collect enterprise-wide antibiotic use and resistance data but are focused on internal reporting. There has not been an enterprise-level ASP evaluation to determine if programs are set up as recommended by the CDC and directed by policy.

The objective of this study was to describe Military Health System (MHS) hospitals (which in this context includes all DoD hospitals) using facility characteristics and the CE framework to evaluate DoD ASPs. Results will be used to inform practice improvement and policy. The hypothesis for the CEs adherence portion was that DoD ASPs would follow 2020 national trends and be lagging most in Reporting and Education followed by Tracking, due to the DoD’s unique centralized data reporting structure to the National Healthcare Safety Network (NHSN).

Methods

Facility characteristics

Facility characteristics pertaining to all DoD hospitals from 2017 to 2021 were collected from NHSN’s Patient Safety Component annual hospital survey, a requirement for all reporting hospitals, usually completed by the infection prevention and control team.⁸ Facility characteristics were used to complete descriptive statistics for all DoD hospitals in 2021 and in bivariate analyses of facility characteristics to determine relationships that could impact CE adherence. Pearson’s correlation was used for pairs of continuous variables, with one continuous and one dichotomous variable t-test was used, and for a continuous variable and a categorical variable ANOVA was used. Fisher’s exact test was used for pairs where both variables had two or more categories. The α level was set at 0.05 for all tests of significance.

Supplemental survey

A supplemental survey (Appendix 1) was administered via SurveyMonkey after an email invitation to all DoD hospitals (96 ASP points of contact at 48 facilities). Points of contact were obtained from the Defense Health Agency’s Antimicrobial Stewardship Program Committee (ASPC) and through researcher calls/emails to facilities. The 47-question survey focused on the prior year and was based on the CDC’s ASP Assessment Tool,⁹ the Agency for Healthcare Research and Quality’s Gap Analysis Tool,¹⁰ and to garner specific military context, such if ASP leaders were assigned or volunteered and length of tenure. Some non-required NHSN annual survey questions were included in case of nonresponse on the NHSN survey; other questions were duplicative due to protocol approval prior to the 2021 NHSN survey update. The survey was estimated to take 30 minutes to complete and was reviewed by the ASPC. The invitation included a copy of the survey and asked that ASP teams collaborate to submit a single response per hospital. The survey opened on January 9, 2023 and closed on June 25, 2023. Iterative reminders and response follow-ups were done via email, and surveys were screened regularly. Descriptive statistics were completed for all survey questions. NHSN-based facility characteristics were compared for respondent facilities and all DoD hospitals to assess possible bias related to survey completion status (Table 2). Correlations between facility characteristics and survey responses (questions 1, 3–14) were assessed. NHSN-based facility characteristics that were compared in bivariate analyses included number of admissions, number of patient days, categorical number of beds, number of intensive care unit beds, medical type (graduate teaching, major teaching, or non-teaching), medical affiliation (teaching or non-teaching), facility located within the U.S. or not, U.S. region (hereafter Region), and military branch of service (hereafter Service). Major teaching facilities have a program for medical students and post-graduate medical training (residencies/fellowships), while graduate teaching facilities only have post-graduate training.⁸ Region was based on the U.S. Department of Health and Human Services regions,¹¹ with the exception of a hospital in Guam being removed from Region 9 and considered overseas.

Core Elements adherence

A framework analysis was conducted to determine how DoD ASPs were structured, using the CDC’s CEs as a guide. Data were obtained from the NHSN annual survey where survey questions in the antibiotic stewardship practices section are mapped to CEs; the survey directs this section to be completed with ASP leaders.^12,13 Though survey questions remain relatively stable, there were more major changes in 2018 and in 2021.¹² DoD adherence to each CE and for programs with all seven CEs in place was determined for each year 2017–2021, then as a five-year average. DoD data were compared to national adherence for the same time frame. National-level data were obtained from CDC’s Antibiotic Resistance and Patient Safety Portal, which includes DoD hospital data.¹⁴ Adherence was measured using CDC methodology, where a facility’s positive response to at least one question within a CE led to that CE being considered met. DoD adherence to the Priority CEs was also assessed for 2021 using the mapping guide,¹² and compared to published 2021 national data.¹⁵

All data were analyzed in Stata (Stata/IC 16.1 for Windows, StataCorp LLC, College Station, TX). This study was reviewed by the Uniformed Services University Human Research Protection Program and determined to be exempt.

Results

Facility characteristics

There were between 46 and 48 NHSN-reporting DoD hospitals from 2017 to 2021 with 47 in 2021. Compared to all hospitals reporting to NHSN, DoD facilities were smaller on average (24/47 DoD facilities had 50 or fewer beds, compared to 2,105/5,067 non-DoD facilities, RR = 1.23, 95% CI 0.93–1.63) (Table 1).^15,16 Of the 21 DoD teaching hospitals, 13 were graduate teaching and eight were major teaching (Table 2). Thirty-four of the hospitals were within the U.S. and 13 were outside.

Table 1. Bed size comparison of Department of Defense (DoD) and National Healthcare Safety Network (NHSN) reporting national hospitals

Abbreviations: DoD, Department of Defense; NHSN, National Healthcare Safety Network.

^a Data from O’Leary et al. (2017).¹⁶

^b Data from O’Leary et al. (2024).¹⁵

^c DoD hospitals are included in the nationally reporting facilities.

Table 2. Department of Defense (DoD) hospital characteristics based on 2021 National Healthcare Safety Network (NHSN) data and supplemental survey facilities

Abbreviations: DoD, Department of Defense; HHS, Department of Health and Human Services; ICU, intensive care unit; Min, minimum; Max, maximum; NHSN, National Healthcare Safety Network; Std. Dev., standard deviation.

^a Major teaching facilities have a program for medical students and post-graduate medical training (residencies/fellowships), while graduate teaching facilities only have post-graduate training. (Definition from NHSN Patient Safety Component Manual).⁸

^b Supplemental survey data not provided due to risk of facility identification.

Statistical significance of bivariate associations between facility characteristics is reported in Table 3, and significant associations are described below. Hospital location was associated with Service, teaching status, and size. Half of Air Force hospitals (5/10 were overseas versus 41.2% of Navy hospitals (7/17, RR 0.82, 95% CI 0.36 – 1.91), 11.1% of Army hospitals (2/18, RR 3.71, 95% CI 0.89–15.41; Navy comparator), and none of the five Joint hospitals. Only one of 14 overseas hospitals was a teaching hospital, compared with 21 of 36 hospitals located in the U.S. (RR 0.12, 95% CI 0.12–0.83). Twelve of 14 overseas hospitals had ≤ 50 beds compared with 15 of 36 hospitals located in the U.S. (RR 2.06, 95% CI 1.32–3.20), and none of the three largest hospitals were overseas. Hospital size was also associated with teaching status: 19/22 teaching hospitals had >50 beds compared to 4/28 non-teaching hospitals (RR 6.05, 95% CI 2.40–15.21). While of borderline statistical significance, the Navy tended to have more small hospitals (12/17) than the other Services (15/33, RR 1.55, 95% CI 0.96–2.52). (Appendix 2)

Table 3. Statistically significant bivariate associations between facility characteristics and supplemental survey variables

Abbreviations: ASP, antibiotic stewardship program; Q, question (from the supplemental survey).

^a “Unknown” responses removed.

* Statistically significant (p < .05).

^# Borderline statistically significant (.05 ≤ p ≤ .06).

- Not applicable.

Fisher’s exact test p-values are shown; Associations of one facility characteristic with another appear twice in the chart.

See Appendix 2 for additional data on these associations.

Supplemental survey

Forty of 47 (85.1%) DoD hospitals completed the supplemental survey. Duplicate responses were received for three hospitals; responses from the person with longer tenure in the ASP were kept. More respondents (n = 28, 70.0%) reported that they were the ASP Pharmacist Leader, 11 (27.5%) were the Physician Leader, and 1 (2.5%) was the Infection Prevention and Control Leader. As seen in Table 2, there was a significant difference between survey completion status and facility size; the seven non-respondent facilities all had ≤ 50 beds. Region was significantly associated with survey completion; details were not included due to risk of facility identification. No other facility characteristics significantly varied by survey completion status.

Survey results indicated that most hospitals started ASPs in 2017 or prior: 9 hospitals (22.5%) in 2017 and 11 (27.5%) prior to 2017. The “unknown” category (n = 10) was removed for bivariate analyses. There was a significant relationship between when an ASP started and categorical number of beds. The three largest hospitals started ASPs earlier, in 2017 or prior. There was a significant association between when the ASP started and medical affiliation; teaching hospitals started ASPs earlier.

ASP leaders in the DoD civilian/contractor employment category (hereafter civilian) were in their roles longer than military ASP leaders (Table 4). When asked if ASP leaders were more often assigned or requested to be in the role, 15 (37.5%) responded assigned, 9 (22.5%) said evenly requested and assigned, 7 (17.5%) responded that both occur but more often assigned, 5 (12.5%) said both but more often requested, and 4 (10.0%) said requested. When asked what proportion of ASP members were pharmacists, 27 (67.5%) reported 1–25%, followed by 10 (25.0%) reporting 26–50%. There was a statistically significant association between Region and proportion of pharmacists in the ASP, driven by Region 4 which has the most hospitals. While not statistically significant, there may be a weak association with proportion of time ASP pharmacist leaders spend on stewardship activities and hospital teaching status; less time was spent at non-teaching hospitals.

Table 4. Comparing Antibiotic Stewardship Program (ASP) leader employment categories and roles with tenure and ASP proportions by employment category

^a Only 4 “unknown” responses were recorded: 2 military – other type of leader; 1 civilian/contractor – physician; 1 civilian/contractor – other type of leader.

^b Calculated with 1 = 0–1 year; 2 = >1–2 years; 3 = >2 years-3 years; 4 = >3 years-4 years; 5 = >4 years.

When asked if an ASP leader was available on a daily basis, 35 (85.0%) replied “yes;” responses were not associated with facility characteristics. All respondents were asked on average what percent of time did physician and pharmacist ASP leaders dedicate to antimicrobial stewardship activities; 30 (75.0%) answered that physician leaders spent 1–25% of their time, and 8 (20.0%) said 26–50% of their time. Pharmacist leaders spent more time on ASP activities than physicians, but still the majority (23 (57.5%)) reported an average of 1–25% of time on ASP activities; 9 (22.5%) 26–50% of time, 2 (5.0%) 51–75% of time, and 6 (15.0%) 76–100% of time. There was no correlation with categorical number of beds or variation by level for number of admissions, number of patient days, or continuous number of beds with the reported proportion of time ASP leaders spent on stewardship activities.

Core Elements adherence

From 2017 to 2021, between 4,940 and 5,053 hospitals responded to the NHSN annual survey,¹⁴ including 46 to 48 DoD hospitals, meaning DoD represents approximately 1.0% of nationally reporting facilities (Table 5). Reporting facility participation varies year-to-year, thus comparisons over time do not compare the same facilities. National and DoD average percent CE adherence was very high overall, though the largest deficiencies over time were in Education, followed by Tracking and Reporting (Fig. 1). There were differences between national and DoD average adherence, with DoD lower than national levels for several CEs: −1.9% Education, −1.3% for both Action and Tracking, and −4.5% for having all seven CEs in place. For 2021, DoD was above national levels at 100% adherence for the procedural CEs of Action, Tracking, Reporting, and Education, and DoD was 0.7% higher for programs with all seven CEs in place. DoD was lower than national adherence for the structural CEs: −1.0% Leadership, −2.5% Accountability, and −0.9% Pharmacy Expertise. In 2021, Pearson’s χ² tests showed that adherence differences between national and DoD hospitals were not significant (p = 0.484 Leadership, 0.199 Accountability, 0.543 Pharmacy Expertise, 0.480 Action, 0.356 Tracking, 0.456 Reporting, 0.551 Education). Adherence for 2021 was high and thus not able to be stratified by levels of facility characteristics.

Table 5. National and Department of Defense (DoD) adherence to CDC Core Elements, 2017–2021 and Priority Elements, 2021

Abbreviations: CDC, Centers of Disease Control and Prevention; DoD, Department of Defense; NA, not applicable.

^a National Priority Elements data from O’Leary et al. (2024)¹⁵ – denominators differ from the CDC Antibiotic Resistance & Patient Safety Portal (AR & PSP) because 6 states/territories were excluded from the AR & PSP but were included in the O’Leary manuscript (personal email communication with Neuhauser, Melinda); number of facilities not available.

DoD hospitals are included in the nationally reporting facilities; National Core Elements data from the AR & PSP.

Refer to Fig. 1 for a visual representation of these data.

Figure 1. Percent of national and Department of Defense hospital adherence to CDC Core Elements, 2017–2021.

In 2021 for the Priority Elements, DoD hospitals reported lower adherence than national hospitals for four Priority Elements: −35.3% Leadership, −12.9% Accountability, −17.5% Pharmacy/Stewardship Expertise, and −14.8% Action; two Priority Elements were higher: +57.0% Tracking and +9.6% Reporting.¹⁵ DoD hospitals met an average of 3.3 of the Priority Elements, with 3 (6.4%) meeting all Priority Elements; 9.5% of national hospitals met all six.¹⁵ There were 8 DoD facilities (17.0%) meeting five of the Priority Elements and 9 (19.1%) meeting four. Thus, 36.2% of DoD hospitals met four or five Priority Elements, compared to 45.6% of national hospitals. In the MHS, there did not appear to be a facility size association with meeting all six Priority Elements.

Discussion

This study represents the inaugural effort to conduct an enterprise-wide evaluation of MHS ASPs to inform policy and quality improvement. Results indicate similarities between DoD and national CE adherence but provide areas for increased emphasis and policy context. Similar performance improvement research within the MHS is generated by the Centers of Excellence, the majority of which are congressionally mandated organizations established to inform policy and drive DoD-wide health outcome improvements.^17,18 Comparable efforts have been conducted at some state health agencies, such as Nebraska’s Antimicrobial Stewardship Assessment and Promotion Program which completed statewide CE assessments for critical access hospitals and long-term care facilities.¹⁹

Many of the facility characteristics results were expected but provide important context to inform policy. ASP leaders in the MHS are more often assigned rather than volunteer and are infrequently in place > 4 years, which aligns with the typical military rotation cycle. This practice likely impacts continuity of the ASPs and how interventions are implemented and evaluated. While civilians may be in their roles longer, staffing mix results indicate that they are usually at US-located, larger hospitals where ASP teams are likely larger, making this mix possible. Thus, continuity impacts are felt more at the smallest hospitals in the MHS, which are the majority. Another optimal staffing mix consideration is that pharmacist leaders tend to spend more time on ASP activities than physicians, which was possibly biased due to more pharmacists responding to the survey, though this is supported by similar national-level data.¹⁵ The finding that the proportion of time spent on ASP activities did not vary by size/volume characteristics was surprising but should be investigated to determine if larger hospitals have larger ASPs and leader time does not increase proportionately but overall time spent does. Future studies should determine if ASP leaders are provided protected time to complete stewardship duties, which is necessary to successfully drive improvements.¹⁵

The findings related to CE average adherence partially supported the research hypothesis and suggested that there are opportunities for the DoD to improve where CE adherence lags behind national levels, namely for the recent differences in structural CEs. Work should first focus on improving the Priority Elements within these CEs. For Leadership Priority CEs, this could mean increasingly including stewardship responsibilities in contracts or job descriptions (DoD physicians 15.2% adherence; 58.3% nationally; DoD pharmacists 33.3%; 60.5% nationally). Accountability CE data suggest emphasizing having ASP pharmacist and physician co-leaders. Physicians were more often solo ASP leaders in the DoD (19.1% DoD; 6.5% nationally), and ASPs were less often co-led (51.1% DoD; 63.9% nationally). The Stewardship/Pharmacy Expertise Priority CE indicates additional infectious disease (ID) training could be beneficial for ASP leaders. Only 48.5% of DoD ASP physician leaders had completed an ID fellowship compared to 69.7% nationally (certificate programs were 10.1% lower, training courses were similar to national levels). Similarly, for DoD pharmacist ASP leaders, 11.1% had completed an ID residency/fellowship compared to 21.1% nationally (certificate programs were 23.3% lower, training courses were 18.7% lower than nationally).

This analysis is limited by the relatively small number of hospitals in the DoD, missing supplemental survey results for nearly 30% of the smallest hospitals, and variable adherence. There is a possibility of type I error due to the number of comparisons done between facility characteristics and survey questions. Drops in DoD adherence from 2019 to 2020 were likely influenced by the military’s considerable support of COVID-19 efforts in civilian healthcare settings. While COVID-19 was not a major U.S. issue in 2019, reporting for the 2019 annual survey was due in March 2020, leading researchers and the ASPC to hypothesize that the 2020 ASP status was reported rather than the preceding year’s status. DoD CE adherence quickly rebounded in 2021. While authors are limited by not having 2022 data to further analyze trends, they are not aware of other factors that would have contributed to this outside of resource constraints driven by COVID-19 support and policy-driven incremental increases in adherence over time as has also been noted nationally.¹⁵ In 2021 all DoD hospitals were considered to have met the four CEs of Action, Tracking, Reporting, and Education. Nationally, only 3% of hospitals with ≤ 50 beds met the Priority Elements, while 18% of hospitals with > 200 beds met them.¹⁵ Since over half of MHS hospitals are in the smallest size range, the DoD will be more apt to lag overall national adherence. DoD was well-positioned to meet the Tracking Priority Element for all hospitals, which requires reporting into the NHSN Antibiotic Use module; the common electronic health record and centralized reporting structure in the DoD facilitate this. Future MHS studies should explore possible geographical variation in CE adherence to determine if it exists as in civilian settings,¹⁴ and comparisons to other healthcare systems would be useful.

A strength of this study is that it included all MHS hospitals and supplemental survey data represented 85.1% of hospitals. However, the MHS population and hospitals are not evenly spread across the U.S.²⁰ Compared to the general population, the MHS has more male and older beneficiaries (though those > 65 are not often seen at military hospitals), and with 16.5% serving currently, there could be some impact of the healthy worker effect.^20,21 The active duty and general populations ages 18–44 are similar in race and gender proportions, though this varies by Service.²² Therefore, these results are not generalizable to other healthcare systems. Though similar methods may be beneficial to healthcare systems and health agencies to guide high-level ASP improvement efforts.

These findings indicate that the CDC’s methodology for measuring Core Elements adherence lacks sensitivity as illustrated by all DoD hospitals meeting four of the seven CEs in 2021. As O’Leary and colleagues point out, there has not been strong evidence that higher adherence to the CEs directly improves antibiotic use.¹⁵ Better ways to quantitatively measure CE adherence and to tie these measures to ASP-related and patient outcomes are a focus of future research for this group and are needed to shape more granular, data-driven guidance and improvement initiatives.