Background
One in four hospitalized patients with community-acquired pneumonia undergo testing for respiratory viruses. Reference Klompas, Imrey and Yu1 Though nucleic acid amplification tests (NAATs) are considered the reference standard for diagnosing viral respiratory infections, issues related to cost, availability, turnaround time, and clinical relevance have led to institution-specific implementation in clinical settings. Reference Hanson, Azar and Banerjee2,Reference Baghdadi, O’Hara and Johnson3 Decisions about which molecular tests to use and how to support high-value testing are further complicated by the variety of multiplex panels that are available. Reference Ramanan, Bryson and Binnicker4,Reference Huang, Tsai and Chang5
Implementation of a diagnostic test refers to its deployment for clinical use within a care pathway. Reference John, O’Kane and Jülicher6 The COVID-19 pandemic had widespread spillover effects on the epidemiology of community-acquired respiratory viruses, such as influenza and respiratory syncytial virus (RSV). Reference Chow, Uyeki and Chu7,Reference Olsen, Winn and Budd8 However, the pandemic also impacted how NAATs for respiratory pathogens were implemented. When test makers added SARS-CoV-2 to commercially-available multiplex respiratory panels, many medical facilities sought access to the new tests as quickly as possible, even if multiplex respiratory panels had not been in use at those facilities prior to the pandemic. Reference Mitchell, St George and Rhoads9,Reference Dien Bard and Babady10 At the same time, pandemic-era supply shortages meant that which tests were implemented was not always driven by clinical utility. Reference Amirian11–13 To ensure continuous access to testing, many laboratories implemented multiple platforms and panels simultaneously, creating opportunities for head-to-head comparisons. Reference Luethy and Johnson14 Implementation also varied depending on whether testing was being used for diagnosis of symptomatic patients or for infection prevention and surveillance among asymptomatic patients.
The purpose of this study was to understand how implementation and use of multiplex NAATs for respiratory pathogens changed before and during the COVID-19 pandemic. By extension, we sought to infer what types of NAATs may be most clinically useful and what patterns of testing might best reflect the standard of care. The results of this analysis have implications for diagnostic stewardship of molecular testing.
Methods
We performed a retrospective observational study of acute care discharge data from PINC-AI (formerly the Premier Healthcare Database). PINC-AI is an all-payer deidentified administrative data set that contains microbiologic testing data from >100 hospitals that are diverse in geography, size, and patient-mix, though the number of contributing facilities varies by quarter. 15 Acute care visits for patients of all ages, including inpatient admissions, observation admissions, and emergency department from January 2019 to April 2021 were included. Admissions under observation status are intended for short-term assessment and treatment when longer hospitalization may not be required. Reference Sheehy, Graf and Gangireddy16 General characteristics of the hospitals contributing data from the same period of our study to PINC-AI has been reported elsewhere. Reference Goodman, Magder and Baghdadi17,Reference Baghdadi, Coffey and Adediran18
NAAT tests were identified in microbiologic data based on inclusion of a relevant keyword in the test name (e.g., “dna,” “rna,” “gene,” “pcr”). Multiplex panels were either explicitly labeled as a panel (e.g., “respiratory pathogen panel”) or inferred based on multiple NAATs collected at the same moment in time for a single patient from the same specimen source. Panels targeting upper respiratory pathogens or pneumonia pathogens from an upper or lower respiratory source were included as respiratory panels. Permissible specimen types were nasal, nasopharyngeal, oral, oropharyngeal, bronchial, bronchoalveolar, and sputum. Multiplex panels designed to diagnosis streptococcal infection in oropharyngeal specimens were included, because these tests are in the clinical evaluation of symptomatic patients presenting for acute care with cough or other respiratory symptoms. Standalone NAATs targeting only Staphylococcus aureus were excluded because these tests are used for surveillance throughout the hospital for patients without suspected respiratory illness.
The composition of panels was determined based on reported results. For instance, a panel reporting three results for SARS-CoV-2, respiratory syncytial virus, and influenza was considered distinct from a panel reporting four results for SARS-CoV-2, respiratory syncytial virus, influenza A, and influenza B. Using size cut-offs adapted from Current Procedural Terminology (CPT) coding, panels were categorized as small (2–5 targets), medium (6–11), and large (12 or above) based on the number of distinct pathogenic targets reported. Average costs were estimated using encounter-level billing data associated with relevant CPT codes.
The purpose of this analysis was to evaluate how volume and selection of respiratory panel testing changed after onset of the COVID-19 pandemic. The reference period was the immediate prepandemic respiratory viral season, which lasted from November 2019 until February 2020. To account for initial testing shortages and the evolving epidemiology of COVID-19 in the US, the postpandemic period was divided into an early phase (March 2020 until October 2020) and a late phase (November 2020 until April 2021). Reference Pei, Yamana and Kandula19 November 2020 was selected as the cut-point between periods as this was the first month in which the United States reported 100,000 cases in a single day. Based on observed skewness in the testing data, testing patterns were also described repeated among the top decile of respiratory panel utilizers. Statistical comparisons among periods were performed by ANOVA.
Results
Microbiologic testing data was obtained for 245 distinct medical facilities during the study period, including 178 facilities during the prepandemic respiratory viral season, 208 facilities during the early pandemic, and 209 facilities during the late pandemic. Each facility contributed a median of 24 months (interquartile range 9–28) of data. Facilities were 71% urban and 28% teaching. Fifty-eight percent were located in the South, 25% were in the Midwest, 13% were in the Northeast, and 4% were in the West. Nineteen percent of facilities had 400 or more beds, 29% had 200–399 beds, and 53% had 199 beds or fewer.
Changes in volume of respiratory panel testing by phase of the pandemic
The overall volume of respiratory panels performed per month at contributing hospitals are shown in Figure 1 (monthly averages and medians are presented in Table 1). During the prepandemic respiratory viral season, each facility performed an average of 103 (standard deviation 138) multiplex panels per month, including 79 large panels (sd 126), 7 medium panels (sd 31), and 16 small panels (sd 73). In the early pandemic, panel use decreased by an average of 41 panels monthly per facility (95% confidence interval for the difference: 29–52) to 62 (sd 112), with large panel use decreasing on average by 28 monthly per facility (95% CI: 17–39) to 52 (sd 106), small panel use decreasing on average by 10 monthly per facility (95% CI: 5–15) to 6 (sd 36), and medium panel use decreasing on average by 3 monthly per facility (95% CI: 0–5) to 4 (sd 23). Relative to the prepandemic baseline, panel use during the late pandemic increased by an average of 4 panels monthly per facility (95% CI: –23–14) to 107 (sd 211), with small panel use increasing on average by 42 monthly per facility (95% CI: 29–55) to 58 (sd 156), large panel use decreasing by 32 monthly per facility (95% CI: 20–44) to 47 (sd 116), and medium panel use decreasing by 5 monthly per facility (95% CI: 3–7) to 2 (sd 13). Comparisons across periods demonstrated significant differences in overall volume of testing (P < 0.0001) and use of large panels (P < 0.0001), medium panels (P = 0.0001), and small panels (P < 0.0001).
Figure 1. Utilization of respiratory panels over time by panel size. The plot represents the total volume of multiplex nucleic acid amplification test panels for respiratory pathogens performed per month at ∼200 medical facilities contributing microbiology data to PINC-AI (formerly Premier) during the study period. Panels were categorized as small (2–5 tests), medium (6–11), or large (12 or above). “Fourplex” panels include a subset of small panels testing for 3–4 pathogens, most commonly COVID-19, respiratory syncytial virus, and influenza A/B. Vertical dashed lines delineate periods, including the prepandemic respiratory viral season (November 2019–February 2020), the early pandemic period (March 2020–October 2020), and the late pandemic period (November 2020–April 2021).
Table 1. Multiplex nucleic acid amplification respiratory panel testing by period
Atypical bacterial pathogens included Mycoplasma pneumoniae, Chlamydia pneumoniae, Legionella pneumoniae, and Bordetella species.
Sd, standard deviation; IQR, interquartile range; RSV, respiratory syncytial virus.
Changes in respiratory panel testing at hospitals with the highest utilization
Frequency of testing varied widely among medical facilities (see Table 2). Facilities in the top decile of utilization were 88% urban, 67% teaching, and 46% had 400 or more beds. Among facilities in the top decile of utilization, the monthly average per facility during the prepandemic period was 349 (sd 151) respiratory panels, including 290 large panels (sd 167), 47 (sd 153) small panels, and 13 (sd 62) medium panels. In the early pandemic period, respiratory panel testing at facilities in the top decile of utilization decreased by a monthly average per facility of 125 (95% CI: 82–168) panels to 224 (sd 181), including a decrease of 84 large panels (95% CI:41–128) to 205 (sd 178), a decrease of 35 small panels (95% CI: 8–61) to 12 (sd 72), and a decrease of 5 medium panels (95% CI: –7–18) to 8 (sd 43). In the late pandemic period, testing at the top decile of hospitals increased by an average difference of 112 panels per month (95% CI: 35–189) to 461 (sd 353), including an increase of 184 small panels (95% CI: 114–253) to 230 (sd 313), a decrease of 64 large panels (95% CI: 10–119) to 225 (sd 224), and a decrease of 7 medium panels (95% CI: –5–19) to 6 (sd 30). Comparisons across periods demonstrated significant differences in overall volume of testing (P < 0.0001), use of small panels (P < 0.0001), and use of large panels (P < 0.025).
Table 2. Characteristics of highest utilizers of multiplex respiratory panel testing
Highest utilizers were defined as being in the top decile of overall volume of panels performed.
* Comparisons were performed using Fisher’s exact test.
** 12 of 24 highest utilizer facilities were located in the South Atlantic Division.
Positive results from multiplex respiratory panels
Out of 258,724 respiratory panels performed at contributing hospitals during the study period, 40,997 (16%) were positive, including 16,227 positives in the prepandemic respiratory viral season (25% of panels from that period), 9,695 early pandemic positives (12% of panels during the early pandemic), and 15,075 late pandemic positives (14% of panels from the late pandemic). In the prepandemic respiratory viral season, the most common pathogens detected by multiplex respiratory panels were rhinovirus/enterovirus (5,806 positive results, 36% of positives), RSV (3,688 positives, 23% of all positives) and human metapneumovirus (1,436 positives, 9% of all positives). During the early pandemic, the most common pathogens were rhinovirus/enterovirus (4,355 positives, 45% of all positives), human metapneumovirus (1,071 positives, 11% of all positives), and SARS-CoV-2 (703 positives, 7% of all positives). In the late pandemic period, the most common pathogens were SARS-CoV-2 (9,307 positive results, 62% of positives) and rhinovirus/enterovirus (2,973 positive results, 20% of positives). 5,504 tests (1.4% of all panels) were positive for more than one pathogen.
Positive results for bacterial pathogens
Across all three periods, 497 respiratory panels were positive for atypical bacterial pathogens (0.2% of tests performed). Bacterial pathogens detected were Mycoplasma pneumonia in 340 instances (0.1% of tests performed), Bordetella pertussis or parapertussis in 119 instances (0.05% of tests performed), Chlamydia pneumoniae in 42 instances (0.03% of tests performed), and Legionella pneumophila in 1 instance (0.00% of tests performed).
Standalone PCR
The volume of standalone PCR testing in comparison with the volume of respiratory panel testing is shown in Figure 2. Overall, the most frequently performed standalone respiratory PCR test was for SARS-CoV-2 (1,872,425 total tests performed, average of 580 per hospital per month when tests were available), followed by RSV (107,144 total tests, average of 33 per hospital per month). In the prepandemic period, the most common viral test performed was for RSV. Out of 173,520 standalone respiratory PCR tests that were positive, 163,821 (94%) were positive for SARS-CoV-2.
Figure 2. Comparison of multiplex versus standalone molecular testing over time. The plot represents the volume of standalone nucleic acid amplification tests for respiratory pathogens performed at ∼200 medical facilities contributing microbiology data to PINC-AI (formerly Premier) during the study period. Vertical dashed lines delineate periods, including the prepandemic respiratory viral season (November 2019–February 2020), the early pandemic period (March 2020–October 2020), and the late pandemic period (November 2020–April 2021).
Cost of respiratory panels
During the prepandemic period, the total cost to healthcare facilities was $164 (sd $145) for performing a large respiratory panel, $265 (sd $293) for a medium panel, and $48 (sd $41) for a small panel. During the early pandemic period, these costs increased to $202 (sd $192), $250 (sd $258), and $98 (sd $82), respectively. By the late pandemic period, the costs of large and medium panels had settled to $155 (sd $187), $154 (sd $216), and $75 (sd $56), respectively.
Discussion
Based on real-world testing data from over 250 hospitals, we observed a shift in practice from clinical use of predominantly large multiplex NAAT respiratory panels prepandemic to small panels later in the pandemic after a transient disruption in use of panels of all sizes during the early pandemic period. This study represents the largest to date to examine patterns of respiratory NAAT testing across multiple hospitals, and our findings likely match the lived experience of most frontline clinicians. Use of small multiplex NAAT respiratory panels has become much more common in clinical care.
Multiple factors may drive implementation and clinical use of diagnostic testing, and we were unable to evaluate why specific tests were performed. During the early pandemic period, decision-making about which tests to implement was likely driven by the availability of supplies and staff. 13 We observed decreased use of respiratory panels of all sizes consistent with testing shortages during this period. Later in the pandemic, after disruptions in the supply chain began to ease, increased use of small panels may have reflected growing familiarity with and acceptance of point-of-care testing. Reference Babic, Garner and Hirschhorn20 However, beyond what tests are available, real-world data can reveal the preferences of laboratorians, hospital administrators, and clinicians on which multiplex NAAT respiratory panels have the highest relative clinical value and utility. We suspect that the observed trends in testing reflect the perceived relative value and utility of different respiratory panels. For instance, we suspect that the observed shift from predominantly large panel use prepandemic to predominantly small panel use postpandemic was at least partially driven by the difference in cost of large and small panels. Reference Baghdadi, O’Hara and Johnson3 Issues related to cost are exacerbated by more selective coverage for the large panels by health insurance.
However, the postpandemic shift toward increased use of small panels may also be attributable to the higher relative clinical utility of small panels. The potential appeal of large NAAT respiratory panels is that they test “for everything.” Reference Schreckenberger and McAdam21 In practice, this approach means that large panels tend to include pathogens that clinicians are either less aware of or less concerned about, such as human rhinovirus, metapneumovirus, or bocavirus. Small panels, on the other hand, are typically limited to viral pathogens that clinicians care about, either because the results will inform antiviral treatment, as with COVID-19 or influenza, or because of important related clinical syndromes, as with RSV. Thus, small panels may be perceived as providing greater value both in terms of clinical yield and costs.
Some diagnostic tests, such as NAATs for SARS-CoV-2, produce meaningful results even when negative. But most clinicians do not perform respiratory panel testing with the intention of ruling out disease due to seasonal coronavirus HKU1. Testing for this viral pathogen, like most of the viral pathogens included on large respiratory panels but excluded from small respiratory panels, is only meaningful when positive. Human coronavirus HKU1 is “marginal” both in the sense that it is less clinically important and the sense that testing for it is a marginal benefit offered by larger respiratory panels. For marginal pathogens, the rate of test positivity can be considered an indirect measure of clinical utility. Thus, the observation that multiplex NAAT respiratory panels are rarely positive for marginal pathogens further supports our inference that large respiratory panels do not provide much marginal utility over small panels.
When considering the relative value and utility of each pathogen included on a respiratory panel, atypical bacteria represent a special case. Classically, M. pneumoniae, C. pneumoniae, and Legionella, were believed to cause 10-20% of all cases of community-acquired pneumonia. Reference Bartlett and Mundy22 More recent studies have estimated the incidence of these pathogens as closer to 1%–3%. Reference Schreckenberger and McAdam21,Reference Jain, Self and Wunderink23 When including all patients tested by respiratory panel, the incidence in our sample was <1%. Theoretically, tests for atypical bacteria should be meaningful even when negative, because a negative test can support appropriate discontinuation or de-escalation of antibiotics. However, multiple studies have demonstrated that respiratory panel testing does not influence antibiotic use by clinicians in practice among adult patients. Reference Hatzileris, Holder and Braun24–Reference Brendish, Malachira and Armstrong26 Indeed, many clinicians are not even aware many respiratory panels include atypical bacteria. Reference Baghdadi, Yang and Lynen27 Thus, our finding that respiratory panels are almost never positive for atypical bacteria supports the inference that large respiratory panels are less useful than small ones. If a clinician’s reason for testing with a large respiratory panel is to evaluate for atypical bacterial infection, they might as well not bother.
This study is subject to the limitations of an analysis of administrative discharge data. We ascertained the composition and size of respiratory panels based on the way groups of NAATs were collected and reported, rather than based on a test order or panel name. Thus, though our method may not capture the true composition of every panel, it does reflect the way each panel was reported to clinicians on the treatment team. Further, we were unable to account for why testing was being performed. We likely captured testing that was performed both for clinical diagnosis and testing performed for surveillance and infection prevention. Though we are unable to understand why specific tests or groups of tests were ordered, the observed trends in testing may generate hypotheses related to clinical utility of respiratory panels of different sizes.
Conclusions
After onset of the COVID pandemic, clinical use of multiplex NAAT respiratory panels shifted from predominant use of large panels targeting 12 or more pathogens to predominant use of small panels targeting 5 or fewer. This change may have been driven by multiple factors, including the availability of testing resources, cost of testing, low positivity rates of marginal pathogens, and higher relative clinical utility of small respiratory panels that target important pathogenic viruses rather than testing “for everything.”
Financial support
DB received support from the funding from the Agency for Healthcare Research and Quality (1K08HS028854-02) and the Institute for Health Computing at the University of Maryland, Baltimore.
Competing interests
None.
