COVID-19 seroprevalence amongst healthcare workers: potential biases in estimating infection prevalence

Abstract

SARS-CoV-2 serological tests are used to assess the infection seroprevalence within a population. This study aims at assessing potential biases in estimating infection prevalence amongst healthcare workers (HCWs) when different diagnostic criteria are considered. A multi-site cross-sectional study was carried out in April–September 2020 amongst 1.367 Italian HCWs. SARS-CoV-2 prevalence was assessed using three diagnostic criteria: RT-PCR on nasopharyngeal swab, point-of-care fingerprick serological test (POCT) result and COVID-19 clinical pathognomonic presentation. A logistic regression model was used to estimate the probability of POCT-positive result in relation to the time since infection (RT-PCR positivity). Among 1.367 HCWs, 69.2% were working in COVID-19 units. Statistically significant differences in age, role and gender were observed between COVID-19/non-COVID-19 units. Prevalence of SARS-CoV-2 infection varied according to the criterion considered: 6.7% for POCT, 8.1% for RT-PCR, 10.0% for either POCT or RT-PCR, 9.6% for infection pathognomonic clinical presentation and 17.6% when at least one of the previous criteria was present. The probability of POCT-positive result decreased by 1.1% every 10 days from the infection. This study highlights potential biases in estimating SARS-CoV-2 point-prevalence data according to the criteria used. Although informative on infection susceptibility and herd immunity level, POCT serological tests are not the best predictors of previous COVID-19 infections for public health monitoring programmes.

Introduction

Since 21st February 2020 Italy reported cases of the 2019 coronavirus disease (COVID-19). As a consequence of the extraordinary containment measures implemented by the Italian Government between 8th March and 18th May 2020, the incidence of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) infection progressively reduced and the first wave of the pandemic ended in early summer 2020 [1]. In total, 225 435 COVID-19 cases occurred in this period of time, 13.8% of them were lethal [2]. The Veneto Region resulted as one of the most affected area accounting, by October 2020, for the 8.7% of overall Italian cases (46.992) [3]. Within this scenario, healthcare workers (HCWs) were particularly exposed to SARS-CoV-2 infection considering their role of frontline workforces in the response to COVID-19 pandemic [4, 5].

Since April 2020, many serological tests have been developed to assess the infection seroprevalence and to estimate the progress towards the goal of reaching the herd immunity. As expected, for a novel disease like COVID-19, the diagnostic tests have limitations that might reduce their clinical and epidemiological validity. The first limit is related to the not perfect intrinsic analytical characteristics of the tests, particularly concerning the sensitivity and specificity, as pointed out by many authors [6–9]. The second, that is particularly evident when considering the point-of-care tests (POCT), is the gap between the performances achieved in the lab (usually based on plasma or serum) and those reported in real-life studies on fingerprick blood [10–13]. In addition, serological tests to detect SARS-CoV-2 are affected not only by the kinetic of antibody production, which requires few weeks to produce antibodies in such concentration to be detected in human blood [14], but also by the fact that SARS-CoV-2 antibodies decay over time [13].

Many papers have been published reporting SARS-CoV-2 seroprevalence amongst HCWs during COVID-19 first wave. However, their results are far from being solid and definitive as based on point-prevalence studies which consider different types of tests with limited samples and different criteria for HCWs enrolment (i.e. surveillance, contact tracing/presence of symptoms, etc.). Positivity rates amongst HCWs ranged between 8.5% as described by Galanis et al. in their systematic review [15] to 11.8% as recently found in a large cross-sectional study in Spain [16]. However, SARS-CoV-2 prevalence varied largely according to clinical presentation as well described by Pallett et al. in the UK: 10.6% and 44.7% for asymptomatic and symptomatic HCWs, respectively, with an overall positivity rate of 18.0% [17]. The World Health Organization (WHO) estimated that HCWs accounted for approximately 14% of COVID-19 cases [18]. Similar rates have been reported in Italy where the proportion of HCWs amongst COVID-19 patients was found to be 9% [19]. This prevalence could be lower when considering HCWs as a target of specific monitoring programmes as part of occupational risk surveillance. This is the case of a serosurvey carried out in the Veneto Region where SARS-CoV-2 infection seroprevalence among HCWs was found to be 4.6% [20]. As pointed out by a recent Cochrane systematic review on SARS-CoV-2 antibody tests [13], variation in SARS-CoV-2 seroprevalence is usually due to non-homogeneous data as they often consider either hospitalised or symptomatic cases or asymptomatic individuals.

This paper describes the findings of a study implemented with the aim of triangulating different sources of information (RT-PCR testing, POCT serological test results and self-reported symptoms) and assessing the potential bias in estimating SARS-CoV-2 prevalence when basing this calculation on different case definitions.

Materials and methods

Population, setting, tools and procedure

This is a cross-sectional multi-centre study carried out in the four main public hospitals in the Verona Province (Veneto Region, Italy), from 21st April to 14th September 2020. All hospitals were either indicated as COVID-19 province hub (devoted to confirmed SARS-CoV-2 cases) or general hospitals (providing essential non-COVID-19 services) with dedicated COVID-19 wards.

HCWs from COVID-19 and non-COVID-19 units were recruited prospectively and asked to participate in the study signing the informed consent and completing an ad hoc online survey. An HCW was defined as a person who provide healthcare services both directly (i.e. doctors, nurses, midwives, etc.) or indirectly (i.e. laboratory technicians, admin staff, etc.) within the selected hospitals.

The survey was designed to collect socio-demographic characteristics, professional role, contacts with COVID-19 patients and colleagues, SARS-CoV-2 testing history and a list of signs and symptoms (S&S) HCWs might have experienced since February 2020. At the enrolment, HCWs were asked to fill in the electronic survey and being tested using the COVID-19 IgG/IgM rapid test (Healgen Scientific LLC, USA), henceforth named POCT, on fingerpicked blood. POCTs were performed and read by trained staff following manufacturer's instructions and their results were recorded on a specific form. As described in POCT package insert, sensitivity is 96.7% (95% confidence interval (95% CI) 90.7–98.9), 86.7 (95% CI 78.1–92.2) and 96.7% (95% CI 90.7–98.9) for IgG, IgM and IgG/IgM component, respectively. As for specificity, the percentages are, respectively, 100% (95% CI 93.1–99.5), 99.0% (95% CI 94.6–99.8) and 97.0% (95% CI 91.6–99.0) [21].

For all HCWs with a positive POCT test and, based on the survey form, no previous history of COVID-19, RT-PCR on nasopharyngeal swab (NPS) was performed to ascertain the risk of SARS-CoV-2 transmission.

The study was approved by the local ethic committee (2653/2857CESC).

Case definitions

A COVID-19 case was defined as a person (i) with a history of positive SARS-CoV-2 RT-PCR on NPS and/or (ii) with a positive result of the POCT during the study and/or (iii) reporting, from February till April 2020, COVID-19 typical S&S [19–22]. The decision of including COVID-19 pathognomonic clinical presentation in the case definition was taken considering that SARS-CoV-2 antibody titres often decrease over time and that a massive molecular screening testing with RT-PCR was implemented since April 2020. Therefore, HCWs who acquired SARS-CoV-2 infection in the first 8 weeks of the pandemic might have resulted negative in both molecular and rapid serological tests. A COVID-19 case was considered (a) certain with COVID-19 pathognomonic S&S such as ‘ageusia/anosmia’ or ‘fever and/or cough and dyspnoea’ or ‘fever and nausea/vomit or diarrhoea’ since February 2020 or (b) possible when the following clinical conditions were reported: ‘fever and arthromyalgia’ or ‘fever and fatigue’ or aspecific symptoms without systemic involvement (headache, dizziness, pharyngalgia, confusion). Only HCWs who reported COVID-19 pathognomonic S&S were considered as SARS-CoV-2 cases.

As for SARS-CoV-2 diagnostic case, only HCWs who did result positive to either RT-PCR on NPS or the study POCT were included.

Therefore, for the estimation of the overall SARS-CoV-2 prevalence, we considered as numerator the number of HCWs who resulted positive to any SARS-CoV-2 diagnostic tests and/or reporting SARS-CoV-2 pathognomonic S&S in the first 8 weeks of the pandemic, while as denominator all study participants.

Statistical analysis

Mean, median, standard deviation and Wilcoxon–Mann–Whitney test were respectively used for quantitative variables and group comparison. For nominal variables, percentages were estimated, while Fisher's exact test was used to estimate the association between categorical variables. The prevalence was estimated as percentage of proportion and 95% CI was based on logit transformation. A multivariate logistic model was used to estimate the probability of serological positive test result in relation to the date in which the test was performed. STATA Version 16.2 was used for analyses (College Station, TX, USA: StataCorp LP).

Results

Study participants

Overall, 1367 HCWs were enrolled in the study with a median age of 41.3 years. Socio-demographic characteristics of the study population are summarised in Table 1.

Table 1. Socio-demographic characteristics of the HCWs enrolled in the study, by COVID-19 units

P values refer to the comparison of diagnostic results between COVID-19/non-COVID-19 units.

* Statistically significant (P < 0.05)

In total, 946 (69.2%) HCWs were working in COVID-19 units. In total, 399 (29.3%) were medical doctors, 724 (53.1%) were nurses, midwifes or physiotherapists, 224 (16.4%) were healthcare assistants. Sixteen (1.2%) were diagnostic radiographers, biologists, administrative personnel, data managers or technicians.

In COVID-19 wards personnel was more frequently younger, male, healthcare assistant compared to non-COVID-19 units. A detailed list of statistically significant differences is presented in Table 1.

SARS-CoV-2 infection prevalence

In line with the case definitions, different prevalence estimates were calculated: 8.1% (95% CI 6.8–9.7) with RT-PCR on NPS, 6.7% (95% CI 5.5–8.1) with POCT, 9.6% (95% CI 8.1–11.3) with pathognomonic S&S. When both diagnostic tests were combined, the prevalence was 10% (95% CI 8.5–11.7), however, when also the pathognomonic S&S were included, the overall prevalence resulted 17.6% (95% CI 15.6–20.0). Table 2 presents the distribution of different COVID-19 estimates by COVID-19/non-COVID-19 units based on the different case definitions considered.

Table 2. SARS-CoV-2 prevalence estimates amongst HCWs according to the criterion considered: pathognomonic S&S, COVID-19 previous molecular diagnosis, POCT results, by place of work (in bold the condition for satisfying the criterion and being considered for analysis).

P values refer to the comparison of diagnostic results between COVID-19/non-COVID-19 units.

* Statistically significant (P < 0.05).

Ninety-one HCWs resulted positive to the POCT. Among them, 60 (65.9%) were positive only for IgG, five (5.5%) only for IgM, while 26 (28.6%) for both IgG and IgM. As far as the comparison between COVID-19 and non-COVID-19 wards is concerned, the number of HCWs positive for IgG was 26 (6.8%) and 60 (6.1%) respectively (P = 0.022), while for both IgG and IgM the positives were 72 (7.6%) and 19 (4.5%) (P = 0.034) (Table 2).

In total, 111 (8.1%) HCWs reported a previous COVID-19 diagnosis by RT-PCR on NPS. Among them, 85 (69.2%) were working in COVID-19 wards. Based on RT-PCR on NPS and POCT results, the diagnostic prevalence was 10.9% amongst HCWs working in COVID-19 units and 7.8% in those working in non-COVID-19 units. The difference was not statistically significant to 5% (P = 0.096).

During the period from February till early April 2020, 128 (9.6%) HCWs reported SARS-CoV-2 pathognomonic S&S [22–25]. Amongst them, 97 (75.8%) were working in COVID-19 units and 31 (24.2%) in non-COVID-19 units (P = 0.208).

Considering the presence of at least one of the case definitions considered, the overall SARS-CoV-2 prevalence was found 17.6%. COVID-19 personnel were more frequently positive (19.0%, 95% CI 15.6–20.0) than the non-COVID-19 HCWs (14.3, 95% CI 11.2–17.9) (P = 0.037).

Seropositivity over time

Amongst the 111 HCWs with a reported previous SARS-CoV-2 diagnosis based on NPS RT-PCR, the number of individuals with a positive (IgG and/or IgM) POCT result was 66 (59.5%). As the date of the POCT and the first positive RT-PCR on NPS was recorded, it was possible to estimate the time interval. The median time was 125 days (102.2 ± 49.7). The median time difference between those who resulted positive (63.5 days ± 49.8, min 16–max 182) and negative (136 days ± 45.9, min 0–max 161) at the POCT was statistically significant (P = 0.0196).

Considering the HCWs working in COVID-19 units, we did not find any significant difference between those with a positive and a negative serology. In addition, amongst HCWs reporting both a previous RT-PCR positivity and COVID-19 pathognomonic S&S, the median time between these variables was found to be 1 day (iqr 5).

Based on a logistic model, the probability of seropositivity in relation to the time elapsed between the infection and the POCT was estimated. The model showed a decrease of 1.1% every 10 days since the date of RT-PCR first positive result for the IgG and/or IgM (OR 0.99, 95% CI 98.1–99.7, P = 0.009). This estimate varies according to the type of antibody considered. In particular for the IgM only, this probability seemed to decrease more rapidly (1.8%, OR 0.98, 95% CI 97.2–99.3, P = 0.001) compared to the IgG-only component (1.1%, OR 0.99, 95% CI 98.1–99.7, P 0.010). Figure 1 shows the predicted probability of a positive result for IgG (a) and IgM (b).

Fig. 1. Probability in having an IgG (a) or IgM (b) positive result according to the time between SARS-CoV-2 infection diagnosis (RT-PCR on NPS) and the POCT execution.

Figure 2 shows the number of HCWs who could be classified as having had a SARS-CoV-2 infection, based on the combination of the three case definitions used in this paper. In total, 240 HCWs resulted positive to at least one criterion, 76 to at least two criteria and 13 to all the three ones. Despite the limitation of pathognomonic S&S case definition, 104 HCWs with a probable SARS-CoV-2 infection would be missed if not considered.

Fig. 2. SARS-CoV2 infection frequency according to the diagnostic criteria considered.

Discussion

Italy was the first European Country to report autochthonous COVID-19 cases and the Veneto Region was one of the first areas in which these cases were diagnosed. The rapid onset of the pandemic's spreading heavily exposed HCWs to SARS-CoV-2 infection. By 28th October 2020, 39.578 HCWs were diagnosed with COVID-19 (7.3% of the total cases) [26].

Since the development of the first serological test for SARS-CoV-2 infection, testing tools have been widely used [8] and a wide range of estimates on COVID-19 prevalence among HCWs ranging from 0% to 44.7% were published across countries [16, 17, 27–29].

This study prevalence variation may reflect the non-homogeneity in SARS-CoV-2 risk for HCWs according to the considered country and its COVID-19 containment measures as well as the development of proper diagnostic algorithm, screening programmes at the workplace as well as effective contact tracing and management of index/suspected cases. At the same time, this heterogeneity might be, at least partially, attributable to the serological tests performance and to the kinetic of antibody production. There are intrinsic and extrinsic limitations in diagnostic tests that, if not fully considered, from an epidemiological and clinical viewpoint, might lead to potential biases on estimates and on medical decision-making.

The idea behind the use of three different case definitions for the study was based on the limitations reported in the literature concerning the current development of the SARS-CoV-2 testing technologies.

Therefore, assessing the impact of testing results, pathognomonic S&S and the kinetic of antibody production, when determining the SARS-CoV-2 prevalence, indicate that a test-only approach might reduce the validity of these studies. Using the data gathered in our multicentre serological study amongst HCWs, the aim was to highlight that SARS-CoV-2 serological point prevalence might result in an underestimation of the real percentage of HCWs infected by SARS-CoV-2.

In fact, when the prevalence of COVID-19 is based exclusively on the POCT results, the estimate was 6.7%, with only 59.5% of HCWs who reported a previous COVID-19 diagnosis were amongst those who received a positive POCT result. These findings are consistent with others recently reported in a large seroprevalence study carried out during COVID-19 first wave in Spain. In this study, the overall SARS-CoV-2 prevalence was found to be 11.8% (PCR and/or serological testing) and less than a third of those who resulted positive to the serological laboratory test reported a previous COVID-19 infection [16]. Whether in this study the unawareness could be related to the low proportion of workers tested with RT-PCR (COVID-19 symptoms or contact with index cases), in our study the discrepancy between those with a previous known infection and a POCT-positive result might suggest a rapid decay of antibody titres over a 4-month period. This is further corroborated by the fact that the SARS-CoV-2 infection prevalence found in our sample was lower, according to POCT result, than the one based on COVID-19 RT-PCR. In addition, considering that massive molecular screening test started in April 2020, SARS-CoV-2 infections occurred during the first 8 weeks of the pandemic might have been missed as resulted negative to both the first RT-PCR and the POCT, due to the delay in performing the test from the potential infection.

In order to further explore the potential SARS-CoV-2 prevalence in the context of this study, authors included an additional layer, that is the presence of self-reported COVID-19 pathognomonic S&S [22–25]. Although this case definition has probably low specificity, when compared to the one based on any diagnostic test, it must be considered that at the beginning of the epidemic the clinical features of SARS-CoV-2 infection were the only available criteria to be used for reaching a diagnosis. In addition, this latter clinical diagnostic approach is the only one extensively accessible in many remote areas of the world, even nowadays.

When including the HCWs who reported on the survey form COVID-19 pathognomonic S&S from February till April 2020, the prevalence was in line with the one obtained using diagnostics (9.6% and 10.0%, respectively) and higher than the one based on either POCT or RT-PCR (6.7% and 8.1%, respectively). As shown in Figure 2, the majority of positive HCWs would have been missed in the prevalence estimation if only diagnostic tests had been used.

The present study contributed to highlight at least two key issues when considering SARS-CoV-2 prevalence data among high-risk populations, such as HCWs, during the first wave of the pandemic.

First, diagnostic tests have relevant limitations [30] such as the fact that molecular tests might produce false-negative results when performed too early or too late during infection, that antigenic tests might have false-negative results in asymptomatic subjects [31] and that serological tests, mainly but not limited to POCT, have considerable risk of false-negative results both at the very beginning and months later the infection [13].

Second, COVID-19 prevalence could significantly change considering the diagnostic approach used. Indeed, serological tests are often considered as a whole, with no or minimal difference between point-of-care and laboratory tests, even if the analytical characteristics of the former are lower than those of their laboratory counterpart. As pointed out in a Cochrane systematic review, lateral flow assays have a sensitivity of at least 4% lower compared to laboratory tests, and this is not linked to the type of antibody (IgG 76%, IgM 51.4%, IgG and IgM 85.8%) [13]. Although a 4% decrease in sensitivity represents a small difference in a test, it could have a massive impact on positive predictive value (PPV) in setting with low prevalence and selected populations.

Based on the analytical characteristics of the POCT used in our study for the IgG component (sensitivity 97%, specificity 100%, as declared by the manufacturer [21]) and assuming the seroprevalence found in the Veneto Region serosurvey amongst HCWs (4.6%), the estimated predictive values were extremely high (PPV 100%, NPV 99.9%). However, based on the Cochrane systematic review previously mentioned, the analytical performance of this kind of tests should be carefully evaluated. In fact, the paper reports an average IgG/IgM sensitivity of 96% (95% CI 90.6–98.3) [13] which can have a relevant impact on the field use of these tests. Assuming that the estimated impact of the time elapsed on the antibodies detection ability is around 1% reduction, as found in our paper, reaching the above-mentioned 96% and applying the same infection prevalence, the PPV of the POCT resulted 82.2% (95% CI 39.7–97). This 18% reduction means that, amongst the HCWs who resulted positive at the POCT, only 82% (75/91 HCWs) were real positives.

A possible explanation for this performance may lay on the impact of SARS-CoV-2 antibody dynamics on serology. Antibodies are increasingly produced during the infection reaching the peak of detectability during the third week from symptoms onset [13]. Unfortunately, there are insufficient data to assess serological test sensitivity beyond the fifth week of infection [8, 13] and, therefore, as the median time from COVID-19 molecular diagnosis to POCT execution in our study was 125 days, we cannot be sure whether that 96% would remain stable or, more probably, decreases over time. In addition, our study provides a numerical estimation of the impact of time after COVID-19 diagnosis on the likelihood of having a serological positive result (−1.1% every 10 days). Although further studies might be warranted to generalise our result to other POCT serological tests and to serology in itself, this estimate might be useful in understanding SARS-CoV-2 serological data and planning interventions for public health purposes.

Other studies are needed to really understand the meaning of our paper insights and if these findings could lead to the indication that prevalence studies already published should be at least interpreted in light of these tests' limitations. For instance, the 8.7% (95% CI 6.7–10.9%) overall SARS-CoV-2 seroprevalence among HCWs, as reported by a recent meta-analysis [12], could probably represent an underestimation of the proportion of this target population who acquired COVID-19 over time.

Based on the current knowledge about the duration of SARS-CoV-2 antibodies in human blood, we can conclude that serological tests, certainly but presumably not limited to POCT, may not be the best predictors of real previous COVID-19 infections, unless carried out 3–5 weeks from the onset of infection. However, in case of seropositivity monitoring (i.e. public health case management purposes), the time interval between consecutive testing should be very limited, ideally 4 weeks, particularly when high-risk populations are considered.