The short-term effectiveness of coronavirus disease 2019 (COVID-19) vaccines among healthcare workers: a systematic literature review and meta-analysis

Abstract

Objective:

Healthcare workers (HCWs) are at risk of COVID-19 due to high levels of SARS-CoV-2 exposure. Thus, effective vaccines are needed. We performed a systematic literature review and meta-analysis on COVID-19 short-term vaccine effectiveness among HCWs.

Methods:

We searched PubMed, CINAHL, EMBASE, Cochrane Central Register of Controlled Trials, Scopus, and Web of Science from December 2019 to June 11, 2021, for studies evaluating vaccine effectiveness against symptomatic COVID-19 among HCWs. To meta-analyze the extracted data, we calculated the pooled diagnostic odds ratio (DOR) for COVID-19 between vaccinated and unvaccinated HCWs. Vaccine effectiveness was estimated as 100% × (1 − DOR). We also performed a stratified analysis for vaccine effectiveness by vaccination status: 1 dose and 2 doses of the vaccine.

Results:

We included 13 studies, including 173,742 HCWs evaluated for vaccine effectiveness in the meta-analysis. The vast majority (99.9%) of HCWs were vaccinated with the Pfizer/BioNTech COVID-19 mRNA vaccine. The pooled DOR for symptomatic COVID-19 among vaccinated HCWs was 0.072 (95% confidence interval [CI], 0.028–0.184) with an estimated vaccine effectiveness of 92.8% (95% CI, 81.6%–97.2%). In stratified analyses, the estimated vaccine effectiveness against symptomatic COVID-19 among HCWs who had received 1 dose of vaccine was 82.1% (95% CI, 46.1%–94.1%) and the vaccine effectiveness among HCWs who had received 2 doses was 93.5% (95% CI, 82.5%–97.6%).

Conclusions:

The COVID-19 mRNA vaccines are highly effective against symptomatic COVID-19, even with 1 dose. More observational studies are needed to evaluate the vaccine effectiveness of other COVID-19 vaccines, COVID-19 breakthrough after vaccination, and vaccine efficacy against new variants.

The first coronavirus disease 19 (COVID-19) vaccine was authorized for emergency use by the US Food and Drug Administration on December 11, 2020, for prevention against infection in individuals 16 years or older who are healthy or have stable chronic medical conditions and were eligible for participation in the trial. That mRNA vaccine demonstrated an efficacy of 95%. ¹ Subsequently, 8 more vaccines have been authorized for full use. ²

During the first year of the COVID-19 pandemic, healthcare workers (HCWs) were at high risk of acquiring COVID-19. ^3,4 Compared to the community, some studies have shown that frontline HCWs had >10 times higher risk of testing positive for severe acute respiratory coronavirus virus 2 (SARS-CoV-2) and that those who reported that they had inadequate access to personal protective equipment (PPE) had a 23% higher risk. ^3,4 Also, compared to HCWs reporting adequate PPE who did not care for patients with COVID-19, workers caring for patients with documented COVID-19 had a nearly 5 times higher risk of testing positive if they had adequate PPE and a nearly 6 times higher risk if they had inadequate PPE. ⁴

Over the past few months, research studies have contributed a large amount of data from different institutions on COVID-19 vaccine roll-out, making available real-world data on short-term vaccine effectiveness. ^5,6 These vaccines are effective for a wide range of COVID-19–related outcomes, a finding consistent with that of the randomized trials, ^1,7 and they show benefits in HCWs. ^8,9

We reviewed the literature on the impact of the short-term effectiveness of COVID-19 vaccines among HCWs to prevent laboratory-confirmed COVID-19. Pooling the results of published studies allows for more precise estimates of vaccine effectiveness and for subset analyses, such as evaluating the effectiveness of the vaccine against symptomatic COVID-19 and asymptomatic COVID-19 separately.

Methods

Systematic literature review and inclusion and exclusion criteria

This review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement ¹⁰ and the Meta-analysis of Observational Studies in Epidemiology (MOOSE) guidelines. ¹¹ This study was registered on Prospero (https://www.crd.york.ac.uk/PROSPERO/) on May 21, 2021 (registration no. CRD42021255589). Institutional review board approval was not required. We applied the following inclusion criteria: original research manuscripts; articles published in peer-reviewed scientific journals; studies involving vaccinated and unvaccinated HCWs; studies conducted in acute-care settings or nursing homes that evaluated the effectiveness of COVID-19 vaccine in HCWs after phase 3 clinical trials; and studies with an observational design. The literature search was limited to the period from December 2019 to June 11, 2021. Randomized clinical trials (phase 3), editorials, commentaries, and published studies from non–peer-reviewed studies (eg, MedRxiv) were excluded. Studies in which there was no comparison between vaccinated and unvaccinated HCWs, and those in which no vaccine effectiveness data were published were also excluded.

Search strategy

We performed literature searches in PubMed, Cumulative Index to Nursing and Allied Health (CINAHL), Embase (Elsevier Platform), Cochrane Central Register of Controlled Trials, Scopus (which includes EMBASE abstracts), and Web of Science. The entire search strategy is described in Supplementary Appendix 1. We reviewed the reference lists of retrieved articles to identify studies that were not identified from the preliminary literature searches. After applying exclusion criteria, we reviewed 35 papers; 16 of these met the inclusion criteria and were included in the systematic literature review (Fig. 1).

Fig. 1. Literature search for articles on COVID-19 vaccine effectiveness among healthcare workers.

Data abstraction and quality assessment

Titles and abstracts of all articles were screened to assess whether they met inclusion criteria. The reviewers (A.R.M., H.S., M.A.A., and T.K.) abstracted data from each article. Reviewers resolved disagreements by consensus.

The reviewers abstracted data on study design, population and setting, and the time (in days) of vaccination status (1 dose or 2 doses). The FDA recommends defining the COVID-19 end point as virologically confirmed SARS-CoV-2 infection accompanied by symptoms. ¹² For that reason, we defined the primary outcome as symptomatic COVID-19. For our stratified analysis, we also investigated symptomatic and asymptomatic COVID-19 combined and only asymptomatic COVID-19.

We also collected information about the incidence rate ratio (IRR), the rate reduction (RRed), the hazard ratio (HR), the relative risk (RR), the odds ratio (OR) with 95% confidence interval (CI), the vaccine effectiveness with 95% CI, and the statistical analysis performed by each included study. We also assessed the potential risk of bias for each study using the Downs and Black scale. ¹³ Reviewers followed all questions from this scale as written except for question 27 (a single item on the power subscale scored 0 to 5), which was changed to a yes or no. Also, 2 authors performed component quality analyses independently, reviewed all inconsistent assessments, and resolved disagreements by consensus. ¹⁴

Statistical analysis

To meta-analyze the extracted data, we calculated the pooled diagnostic odds ratio (DOR) with the 95% confidence interval for symptomatic COVID-19 between vaccinated and unvaccinated HCWs. Vaccine effectiveness was estimated as 100% × (1 − DOR). We also performed stratified analyses with the association between the HCW vaccination status (ie, 1 dose or 2 doses) and COVID-19 symptomatic status (ie, symptomatic, symptomatic and asymptomatic, or asymptomatic). If the study reported 2 doses, we calculated the vaccine effectiveness after the second dose. If the study reported only 1 dose, we calculated the vaccine effectiveness after the first dose. If the study reported >1 vaccine effectiveness rate with different postvaccination periods, we used the vaccine effectiveness of the longest period. We performed statistical analyses using R version 4.1.0 software with the mada package version 0.5.4. ¹⁵ Analogous to the meta-analysis of the odds ratio methods for the DOR, an estimator of random-effects model following the approach of DerSimonian and Laird is provided by the mada package. ¹⁵ For our meta-analysis of estimates of COVID-19 vaccine effectiveness, we used a bivariate random effects model, adopting a similar concept of performing the diagnostic accuracy, which enabled simultaneous pooling of sensitivity and specificity with mixed-effect linear modeling while allowing for the trade-off between them. ^16,17 Heterogeneity between studies was evaluated using I ² estimation and the Cochran Q statistic test.

Results

Characteristics of included studies

Overall, 16 studies met the inclusion criteria ^18–33 and were included in the final review (Table 1). All of these studies were nonrandomized: 8 were retrospective cohort studies, ^{18,19,21–23,26,30,33} 6 were prospective cohort studies, ^{20,25,28,29,31,32} and 2 were case–control studies. ^24,27 All of the studies evaluated the Pfizer/BioNTech mRNA COVID-19 vaccine, ^18–33 2 studies also analyzed the Moderna mRNA COVID-19 vaccine, ^27,31 and another study also analyzed the AstraZeneca COVID-19 vaccine ²⁵ (but this study was not considered in the vaccine effectiveness analysis). No study evaluated the vaccine effectiveness for the Johnson & Johnson/Janssen vaccine. Nearly all HCWs (99.9%) were vaccinated with the Pfizer/BioNTech COVID-19 mRNA vaccine. ^18–33

Table 1. Summary of Characteristics of Studies Included in the Systematic Literature Review

* Genomic investigation about the new variants: (Cavanaugh 2021: R.1 lineage variant; Jones 2021: B.1.1.7 [alpha] variant; Sansone 2021: B.1.1.7 [α] variant)

Note. S/A, symptomatic and asymptomatic; S, symptomatic; SD, standard deviation; IQR, interquartile range; IRR, incidence rate ratio; RRed, rate reduction; HR, hazard ratio; RR’, relative risk; OR, odds ratio; CI, confidence interval; VE, vaccine effectiveness; D&B, Downs and Black scale; NR, not reported, N, no. reported.

Most of the studies included in our review were conducted in the United States (6 studies) ^{21,27,28,30–32} ; 3 studies were performed in Italy ^20,22,29 ; 3 were performed in the United Kingdom ^23,25,26 ; 2 studies were performed in Israel ^18,19 ; 1 was conducted in Spain ²⁴ ; and 1 was conducted in Ireland. ³² All studies were performed between December 2020 and April 2021. ^18–33

Overall, we included 195,801 HCWs in the qualitative analysis. Moreover, 6 studies evaluated vaccine effectiveness >7 days after the second dose, ^{19,20,22,25,27,29} 3 studies evaluated vaccine effectiveness >14 days after the second dose, ^21,30,32 1 study evaluated vaccine effectiveness after HCWs received the second dose on day 21 or day 22 after the first dose, ¹⁸ and the other 6 studies did not report the time the vaccine was considered effective. ^{23,24,26,28,31,33} Of the HCWs included that received the first dose, 6 studies evaluated the effectiveness of COVID-19 vaccine >14 days after the first dose, ^{20,22,27,28,30,32} 2 studies evaluated the vaccine effectiveness >12 days after the first dose, ^24,26 1 study evaluated the vaccine effectiveness >10 days after the first dose, ²³ and 1 study evaluated >21 days after the first dose. ²⁰ Also, 1 study evaluated vaccine effectiveness from day 1 to day 14 after the first dose. ¹⁸ Another study evaluated vaccine effectiveness from day 7 to day 28 after the first dose, ²¹ and another study evaluated vaccine effectiveness up 39 days after the first dose: 0−7 days, 8−14 days, 15−21 days, 22−30 days, and 39 days. ³³ Furthermore, 3 studies did not report the period after vaccination ^27,29 and 5 studies reported asymptomatic cases in vaccinated and unvaccinated HCWs. ^{19,22,25,30,31}

The studies we reviewed varied regarding the reportage of the infection rates and the type of statistical analyses performed. To determine vaccine effectiveness, 4 studies used Poisson distribution for adjusted logistic regression, ^18,19,25,30 3 studies used adjusted regression and Cox proportional hazard models, ^22,23,32 and 3 studies used unadjusted odds ratio calculation. ^28,29,33 In addition, 2 studies used conditional logistical regression, ^24,27 2 studies used survival curves for the vaccinated and unvaccinated groups using Kaplan-Meier, ^20,31 1 study used the Fisher exact test, ²⁶ and 1 study used the risk ratio calculation to determine the vaccine effectiveness. ²¹

Among the studies we reviewed, genomic surveillance detection of the new SARS-CoV-2 B.1.1.7 or α variant ^26,29 was conducted in 2 studies, and 1 outbreak study identified a new SARS-CoV-2 variant (R.1 lineage variant). ²¹ Genomic surveillance was not performed in most of the studies we reviewed. ^{18–20,22–25,27,28,30–33}

Among the 3 studies that reported the presence of underlying medical conditions, ^25,27,32 1 study showed that 75% of participants had no underlying medical condition, ²⁵ 1 study showed that ∼70% had no medical condition but that ∼30% had at least 1 chronic condition. ³² In 1 case–control study, 75% of the case patients (symptomatic COVID-19) and the controls (non–COVID-19 patients) had at least 1 underlying condition or risk factor associated with increased risk for severe COVID-19. ²⁷ Proportions of immunocompromised HCWs were reported in only 2 studies with a rate at 2% ²⁵ to 5%. ²⁷ None of the included studies reported rates of adverse events after vaccination. ^18–33

Regarding the quality assessment scores of the 16 included studies, 9 studies were considered good quality (19–23 of 28 possible points) on the Downs and Black quality tool, ^{18–20,22,24,25,27,30,32} 5 studies were considered fair quality (14–18 points), ^{21,28,29,31,33} and 2 studies were considered poor quality (<14). ^23,26

Results pooled by each COVID-19 vaccination dose and COVID-19 status

The review included 13 studies in which 173,742 HCWs were evaluated for vaccine effectiveness and were included in the meta-analysis. ^{19–25,27–31,33} The pooled DOR for symptomatic COVID-19 among HCWs vaccinated with at least 1 dose was 0.072 (95% CI, 0.028–0.184), with an estimated the vaccine effectiveness of 92.8% (95% CI, 81.6%–97.2%). Among 13 studies, 7 studies evaluated vaccine effectiveness of 2 doses in HCWs. ^{19–22,27,29,30} The pooled DOR for this group of studies was 0.065 (95% CI, 0.024–0.175) and the estimated vaccine effectiveness was 93.5% (95% CI, 82.5%–97.6%). Also, 6 studies evaluated vaccine effectiveness of 1 dose of vaccine in HCWs. ^{19,22,27,28,30,32} The pooled DOR for these studies was 0.179 (95% CI, 0.059–0.539) and the estimated vaccine effectiveness was 82.1% (95% CI, 46.1%–94.1%).

Stratifying the analysis for studies reporting both symptomatic and asymptomatic COVID-19, 10 studies evaluated vaccine effectiveness among vaccinated HCWs who had received 1 and 2 doses. ^{19–23,25,29–31,33} The pooled DOR for this group of studies was 0.082 (95% CI, 0.030–0.223) and the estimated vaccine effectiveness was 91.8% (95% CI, 77.7−97.0%). In 3 studies evaluating vaccine effectiveness among HCWs with 2 doses, ^19,22,30 the pooled DOR was 0.035 (95% CI, 0.013−0.100) and the estimated vaccine effectiveness was 96.5% (95% CI, 90.0−98.7%). In 6 studies evaluating vaccine effectiveness among HCWs who had received only 1 dose of vaccine, ^{19,22–25,30} the pooled DOR was 0.213 (95% CI, 0.040−1.138) and the estimated vaccine effectiveness was 78.7% (95% CI, −13.8% to 96.0%). Stratifying the analysis for only asymptomatic COVID-19, 4 studies evaluated vaccine effectiveness among HCWs who had received 2 doses of vaccine. ^19,21,22,30 The pooled DOR for this group of studies was 0.089 (95% CI, 0.029−0.274) and the estimated vaccine effectiveness was 85.3% (95% CI, 47.7%−95.9%). In 3 studies evaluating vaccine effectiveness among HCWs who had received only 1 dose of vaccine, ^19,22,30 the pooled DOR for this group of studies was 0.364 (95% CI, 0.104−1.276) and the estimated vaccine effectiveness was 63.3% (95% CI, −27.6% to 89.6%).

The results of both meta-analyses were homogeneous for symptomatic COVID-19 (all studies evaluating vaccinated HCWs: heterogeneity P = .86, I ² = 0%; 2 doses: heterogeneity P = 0.70, I ² = 0%; 1 dose: heterogeneity P = 0.43, I ² = 0%). The results were homogeneous for symptomatic and asymptomatic COVID-19 (all studies evaluating vaccinated HCWs: heterogeneity P = .78, I ² = 0%; 2 doses: heterogeneity P = .49, I ² = 0%; 1 dose: heterogeneity P = .56, I ² = 0%). The results were also homogenous for only asymptomatic COVID-19 in 2 doses and 1 dose: heterogeneity P = .25, I ² = 27.0%; heterogeneity P = .43, I ² = 0%, respectively. The reasons for not including the other 3 COVID-19 vaccine HCW studies in the meta-analysis are summarized in Supplementary Appendix 2 .

Discussion

Based on studies evaluating short-term vaccine effectiveness between December 2020 to April 2021, this systematic literature review and meta-analysis showed that COVID-19 vaccines (primarily the mRNA COVID-19 vaccines) decrease symptomatic COVID-19 with a vaccine effectiveness of 92.8%. This number was comparable to vaccine effectiveness among the general population reported in the randomized trials ^1,7 and in a noncontrolled setting. ⁵ COVID-19 vaccines were also effective in reducing asymptomatic COVID-19.

Multiple vaccines are being distributed worldwide under emergency use authorizations, and additional vaccine candidates are already in phase 3 studies assessing efficacy. ³⁴ In our systematic literature review, we were only able to analyze the vaccine effectiveness for the mRNA COVID-19 vaccines (Pfizer/BioNTech and Moderna). These were the first COVID-19 vaccines authorized by the FDA, ^35,36 and HCWs were considered the priority group to receive them. ³⁷ The short duration of the studies, from 0.5 to 3 months, included in our systematic literature review among HCWs is justified particularly to understand the short-term vaccine effectiveness in the context of a global pandemic with a novel pathogen (Table 1). ³⁴ This factor also explains the wide confidence intervals (and the negative lower bound) around the vaccine effectiveness of single-dose Pfizer/BioNTech mRNA in our meta-analysis (Table 2).

Table 2. Subset Analyses Evaluating the COVID-19 Vaccine Effectiveness among Healthcare Workers (13 studies) ^a

Note. DOR, diagnostic odds ratio; HCW, healthcare worker; CI, confidence interval.

^a Reasons for not including the other 3 COVID-19 vaccine HCW studies in the meta-analysis: Amit 2021¹⁸ reported the number of exposure days; Jones 2021²⁶ reported the number of positive tests; and Thompson 2021³² reported the number of person days. Other reasons for not including studies in the stratified analysis: Bianchi 2021²⁰ did not report the total number of HCWs that received the first dose; Hall 2021²⁵ reported the number of person days for HCWs that received the second dose; Tang 2021³¹ did not report the total number of HCWs who received the first and the second dose.

^b Vaccine effectiveness was estimated as 100% × (1 − DOR).

^c Vaccinated HCWs considering any vaccination status (1 dose or 2 doses). If the study reported 2 doses, we have considered the second dose; if the study reported only 1 dose, we have considered the first dose with a longer time (eg, 0–14 days; 14–21; and ≥21 days, the last 1 was selected for the analysis).

Stratified analyses with 4 studies investigating vaccine effectiveness against asymptomatic COVID-19 also revealed high vaccine effectiveness among HCWs with 1 dose and 2 doses: 63.6% and 85.3%, respectively. ^19,21,22,30 Given that most SARS-CoV-2 is transmitted by asymptomatic individuals or prior to symptom onset in symptomatic individuals, COVID-19 vaccines might have a bigger role in preventing SARS-CoV-2 transmission than is recognized currently with reported symptomatic cases. ^37,38 Symptomatic COVID-19 is well recognized, and individuals with COVID-19 symptoms are more likely to isolate themselves, which further reduces the proportion of transmission from symptomatic individuals. The knowledge that COVID-19 vaccines are effective even in asymptomatic people could contribute to substantially reducing the transmission of SARS-CoV-2 and controlling the COVID-19 pandemic. ^38,39

Only 1 study reported an R.1 lineage variant. ²¹ This study was conducted in a nursing facility after a vaccination program and showed that vaccinated HCWs were 87% less likely to have symptomatic COVID-19 than those who were unvaccinated. Also, 2 studies performed genomic surveillance detecting the B.1.1.7 variant or α variant. ^26,29 The other studies did not include genomic surveillance. Hall et al ²⁵ reported that the HCW cohort was vaccinated when the dominant variant in circulation was B1.1.7 and showed effectiveness against this variant. Our systematic review included studies prior to the widespread circulation of the delta variant, which has contributed to most recent breakthrough infections among HCWs. ^40,41 More studies are needed regarding the SARS-CoV-2 variants of concerns (VOC) that have multiple spike protein mutations and that appear to be more infectious or cause more disease than other circulating SARS-CoV-2 variants. ⁴² Some deletions in the spike protein mutations can alter the shape of the spike and may help it evade some antibodies. ⁴³ No COVID-19 vaccine is 100% effective against SARS-CoV-2 infection, which is consistent with COVID-19 breakthrough infections reported among HCWs after COVID-19 vaccination. ^44,45

Our study had several limitations. We only included observational studies for the meta-analysis, which are subject to multiple biases ⁴⁶ ; however, this is the most common study design in the infection prevention literature. ⁴⁶ We could not investigate vaccine effectiveness of other COVID-19 vaccines due to lack of published studies. We estimated the vaccine effectiveness based on only short-term study durations, and longer-term observational studies are needed to assess sustained immune response and vaccine effectiveness. Due to the uncertainty related to the number of days required to develop immunity postvaccination, each study adopted a different definition of a fully vaccinated or partially vaccinated person. The CDC defines people fully vaccinated as being ≥14 days after the second dose in a 2-dose series (Pfizer/BioNTech or Moderna) or ≥14 days after a single dose vaccine (Johnson & Johnson/Janssen). ⁴⁷ Currently, no postvaccination time limit on fully vaccinated status has been established. In addition, the CDC defines unvaccinated people as individuals of all ages, including children who have not completed a vaccination series or received a single-dose vaccine. ⁴⁷ No consensus had been reached regarding fully vaccinated versus partially vaccinated in the included studies, and the studies used different criteria (eg, fully vaccinated for ≥7–14 days after the second dose, partially vaccinated for ≥14 days after the first dose, or just reporting the first dose available). ^18–33 None of the included studies reported information about possible adverse events after vaccine administration. For that reason, we were not able to report any evidence of severe complications and we were unable to assess whether vaccinated HCWs sought further COVID-19 testing. We could not perform further analyses stratified by immunocompromised status due to the limited studies available. We did not investigate the association between vaccine effectiveness and personal protective equipment, although vaccine effectiveness might have been affected by the PPE recommended at each institution. Because our study focused on only the short-term vaccine effectiveness among HCWs, we did not evaluate the need for the third dose. Lastly, each study used a different approach to reporting the incidence of COVID-19 (eg, incidence rate per person days and per exposure days). Therefore, we performed our meta-analysis and stratified analyses using a bivariate approach to preserve the 2-dimensional nature of the original data from the selected studies. ^{19–25,27–30,33}

In conclusion, the COVID-19 mRNA vaccines can significantly prevent symptomatic and asymptomatic COVID-19 among HCWs. The COVID-19 vaccines are also effective among HCWs, even after 1 dose. These data are very important for countries struggling to offer COVID-19 vaccines for HCWs because of limited resources. To better understand vaccine effectiveness against the new SARS-CoV-2 variants, more observational studies are needed to evaluate (1) other types of COVID-19 vaccine (eg, viral vector or inactivated virus) effectiveness, (2) the impact of personal protective equipment among HCWs on vaccine effectiveness, (3) COVID-19 breakthrough after vaccination, and (4) genomic surveillance.