The long-term effectiveness of coronavirus disease 2019 (COVID-19) vaccines: A systematic literature review and meta-analysis

Abstract

Background:

Although multiple studies revealed high vaccine effectiveness of coronavirus disease 2019 (COVID-19) vaccines within 3 months after the completion of vaccines, long-term vaccine effectiveness has not been well established, especially after the δ (delta) variant became prominent. We performed a systematic literature review and meta-analysis of long-term vaccine effectiveness.

Methods:

We searched PubMed, CINAHL, EMBASE, Cochrane Central Register of Controlled Trials, Scopus, and Web of Science from December 2019 to November 15, 2021, for studies evaluating the long-term vaccine effectiveness against laboratory-confirmed COVID-19 or COVID-19 hospitalization among individuals who received 2 doses of Pfizer/BioNTech, Moderna, or AstraZeneca vaccines, or 1 dose of the Janssen vaccine. Long-term was defined as >5 months after the last dose. We calculated the pooled diagnostic odds ratio (DOR) with 95% confidence interval for COVID-19 between vaccinated and unvaccinated individuals. Vaccine effectiveness was estimated as 100% × (1 − DOR).

Results:

In total, 16 studies including 17,939,172 individuals evaluated long-term vaccine effectiveness and were included in the meta-analysis. The pooled DOR for COVID-19 was 0.158 (95% CI: 0.157-0.160) with an estimated vaccine effectiveness of 84.2% (95% CI, 84.0- 84.3%). Estimated vaccine effectiveness against COVID-19 hospitalization was 88.7% (95% CI, 55.8%–97.1%). Vaccine effectiveness against COVID-19 during the δ variant period was 61.2% (95% CI, 59.0%–63.3%).

Conclusions:

COVID-19 vaccines are effective in preventing COVID-19 and COVID-19 hospitalization across a long-term period for the circulating variants during the study period. More observational studies are needed to evaluate the vaccine effectiveness of third dose of a COVID-19 vaccine, the vaccine effectiveness of mixing COVID-19 vaccines, COVID-19 breakthrough infection, and vaccine effectiveness against newly emerging variants.

The first coronavirus disease 2019 (COVID-19) vaccine was authorized for emergency use by the US Food and Drug Administration on December 11, 2020. ¹ Over the past several months, research studies have yielded substantial data on short-term (≤ 3 months) vaccine effectiveness ^2–4 against symptomatic COVID-19. For example, the short-term vaccine effectiveness is known to be very high at 95% for the Pfizer/BioNTech COVID-19 vaccine, 94.1% for the Moderna vaccine, 70.4% for the AstraZeneca vaccine, and 66.3% for the Janssen COVID-19 vaccine. ^1,5–7

In the third year of the pandemic, individuals are still at risk of acquiring COVID-19 even with vaccines available. ^8,9 Infection and hospitalization rates among unvaccinated individuals are 5 times and 11–29 times higher than those in vaccinated individuals, respectively. ^10,11 Also, the authorized COVID-19 vaccines protect against the δ variant, ¹² even with increased community transmission. ^10,13

Although these vaccines are effective for a wide range of COVID-19–related outcomes, ^1,6,14,15 the duration of the immune protection following the COVID-19 vaccination is still not well defined, ^16–18 and few studies have assessed the long-term vaccine effectiveness of COVID-19 vaccines.

We reviewed the literature on the long-term vaccine effectiveness of COVID-19 vaccines for COVID-19 and COVID-19 hospitalizations. Pooling the results of published studies allows for more precise estimates of the long-term vaccine effectiveness. The information provided from subset analyses during the δ variant period is significantly important given the ongoing pandemic with this variant.

Methods

Systematic literature review and inclusion and exclusion criteria

This review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement ¹⁹ and the Meta-analysis of Observational Studies in Epidemiology (MOOSE) guidelines, ²⁰ and it was registered on Prospero (https://www.crd.york.ac.uk/PROSPERO/) on September 13, 2021 (registration no. CRD42021278162). The approval of our institutional review board was not required.

The inclusion criteria for studies in this systematic review were as follows: original research manuscripts; published in peer-reviewed scientific journals; involved vaccinated and unvaccinated individuals; evaluated the long-term effectiveness of COVID-19 vaccine; and observational study design. Long-term was defined as >5 months after the second dose for mRNA (Pfizer/BioNTech or Moderna) or AstraZeneca COVID-19 vaccine, or 1 dose of Janssen COVID-19 vaccine. The literature search was limited to December 2019 to November 15, 2021. Editorials, commentaries, and published studies from non–peer-reviewed sources (eg, MedRxiv) were excluded. Studies without comparison between vaccinated and unvaccinated individuals (or other vaccinated control group), and studies without vaccine effectiveness data were also excluded.

Search strategy

We performed literature searches in PubMed, Cumulative Index to Nursing and Allied Health (CINAHL), Embase (Elsevier Platform), Cochrane Central Register of Controlled Trials, Scopus (which includes EMBASE abstracts), and Web of Science. The entire search strategy is described in Supplementary Appendix 1. We reviewed the reference lists of retrieved articles to identify studies that were not identified from the preliminary literature searches. After applying exclusion criteria, we reviewed 55 papers, 17 of which met the inclusion criteria and were included in the systematic literature review (Fig. 1).

Fig. 1. Literature search for articles on the long-term COVID-19 vaccine effectiveness among general population.

Data abstraction and quality assessment

Titles and abstracts of all articles were screened to assess whether they met inclusion criteria. The reviewers (A.R.M., B.M.T., H.S., L.M.B., M.A., M.A.A., and T.K.) abstracted data for each article. Reviewers resolved disagreements by consensus.

The reviewers abstracted data on study design, population and location, study period (months) and the calendar time, demographic and characteristics of participants, types of COVID-19 vaccine, and the date of whole-genome sequencing if available. Laboratory-confirmed COVID-19 was considered the primary outcome to calculate vaccine effectiveness after 2 doses of a COVID-19 vaccine. COVID-19 hospitalization was considered as a secondary outcome. We collected the hazard ratio (HR), the relative risk (RR), the odds ratio (OR), and vaccine effectiveness with 95% confidence intervals (CIs). We have also described the statistical analysis performed per each study to describe the estimated COVID-19 vaccine effectiveness. Risk of bias was assessed using the Downs and Black scale. ²¹ Reviewers followed all questions from this scale as written except for question 27 (a single item on the power subscale scored 0–5), which was changed to a yes or no. Two authors performed component quality analyses independently, reviewed all inconsistent assessments, and resolved disagreements by consensus. ²²

Statistical analysis

To meta-analyze the extracted data, we calculated the pooled diagnostic odds ratio (DOR) for COVID-19 or COVID-19 hospitalization between vaccinated and unvaccinated individuals. Vaccine effectiveness was estimated as 100% × (1 − DOR). We performed stratified analyses by vaccine type (eg, PfizerBioNTech COVID-19 vaccine [2 doses], Janssen COVID-19 vaccine [1 dose]), by COVID-19 status (ie, COVID-19 or COVID-19 hospitalization), and by the δ variant period. ^23–33 We performed statistical analysis using R version 4.1.0 with mada package version 0.5.4. ³⁴ Analogous to the meta-analysis of the odds ratio methods for the DOR, an estimator of random-effects model following the approach of DerSimonian and Laird is provided by the mada package. ³⁴ For our meta-analysis of estimates of COVID-19 vaccine effectiveness, we used a bivariate random effects model, adopting a similar concept of performing the diagnostic accuracy, which enables simultaneous pooling of sensitivity and specificity with mixed-effect linear modeling while allowing for the trade-off between them. ^35,36 Heterogeneity between studies was evaluated with I² estimation and the Cochran Q statistic test.

Results

Characteristics of included studies

In total, 17 studies met the inclusion criteria ^{23–33,37–42} and were included in the final review (Table 1). Almost all of these studies were nonrandomized (16 studies), and of these, 12 were retrospective cohort studies. ^{23,24,26–28,30–33,37,39,40} Also, 1 study was a prospective cohort study ²⁹ and 3 studies were case–control studies. ^25,38,42 Only 1 study was a randomized clinical trial. ⁴¹ All but 1 of these studies evaluated the Pfizer/BioNTech vaccine (16 studies). ^{23–26,28–33,37,39–42} Of these studies, 13 analyzed the Moderna vaccine ^{23,24,26,28–32,37,39,40,42} ; 7 studies analyzed the Janssen vaccine, ^{26,27,29,37–39,42} 1 of which evaluated only the Janssen vaccine ²⁷ ; and 3 studies analyzed the AstraZeneca vaccine, ^31,37,38 1 of which evaluated mixing COVID-19 vaccines. ³¹

Table 1. Summary of Characteristics of Studies Included in the Systematic Literature Review

Note. A, asymptomatic; S, symptomatic; SD, standard deviation; IQR, interquartile range; IRR, incidence rate ratio; HR, hazard ratio [HR]; RR’, relative risk; OR, odds ratio; 95% CI, 95% confidence interval; VE, vaccine effectiveness; NR, not reported; N, number reported; RCT, randomized controlled trial.

Table 2. Subset Analyses Evaluating Long-Term COVID-19 Vaccine Effectiveness Among Fully Vaccinated Individuals

Note. CI, confidence interval; mRNA, Pfizer/BioNTech and Moderna; viral vector, AstraZeneca, and Janssen.

^a Vaccine effectiveness was estimated as 100% × (1 − DOR).

Fully vaccinated is defined as receiving 2 doses of Pfizer/BioNTech, Moderna, or AstraZeneca vaccine, or 1 dose of Janssen vaccine.

Most of the studies included in our review were conducted in the United States (12 studies) ^{23,24,26–30,33,39–42} ; 1 study was a multicenter study performed in Europe (assembling data from England, France, Ireland, Netherlands, Portugal, Scotland, Spain, and Sweden) ³⁸ ; and 1 study was performed in each of these countries: Belgium, ³⁷ Qatar, ²⁵ Sweden, ³¹ and Portugal. ³² All studies were performed between December 2020 and October 2021. ^{23–33,37–42}

Moreover, 10 studies evaluated long-term vaccine effectiveness for COVID-19, ^{25–27,29–31,33,38,40,41} 8 studies evaluated long-term vaccine effectiveness for COVID-19 hospitalizations, ^{23–25,28,31,32,39,42} with 2 studies overlapping. ^25,31 The study duration varied from 5 to 14 months. ^{23–33,37–42}

Furthermore, 13 studies reported genomic surveillance data. ^{23–26,28–33,37,38} Also, 11 studies reported detecting the new SARS-CoV-2 B.1.617.2 δ (delta) variant ^23–33 ; 7 studies reported only δ variant during the long-term vaccine effectiveness evaluation ^{23,24,26,28–31} ; 2 studies reported the B.1.1.7 α (alpha) variant and δ variant ^27,32 ; 1 study reported the B.1.351 β (beta) variant and δ variant, ²⁵ and 1 study reported α, β, γ (gamma or P.1), and δ variants. ²⁵

Studies varied with regards to the type of statistical analysis performed. Nine studies used logistic regression ^{23–25,28,37–39,41,42} ; 5 studies used Cox proportional hazard analysis ^{26,29,31–33} ; 1 study used propensity matched scoring ²⁷ ; 1 study used Poisson distribution for adjusted logistic regression ³⁰ ; and 1 study used mixed-effects modeling. ⁴⁰

Regarding the quality assessment scores of the 17 included studies, >75% of the studies (13 studies) were considered good quality (ie, 19–23 of 28 possible points) per the Downs and Black quality tool. ^{23,25–29,31–33,38–40,42} Also, 3 studies were considered fair quality (ie, 14–18 points) ^24,30,37 and 1 study was considered high quality (ie, >24 points). ⁴¹

Results pooled by COVID-19 vaccine type and COVID-19 outcome

Overall, we included 17,939,172 individuals from 16 studies in the meta-analysis. ^{23–33,38–42} Among them, 10 studies evaluated the long-term vaccine effectiveness of mRNA or viral vector vaccines (ie, AstraZeneca or Janssen). ^{25–27,29–31,33,38,40,41} The estimated long-term vaccine effectiveness for COVID-19 was 84.2% (95% CI, 84.0%–84.3%). Also, 5 studies evaluated the long-term vaccine effectiveness of the Pfizer/BioNTech vaccine, ^{25,26,30,33,41} and 2 studies evaluated the Moderna vaccine. ^26,30 The estimated long-term vaccine effectiveness against COVID-19 of the Pfizer/BioNTech COVID-19 vaccine was 81.5% (95% CI, 81.3%–81.6%). Furthermore, 4 studies evaluated vaccine effectiveness of the mRNA or viral vector vaccines during the δ variant period ^25,29,30,40 ; 2 studies evaluated vaccine effectiveness of the Pfizer/BioNTech COVID-19 vaccine only ^25,30 , and 2 studies reported vaccine effectiveness of the Moderna COVID-19 vaccine only. ^25,30 The estimated long-term vaccine effectiveness for COVID-19 with mRNA or viral vector vaccines during the δ variant–dominant period was 61.2% (95% CI, 59.0%–63.3%).

Among the 16 studies, 8 studies evaluated the long-term vaccine effectiveness of mRNA or viral vector vaccines for COVID-19 hospitalization. ^{23–25,28,31,32,39,42} The estimated long-term vaccine effectiveness against COVID-19 was 88.7% (95% CI, 55.8%–97.1%). In stratified analyses, 6 studies evaluated long-term vaccine effectiveness for COVID-19 hospitalization with the Pfizer/BioNTech vaccine, ^{23–25,28,39,42} and 5 studies with the Moderna vaccine. ^{23,24,28,39,42} The estimated long-term vaccine effectiveness for COVID-19 hospitalization with the Pfizer/BioNTech vaccine was 85.4% (95% CI, 84.8%–86.0%). The estimated long-term vaccine effectiveness for COVID-19 hospitalization with the Moderna vaccine was 89.8% (95% CI, 89.2%–90.4%). Only 1 study evaluated COVID-19 hospitalization during the δ variant period with mRNA vaccines. ²⁴ This study did not report the COVID-19 vaccine effectiveness but reported that the adjusted odds of COVID-19 was higher among unvaccinated and previously infected patients compared with fully vaccinated individuals (adjusted odds ratio, 5.49; 95% CI, 2.75–10.99). ²⁴

The results of meta-analyses were homogeneous for COVID-19 with mRNA or viral vector vaccines (heterogeneity P = .76; I² = 0%); studies evaluating individuals vaccinated with the Pfizer/BioNTech vaccine alone (heterogeneity P = .55; I² = 0%); and studies evaluating individuals vaccinated with mRNA or viral vector vaccines during the δ variant period (heterogeneity P = .50; I² = 0%).

Meta-analysis results were also homogeneous for COVID-19 hospitalization (studies evaluating individuals vaccinated with mRNA or viral vector vaccines (heterogeneity P = .67; I² = 0%); and studies evaluating individuals vaccinated with the Moderna vaccine alone (heterogeneity P = .28; I² = 20%). However, results were not homogenous for studies of COVID-19 hospitalization only evaluating individuals vaccinated with the Pfizer/BioNTech vaccine alone (heterogeneity P = .07; I² = 51%) or for studies of COVID-19 hospitalization only evaluating individuals vaccinated with the Moderna vaccine alone (heterogeneity P = .21; I² = 32%).

Discussion

This systematic literature review and meta-analysis showed that the long-term of vaccine effectiveness with COVID-19 vaccines (primarily the mRNA vaccines) for COVID-19 and COVID-19 hospitalization were high at 84.2% and 88.7%, respectively. However, the long-term vaccine effectiveness against COVID-19 during the δ-variant–dominant period was lower at 61.2%. These results suggest that 2 doses of the COVID-19 vaccine may lose effectiveness after a few months, and more prospective studies are needed to investigate the short- and long-term vaccine effectiveness after the third dose of the COVID-19 vaccines.

A growing body of early global research shows that the authorized COVID-19 vaccines remain highly protective against the disease’s worst outcomes over time with some exceptions among older and immunocompromised people. ^43,44 In our systematic literature review, we analyzed only the estimated pooled vaccine effectiveness for the mRNA COVID-19 vaccines and the viral vector COVID-19 vaccines. These are the first COVID-19 vaccines authorized by the FDA and around the world, ^45–48 and they prevent COVID-19 and COVID-19 hospitalization. ^{2,4,10,12,15,49} The long duration of the studies (from 5 to 14 months, as shown in Table 1) included in our systematic literature review helps to better elucidate the long-term vaccine effectiveness in the context of a global pandemic with new SARS-CoV-2 variants ^12,13 and to better understand that the decrease of vaccine effectiveness is associated with a waning of humoral immune response after a few months. ^13,17 Although the overall long-term vaccine effectiveness against COVID-19 and COVID-19 hospitalization were moderately high (∼80%), a number of published studies demonstrated significantly lower vaccine effectiveness (∼60%) during the δ-variant period. ^{25,26,29,30,39,41}

Our systematic review included 11 studies evaluating the widespread circulation of the δ variant contributing to the majority of recent COVID-19 and COVID-19 hospitalizations. ^23–33 The studies in this systematic review antedate the emergence of the B.1.1.529 (omicron) variant announced by the World Health Organization (WHO) on November 26, 2021. ⁵⁰ We need more studies on the SARS-CoV-2 variants of concerns (VOC) that have multiple spike-protein changes and that may be more infectious or cause more severe disease than other circulating variants. ⁵¹ Some deletions in the spike-protein gene can alter the shape of the spike and may help it evade antibodies. ⁵² No COVID-19 vaccine is 100% effective against SARS-CoV-2 infection, as demonstrated by breakthrough infections, ^8,53 but they are highly effective at preventing severe disease and death. ²⁵ Although the long-term vaccine effectiveness was not as high as the short-term vaccine effectiveness, it is not clear whether the waning of immunity is due to the passage of time or the coincident spread of the δ variant (from June to September 2021). ^23–33

Our study had several limitations. Most of the studies included in the meta-analysis were observational studies, which are subject to multiple biases. ⁵⁴ However, this is the most common study design in the infection prevention literature. ⁵⁴ None of the included studies reported possible adverse events after vaccine administration. We could not perform further analyses stratified by immunocompromised status due to the limited number of studies. Only 1 study compared immunocompromised individuals to immunocompetent individuals and reported that the effectiveness of mRNA vaccination against COVID-19 hospitalization was lower (77%) among immunocompromised individuals than among immunocompetent individuals (90%). ²⁸ Because our study focused on the long-term vaccine effectiveness after the second dose, we could not evaluate the impact of a third dose. Because of the low number of included studies of viral vector vaccines, it was not possible to perform a stratified analysis for these. It was not possible to evaluate the long-term vaccine effectiveness of the Moderna vaccine against COVID-19 because there were not enough studies. ^26,30 There are not enough studies comparing each 1 of the 2 mRNA vaccines to draw conclusions about the vaccine effectiveness for COVID-19 during the δ variant dominant period. ^25,30 Also, it was not possible to evaluate the COVID-19 hospitalization vaccine effectiveness during the δ-variant–dominant period. It was not possible to make any conclusions about the long-term vaccine effectiveness of mixing vaccines because just 1 study assessed this. ³¹ From that study, mixing COVID-19 vaccines (first dose with the AstraZeneca vaccine adding a mRNA prime-boost showed a higher vaccine effectiveness (68%) than that of 2 doses of AstraZeneca vaccine (50%). ³¹ Lastly, each study used a different approach to report the incidence of COVID-19 (eg, incidence rate per person years). Therefore, we decided to perform our meta-analysis and stratified analysis with a bivariate approach to preserve the 2-dimensional nature of the original data from the selected studies. ^{23–33,38–42}

In conclusion, COVID-19 vaccines can effectively prevent COVID-19 and COVID-19 hospitalization for a relatively long period. These vaccines are also effective in preventing COVID-19 during the δ-variant period, though vaccines were less effective. These data are very important to help motivate individuals to seek vaccination. More observational studies are needed to evaluate other types of COVID-19 vaccine (eg, viral vector or inactivated virus) effectiveness, vaccine effectiveness of a third dose, vaccine effectiveness of mixing COVID-19 vaccines, COVID-19 breakthrough infection after vaccination, and genomic surveillance for better understanding vaccine effectiveness against the new viral variants.