Introduction
Vitamin A supplementation (VAS) is a highly cost-effective public health intervention that reaches approximately 250 million children every year, protecting them from blindness and decreasing their risk of mortality from preventable causes(Reference Imdad, Mayo-Wilson and Herzer1). In settings where vitamin A deficiency (VAD) is a public health problem, the World Health Organization (WHO) recommends two high-dose vitamin A supplements annually, spaced 4–6 months apart, for children aged 6–59 months(2,3) . UNICEF estimates that VAD affected about one-third of children aged 6–59 months in 2018, with the highest rates in sub-Saharan Africa (48 %) and South Asia (44 %)(4).
Since 1998, VAS has been delivered mainly through campaigns, such as polio supplementary immunization activities (SIAs)(3). Recently, the high cost of campaign delivery and the reduced frequency and geographical distribution of polio-SIAs due to effective eradication efforts, have encouraged countries to increasingly use routine health services for VAS delivery(4). This change in delivery mechanism has coincided with a shift in programme monitoring tools, with many countries moving from paper-based to electronic-based administrative monitoring systems(4).
The main indicator for VAS programme monitoring is VAS coverage, defined as the percentage of children aged 6–59 months of age receiving an age-appropriate vitamin A supplement in each of two annual semesters(5,Reference Janmohamed and Doledec6) .
In settings where VAS is distributed through campaigns, there is existing guidance on how to measure and validate VAS coverage after an event using survey methods, including the Post Event Coverage Survey (PECS)(7), which employs the Expanded Programme on Immunization (EPI) cluster survey methodology(8). However, this technique was revised by WHO in response to methodological concerns(9). The main changes brought by the 2018 WHO Vaccination Coverage Cluster Surveys Reference Manual include the use of probability-based sampling methods at each stage; households (HHs) selected by a central group of planners rather than interviewers in the field; interview of every eligible child in the HH; and weighted analysis.
Other tools used to measure VAS coverage are represented by large-scale multi-topic HH surveys, such as the Demographic and Health Survey(Reference Corsi, Neuman and Finlay10) or the UNICEF Multiple Indicator Cluster Survey(11). However, such HH surveys have limitations in supporting VAS programme management needs, as they are not designed to measure annual two-dose VAS coverage; moreover, they are expensive and carried out too infrequently (i.e. every 10 years) to allow a real-time monitoring of VAS coverage aimed to identify and implement corrective actions (i.e. supplementary VAS activities in specific areas).
At present, there is no standardised methodology for measuring and validating the coverage of VAS delivered through routine health services. Strengthened methods are therefore required for accurate and timely measurement of VAS coverage. This is particularly important as countries integrate VAS into the routine health systems and shift to administrative, electronic-based monitoring tools, a process that brings certain limitations affecting the quality of administrative VAS coverage, which can impair effective VAS programme monitoring(5,Reference Janmohamed and Doledec6) .
To address this gap, we conducted a systematic review of the literature to identify and recommend methods to measure VAS coverage, with the aim of providing guidance to countries on the collection of consistent data for planning, monitoring and evaluating VAS programmes integrated into routine health systems.
Methods
The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines(Reference Moher, Liberati and Tetzlaff12). The protocol for the review was not registered on the PROSPERO register of systematic reviews but is available on request.
We searched the PubMed®, Embase®, Scopus, Google Scholar and WHO Global Index Medicus databases for peer-reviewed studies reporting original data on VAS coverage among children under 5 years of age and methodologies used for measurement.
To focus on methods currently in use, we only included articles published from 1 January 2000 to 1 January 2021. Studies written in languages other than English, French, Portuguese or Spanish were excluded.
We used a combination of medical subject headings (MeSH) and text words, Boolean operators and synonyms in the thesaurus to create database-appropriate syntax (Table 1).
Table 1. PubMed® search strategy used in the systematic review of methodologies to measure Vitamin A Supplementation Coverage
Note: Search strategies for other databases used (Embase®, Scopus, Google Scholar and WHO Global Index Medicus) are available from the corresponding author.
As vitamin A supplement is delivered together with other health interventions, as immunisations, deworming and other parasite control programmes, the search strategy also includes such-related terms.
Bibliographic information was imported into a citation bibliographic management software for the storage and removal of duplicates. After duplicate citations were removed, titles and abstracts were independently screened for eligibility by authors. The reference lists of relevant articles were also checked to identify further eligible studies. In cases of disagreement, consensus was sought after reading the full-text article.
As all the studies included in the systematic review adopted a cross-sectional design, quality was assessed by two authors using the Newcastle–Ottawa Scale adapted for cross-sectional studies(Reference Modesti, Reboldi and Cappuccio13). Disagreements in quality assessment were resolved through discussion.
Authors extracted the data using an electronic form. The summary of findings tables accompanied by a narrative synthesis was used to synthetise and present results.
The data collected included bibliographic information (authors, year and country of publication); study design; sample size and sampling procedures; data collection methods; data quality assurance methods; data analysis methods; ethical considerations; planning considerations (e.g. number of personnel, study length and month of study implementation) and outcome measured (routine/after event one-dose and/or two-dose VAS coverage). The terminology used in the systematic review is provided in the glossary of terms (Supplementary material 1).
Results
We identified 3325 abstracts through database searches. After removing duplicates and screening out non-relevant abstracts, we assessed twenty-seven full-text articles for eligibility. Of these, eighteen studies that met the selection criteria were included in the systematic review (Fig. 1)(Reference Bharmal and Omair14–Reference Kassa, Mesfin and Gebremedhin31).
Fig. 1. Flowchart of the selection of studies included in the systematic review of methodologies to measure the coverage of vitamin A supplementation (VAS).
Fifteen studies were conducted in the WHO Africa Region(Reference Bharmal and Omair14–Reference Ayoya, Bendech and Baker17,Reference Gebremedhin, Loha and Abebe19–Reference Sesay, Hodges and Kamara25,Reference Ouédraogo, Becquey and Wilson27–Reference Kassa, Mesfin and Gebremedhin31) and three in the WHO South-East Asia Region(Reference Masanja, Schellenberg and Mshinda15,Reference Sachdeva and Datta18,Reference Mahajan, Srivastav and Mukherjee26) during the period 2001–2020. When appraised for quality, only five studies were categorised as high quality(Reference Gebremedhin, Loha and Abebe19,Reference Dhillon C, Subramaniam and Mulokozi21,Reference Sesay, Hodges and Kamara25,Reference Adamu and Muhammad29,Reference Koroma, Conteh and Bah30) (Table 2).
Table 2. Quality appraisal of the eighteen studies included in the systematic review of methodologies to measure vitamin A supplementation coverage – Newcastle–Ottawa Scale (adapted for cross-sectional studies)(11)
Quality threshold: high quality: eight to ten stars; medium quality: five to seven stars; low quality: zero to four stars.
Newcastle–Ottawa Quality Assessment Scale (adapted for cross-sectional studies)
Selection: (Maximum five stars)
(1) Representativeness of the sample:
(a) Truly representative of the average in the target population. * (all subjects or random sampling)
(b) Somewhat representative of the average in the target population. * (non-random sampling)
(c) Selected group of users.
(d) No description of the sampling strategy.
(2) Sample size:
(a) Justified and satisfactory. *
(b) Not justified.
(3) Non-respondents:
(a) Comparability between respondents and non-respondents characteristics is established, and the response rate is satisfactory. *
(b) The response rate is unsatisfactory, or the comparability between respondents and non-respondents is unsatisfactory.
(c) No description of the response rate or the characteristics of the responders and the non-responders.
(4) Ascertainment of the exposure (risk factor):
(a) Validated measurement tool. **
(b) Non-validated measurement tool, but the tool is available or described.*
(c) No description of the measurement tool.
Comparability: (Maximum two stars)
(1) The subjects in different outcome groups are comparable, based on the study design or analysis. Confounding factors are controlled.
(a) The study controls for the most important factor (select one). *
(b) The study control for any additional factor. *
Outcome: (Maximum three stars)
(1) Assessment of the outcome:
(a) Independent blind assessment. **
(b) Record linkage. **
(c) Self-report. *
(d) No description.
(2) Statistical test:
(a) The statistical test used to analyse the data is clearly described and appropriate, and the measurement of the association is presented, including confidence intervals and the probability level (P value). *
(b) The statistical test is not appropriate, not described or incomplete.
Sixteen studies employed a coverage cluster survey (CCS) design(Reference Bharmal and Omair14–Reference Mahajan, Srivastav and Mukherjee26,Reference Hadzi, Asalu and Avedzi28,Reference Adamu and Muhammad29,Reference Kassa, Mesfin and Gebremedhin31) , one used a Longitudinal Cluster Survey (LCS) methodology(Reference Ouédraogo, Becquey and Wilson27) and one a clustered lot quality assurance sampling (CLQAS) approach(Reference Koroma, Conteh and Bah30). Half of the studies (n 9) were conducted at the country level(Reference Masanja, Schellenberg and Mshinda15–Reference Ayoya, Bendech and Baker17,Reference Hodges, Sesay and Kamara20–Reference Sesay, Hodges and Kamara25) and the other half were conducted at the subnational level (e.g. city, district or region). The majority of articles included (n 16) enrolled the whole VAS target age group of children aged 6–59 months(Reference Bharmal and Omair14–Reference Ayoya, Bendech and Baker17,Reference Gebremedhin, Loha and Abebe19–Reference Dhillon C, Subramaniam and Mulokozi21,Reference Sangho, Belemou and Keita23–Reference Kassa, Mesfin and Gebremedhin31) .
Sample size and sampling procedures
Overall, six studies(Reference Bharmal and Omair14,Reference Bendech, Cusack and Konaté16,Reference Sachdeva and Datta18,Reference Hodges, Sesay and Kamara20,Reference Clohossey, Katcher and Mogonchi24,Reference Ouédraogo, Becquey and Wilson27) did not report procedures employed to calculate the sample size (Table 3).
Table 3. Sample size procedures and characteristics of studies included in the systematic review of methodologies to measure the coverage of VAS (n 18)
CCS, coverage cluster survey; LCS, longitudinal cluster survey; CLQAS, clustered lot quality assurance sampling survey; NR, not reported; VAS, vitamin A supplementation.
Anticipated VAS coverage was considered by nine studies(Reference Masanja, Schellenberg and Mshinda15,Reference Gebremedhin, Loha and Abebe19,Reference Babaniyi, Siziya and Mukonka22,Reference Sangho, Belemou and Keita23,Reference Sesay, Hodges and Kamara25,Reference Hadzi, Asalu and Avedzi28,Reference Adamu and Muhammad29,Reference Kassa, Mesfin and Gebremedhin31) , which together with other two studies(Reference Dhillon C, Subramaniam and Mulokozi21,Reference Koroma, Conteh and Bah30) defined a desired precision ranging between ±5 and 10 %. A confidence level of 95 % was set by twelve studies(Reference Masanja, Schellenberg and Mshinda15,Reference Ayoya, Bendech and Baker17,Reference Gebremedhin, Loha and Abebe19,Reference Dhillon C, Subramaniam and Mulokozi21–Reference Sangho, Belemou and Keita23,Reference Sesay, Hodges and Kamara25,Reference Mahajan, Srivastav and Mukherjee26,Reference Hadzi, Asalu and Avedzi28–Reference Kassa, Mesfin and Gebremedhin31) .
Six articles(Reference Babaniyi, Siziya and Mukonka22,Reference Sangho, Belemou and Keita23,Reference Mahajan, Srivastav and Mukherjee26,Reference Hadzi, Asalu and Avedzi28,Reference Adamu and Muhammad29,Reference Kassa, Mesfin and Gebremedhin31) considered the design effect (DEFF) in the sample size calculation process, using a value between 1⋅5 and 5⋅0 (mean 2⋅4; median 2⋅0).
Only four studies(Reference Mahajan, Srivastav and Mukherjee26,Reference Hadzi, Asalu and Avedzi28,Reference Adamu and Muhammad29,Reference Kassa, Mesfin and Gebremedhin31) further increased the sample size for an estimated non-response rate, which ranged between 5 and 10 % (mean and median 7⋅5 %).
There was no relation between the sample dimension, the geographical level where studies were conducted and their design: CCs reported a sample size ranging between 210 units at the municipality level(Reference Ayoya, Bendech and Baker17) to 5880 units at the national level(Reference Sesay, Hodges and Kamara25), the CLQAS(Reference Koroma, Conteh and Bah30) and LCS(Reference Ouédraogo, Becquey and Wilson27) sampled 855 and 10 454 units at the district level, respectively.
Concerning sampling procedures (Table 4), all the included studies used a multistage cluster sampling design. At the first stage, the majority of studies (n 11)(Reference Bharmal and Omair14,Reference Masanja, Schellenberg and Mshinda15,Reference Ayoya, Bendech and Baker17–Reference Gebremedhin, Loha and Abebe19,Reference Mahajan, Srivastav and Mukherjee26–Reference Kassa, Mesfin and Gebremedhin31) selected a subnational administrative division (i.e. districts, provinces and regions) by convenience, and within such strata, most (n 15)(Reference Masanja, Schellenberg and Mshinda15,Reference Bendech, Cusack and Konaté16,Reference Sachdeva and Datta18–Reference Mahajan, Srivastav and Mukherjee26,Reference Hadzi, Asalu and Avedzi28–Reference Kassa, Mesfin and Gebremedhin31) sampled clusters with probability proportional to their size (PPS). Seven studies(Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19,Reference Hodges, Sesay and Kamara20,Reference Babaniyi, Siziya and Mukonka22,Reference Sangho, Belemou and Keita23,Reference Sesay, Hodges and Kamara25,Reference Kassa, Mesfin and Gebremedhin31) defined clusters as enumeration areas (EAs) and five(Reference Dhillon C, Subramaniam and Mulokozi21,Reference Babaniyi, Siziya and Mukonka22,Reference Sesay, Hodges and Kamara25,Reference Koroma, Conteh and Bah30) used the segmentation technique for cluster selection.
Table 4. Sampling procedures used by studies included in the systematic review of methodologies to measure the coverage of vitamin A supplementation (n 18)
CCS, coverage cluster survey; EAs, census enumeration areas; HHs, households; LCS, longitudinal cluster survey; NA, not applicable; NR, not reported; PPS, probability proportional to size; WHO, World Health Organization.
Within the selected clusters, all but one study(Reference Kassa, Mesfin and Gebremedhin31) sampled HHs to find eligible children, with the majority (n 12)(Reference Masanja, Schellenberg and Mshinda15–Reference Clohossey, Katcher and Mogonchi24,Reference Mahajan, Srivastav and Mukherjee26,Reference Adamu and Muhammad29) using the WHO random walk method. Five articles(Reference Babaniyi, Siziya and Mukonka22,Reference Sangho, Belemou and Keita23,Reference Ouédraogo, Becquey and Wilson27–Reference Adamu and Muhammad29) reported that all eligible children within a selected HH were enrolled, while the other thirteen studies reported selecting only one eligible child by random selection. In all the reviewed studies, parents or caregivers were interviewed to collect information about VAS received.
Overall, the number of selected clusters per strata ranged from four to eighty, with eight studies(Reference Masanja, Schellenberg and Mshinda15,Reference Bendech, Cusack and Konaté16,Reference Sachdeva and Datta18,Reference Hodges, Sesay and Kamara20,Reference Dhillon C, Subramaniam and Mulokozi21,Reference Clohossey, Katcher and Mogonchi24–Reference Mahajan, Srivastav and Mukherjee26) opting for thirty clusters. The number of sampled HHs per cluster ranged from seven to forty-five (mean 22⋅5; median 20).
Data collection and quality assurance
The data collection and quality assurance procedures used in the reviewed studies are presented in Table 5. Only two studies(Reference Hadzi, Asalu and Avedzi28,Reference Koroma, Conteh and Bah30) used an electronic tool to collect information on the number of vitamin A supplements received by enrolled children, while the other studies used a standardised questionnaire, which was pre-tested and translated in the local language in thirteen studies(Reference Bharmal and Omair14,Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19–Reference Sesay, Hodges and Kamara25,Reference Ouédraogo, Becquey and Wilson27,Reference Adamu and Muhammad29–Reference Kassa, Mesfin and Gebremedhin31) .
Table 5. Data collection and quality assurance procedures adopted by studies included in the systematic review of methodologies to measure the coverage of vitamin A supplementation (n 18)
HHs, households.
As evidence of VAS, all the included studies collected information using parental recall when the child health card (CHC) was not available. In addition, a sample of VAS capsules was shown to parents to reduce recall biases in eleven articles(Reference Masanja, Schellenberg and Mshinda15–Reference Ayoya, Bendech and Baker17,Reference Gebremedhin, Loha and Abebe19–Reference Babaniyi, Siziya and Mukonka22,Reference Clohossey, Katcher and Mogonchi24,Reference Ouédraogo, Becquey and Wilson27,Reference Koroma, Conteh and Bah30,Reference Kassa, Mesfin and Gebremedhin31) .
Ten studies(Reference Bharmal and Omair14–Reference Ayoya, Bendech and Baker17,Reference Gebremedhin, Loha and Abebe19,Reference Hodges, Sesay and Kamara20,Reference Sesay, Hodges and Kamara25,Reference Ouédraogo, Becquey and Wilson27,Reference Koroma, Conteh and Bah30,Reference Kassa, Mesfin and Gebremedhin31) provided 2 to 5 days of training to interviewers, and nine conducted field supervision(Reference Bharmal and Omair14,Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19,Reference Dhillon C, Subramaniam and Mulokozi21,Reference Sangho, Belemou and Keita23–Reference Sesay, Hodges and Kamara25,Reference Ouédraogo, Becquey and Wilson27,Reference Koroma, Conteh and Bah30) .
The data collected were double-checked and cleaned for analysis in twelve studies(Reference Bharmal and Omair14–Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19–Reference Babaniyi, Siziya and Mukonka22,Reference Clohossey, Katcher and Mogonchi24,Reference Sesay, Hodges and Kamara25,Reference Ouédraogo, Becquey and Wilson27,Reference Adamu and Muhammad29,Reference Kassa, Mesfin and Gebremedhin31) . Only one study(Reference Bendech, Cusack and Konaté16) reported having a plan to replace or revisit empty HHs and clusters that had suddenly become inaccessible.
Ethical and planning consideration
Verbal consent was requested from parents of eligible children in fifteen studies(Reference Bharmal and Omair14–Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19–Reference Clohossey, Katcher and Mogonchi24,Reference Mahajan, Srivastav and Mukherjee26–Reference Kassa, Mesfin and Gebremedhin31) and twelve studies(Reference Masanja, Schellenberg and Mshinda15,Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19–Reference Dhillon C, Subramaniam and Mulokozi21,Reference Sangho, Belemou and Keita23,Reference Mahajan, Srivastav and Mukherjee26–Reference Kassa, Mesfin and Gebremedhin31) also obtained ethical clearance from an ethics committee (Table 6).
Table 6. Planning and ethical considerations of studies included in the systematic review of methodologies to measure the coverage of vitamin A supplementation (n 18)
NR, not reported.
Information on the number of personnel employed and the length to complete the planned sample size was reported in eleven articles(Reference Bharmal and Omair14,Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19–Reference Dhillon C, Subramaniam and Mulokozi21,Reference Sangho, Belemou and Keita23–Reference Ouédraogo, Becquey and Wilson27,Reference Koroma, Conteh and Bah30) , with an average study period of 22⋅5 d (range: 5–60 d). The number of interviewers ranged from ten (for a sample size of 900 units completed in 1 week) to fifty-three (for a sample size of 4480 units completed in 5 d). Interviewers were locally recruited and combined in teams of two, with each team completing an average of 29 HH/d. The number of supervisors varied widely. One study(Reference Dhillon C, Subramaniam and Mulokozi21) reported that gender balance had been ensured in each team to deal with local customs, while another study(Reference Masanja, Schellenberg and Mshinda15) had recruited a statistician to support sample size estimation, sampling procedures and data analysis.
Data analysis and outcome measured
Table 7 shows the data analysis procedures reported in the reviewed articles. Overall, eleven articles(Reference Masanja, Schellenberg and Mshinda15,Reference Ayoya, Bendech and Baker17,Reference Gebremedhin, Loha and Abebe19,Reference Dhillon C, Subramaniam and Mulokozi21–Reference Sangho, Belemou and Keita23,Reference Sesay, Hodges and Kamara25,Reference Hadzi, Asalu and Avedzi28–Reference Kassa, Mesfin and Gebremedhin31) reported 95 % confidence intervals (95 % CI) for estimated VAS coverage, only four articles(Reference Masanja, Schellenberg and Mshinda15,Reference Hodges, Sesay and Kamara20,Reference Clohossey, Katcher and Mogonchi24,Reference Koroma, Conteh and Bah30) calculated weights for analysis to account for differences in population size within the sampled clusters and only three(Reference Masanja, Schellenberg and Mshinda15,Reference Dhillon C, Subramaniam and Mulokozi21,Reference Babaniyi, Siziya and Mukonka22) adjusted VAS coverage estimates for non-response rate. Most studies (n 13)(Reference Masanja, Schellenberg and Mshinda15,Reference Hodges, Sesay and Kamara20–Reference Kassa, Mesfin and Gebremedhin31) reported VAS coverage by the age group (6–11 and 12–59 months) and two(Reference Ayoya, Bendech and Baker17,Reference Koroma, Conteh and Bah30) also by the collection method. Additionally, the CLQAS(Reference Koroma, Conteh and Bah30) reported the number of lots that passed the defined VAS coverage threshold.
Table 7. Data analysis procedures and outcome measured by studies included in the systematic review of methodologies to measure the coverage of Vitamin A Supplementation (n 18)
CI, confidence interval; NA, not applicable; NR, not reported; VAS, vitamin A supplementation.
Eight studies(Reference Masanja, Schellenberg and Mshinda15,Reference Bendech, Cusack and Konaté16,Reference Sachdeva and Datta18,Reference Gebremedhin, Loha and Abebe19,Reference Babaniyi, Siziya and Mukonka22,Reference Mahajan, Srivastav and Mukherjee26,Reference Koroma, Conteh and Bah30,Reference Kassa, Mesfin and Gebremedhin31) measured one-dose routine VAS coverage. Of these, six(Reference Bendech, Cusack and Konaté16,Reference Sachdeva and Datta18,Reference Babaniyi, Siziya and Mukonka22,Reference Mahajan, Srivastav and Mukherjee26,Reference Koroma, Conteh and Bah30,Reference Kassa, Mesfin and Gebremedhin31) were carried out during the June–September period to estimate the proportion of children who received one vitamin A supplement in the first semester of the year.
Among studies measuring two-dose VAS coverage(Reference Sangho, Belemou and Keita23,Reference Ouédraogo, Becquey and Wilson27–Reference Adamu and Muhammad29) , two(Reference Sangho, Belemou and Keita23,Reference Hadzi, Asalu and Avedzi28) were conducted during the December–February period to estimate the proportion of children who had received two vitamin A supplements in the previous year. In the LCS(Reference Ouédraogo, Becquey and Wilson27), children who received a first dose of vitamin A in the first semester of the year were followed up in the second semester to assess whether they had received a second dose.
Discussion
The systematic review revealed that across regions and time, CCS represented the principal method for measuring and validating VAS coverage, both after a vitamin A event distribution and via routine health contacts.
The majority of studies reviewed adapted the methodology of the WHO EPI cluster survey(8), modifying sample size, sampling and data analysis procedures. In making such modifications, these studies partly implemented the new WHO recommendations contained in the 2018 Vaccination Coverage Cluster Surveys Reference Manual(9).
In fact, over time, EPI surveys have increased in complexity, matching the evolution of the EPI since its inception in 1974 with the so-called ‘30 × 7 design’(7).
Although the basic 30 × 7 EPI survey design has been a valuable programme management tool, the use of non-probability sampling and lack of standardised, well-documented quality control procedures may reduce confidence in the results(Reference Danovaro-Holliday, Dansereau and Rhoda32). To address these limitations, the WHO Vaccination CCS Reference Manual was updated in 2005(8) and again in 2018(9) and is still considered the standard guidance for conducting a CCS.
To calculate sample size, most studies used anticipated VAS coverage, a desired precision of ±5–10 %, a confidence level of 5 %, an average DEFF of 2⋅5 and a predefined number of clusters and respondents per cluster. In addition to these WHO-recommended parameters(8,9) , sample size should also be increased for an estimated non-response rate. However, this last parameter was only considered by four studies included in the systematic review(Reference Mahajan, Srivastav and Mukherjee26,Reference Hadzi, Asalu and Avedzi28,Reference Adamu and Muhammad29,Reference Kassa, Mesfin and Gebremedhin31) .
Rather than pre-establishing a certain number of clusters and HHs, the 2018 WHO Manual(9) recommends that at least thirty clusters be selected per stratum of a minimum of ten respondents each. This was done in a majority of reviewed studies.
Most studies (n 15)(Reference Masanja, Schellenberg and Mshinda15,Reference Bendech, Cusack and Konaté16,Reference Sachdeva and Datta18–Reference Mahajan, Srivastav and Mukherjee26,Reference Hadzi, Asalu and Avedzi28–Reference Kassa, Mesfin and Gebremedhin31) also selected clusters with PPS that ensure representativeness, as larger units that represent a greater proportion of the population are more likely to be sampled(9). Another advantage of PPS sampling is that it reduces variation among sampling weights, which reduces confidence interval width for coverage estimates.
Cluster segmentation was considered by five studies(Reference Dhillon C, Subramaniam and Mulokozi21,Reference Babaniyi, Siziya and Mukonka22,Reference Clohossey, Katcher and Mogonchi24,Reference Sesay, Hodges and Kamara25,Reference Koroma, Conteh and Bah30) ; this technique is recommended to optimise resources when there are large clusters that have many more HHs than needed(8,9) .
Consistent with WHO recommendations(9), seven studies(Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19,Reference Hodges, Sesay and Kamara20,Reference Babaniyi, Siziya and Mukonka22,Reference Clohossey, Katcher and Mogonchi24,Reference Sesay, Hodges and Kamara25,Reference Kassa, Mesfin and Gebremedhin31) defined clusters as EAs, which represent the smallest defined geographical units created for the enumeration purposes of the census and may already have maps and defined boundaries.
While most studies (n 12)(Reference Masanja, Schellenberg and Mshinda15–Reference Clohossey, Katcher and Mogonchi24,Reference Mahajan, Srivastav and Mukherjee26,Reference Hadzi, Asalu and Avedzi28) employed the WHO random walk method for HH selection, this approach can introduce selection bias due to field worker decisions and practices(Reference Danovaro-Holliday, Dansereau and Rhoda32,Reference Grais, Rose and Guthmann33) . The current recommendation(9) is to randomly select HHs from a list of those within the selected cluster. This approach was used in two studies(Reference Sesay, Hodges and Kamara25,Reference Koroma, Conteh and Bah30) .
When selected HHs are found empty or selected clusters become inaccessible (e.g. due to conflict, wildfires and flooding), a plan for cluster replacement and at least two HH revisits should be put in place. Yet overall, only one study(Reference Bendech, Cusack and Konaté16) reported having a replace/revisit plan.
Within the selected HHs, most studies enrolled only one eligible child aged 6–59 months. To optimise resources and guarantee that the probability of selection for an individual is equal to the probability of selection for his or her HH, WHO(9) recommends including every eligible child in every selected HH, as was done in five reviewed articles(Reference Babaniyi, Siziya and Mukonka22,Reference Sangho, Belemou and Keita23,Reference Ouédraogo, Becquey and Wilson27–Reference Adamu and Muhammad29) . Moreover, because the target population of VAS programmes is children aged 6–59 months, it is recommended that the whole age group be included to both optimise resources and measure the percentage of children 6–59 months of age who received an age-appropriate vitamin A supplement in each semester(5).
Digital data collection is beneficial because it eliminates the problem of illegible handwriting and can be directly linked, via data transmission, to a central location for storage and analysis. It also makes it easier to check the entries for mistakes and correct them before the data are transmitted(9). For these reasons, electronic tools, as used in only two reviewed studies(Reference Hadzi, Asalu and Avedzi28,Reference Koroma, Conteh and Bah30) , are preferred over paper-based questionnaires, where feasible. In certain contexts, in fact, digital data collection may not be feasible due to lack of electricity, internet connection and capacity in addition to data security issues.
In all the articles included in the systematic review, the CHC and parent recall were the main sources of information on the number of vitamin A supplements received by the child. If no home-based record of VAS was available, the next level of evidence was a verbal history of VAS by parents. In immunisation surveys, the validity of parental recall can be unreliable because of the complexity of immunisation schedules(Reference Miles, Ryman and Dietz34). However, remembering the number of vitamin A capsules received by the child is more straightforward and can be facilitated by showing a sample capsule to the parent, as was done in eleven studies(Reference Masanja, Schellenberg and Mshinda15–Reference Ayoya, Bendech and Baker17,Reference Gebremedhin, Loha and Abebe19–Reference Babaniyi, Siziya and Mukonka22,Reference Clohossey, Katcher and Mogonchi24,Reference Ouédraogo, Becquey and Wilson27,Reference Koroma, Conteh and Bah30,Reference Kassa, Mesfin and Gebremedhin31) . For these reasons, it is acceptable to collect VAS information by parental recall when CHC is not available.
According to WHO(8,9) , to guarantee the quality of collected data, it is necessary to provide interviewers with training and supervision. They should be organised in teams of two completing one cluster of a maximum of 30 HH/day. One supervisor should also be assigned to every two teams to monitor the quality of their work. WHO also recommends that interviewers be familiar with the clusters they are assigned and fluent in the local language. In line with these recommendations, most studies (n 14)(Reference Bharmal and Omair14–Reference Dhillon C, Subramaniam and Mulokozi21,Reference Sangho, Belemou and Keita23–Reference Sesay, Hodges and Kamara25,Reference Ouédraogo, Becquey and Wilson27,Reference Koroma, Conteh and Bah30,Reference Kassa, Mesfin and Gebremedhin31) provided training and supervision to locally recruited interviewers, who surveyed an average of 29 HH/d.
Once data are collected, recommended quality actions(8,9) include double data checking, entry and cleaning. These actions were performed in the majority of reviewed studies (n 12)(Reference Bharmal and Omair14–Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19–Reference Babaniyi, Siziya and Mukonka22,Reference Clohossey, Katcher and Mogonchi24,Reference Sesay, Hodges and Kamara25,Reference Ouédraogo, Becquey and Wilson27,Reference Adamu and Muhammad29,Reference Kassa, Mesfin and Gebremedhin31) .
Subsequently, under the multistage cluster sampling approach with PPS, data analysis must be weighted because sampling probabilities differ for different respondents. To derive a correct coverage estimate, sample weights need to be applied to each cluster to account for differences in population size and for non-response(8,9) . Overall, only four studies(Reference Bendech, Cusack and Konaté16,Reference Hodges, Sesay and Kamara20,Reference Clohossey, Katcher and Mogonchi24,Reference Koroma, Conteh and Bah30) calculated weights for analysis to account for differences in population size within the sampled clusters, and another three studies(Reference Masanja, Schellenberg and Mshinda15,Reference Dhillon C, Subramaniam and Mulokozi21,Reference Babaniyi, Siziya and Mukonka22) adjusted VAS coverage estimates for non-response rate. On the other hand, the majority of studies (n 11)(Reference Masanja, Schellenberg and Mshinda15,Reference Ayoya, Bendech and Baker17,Reference Gebremedhin, Loha and Abebe19,Reference Dhillon C, Subramaniam and Mulokozi21–Reference Sangho, Belemou and Keita23,Reference Sesay, Hodges and Kamara25,Reference Hadzi, Asalu and Avedzi28–Reference Kassa, Mesfin and Gebremedhin31) reported a 95 % CI of estimated VAS coverage, as recommended by WHO(8,9) , including the CLQAS(Reference Koroma, Conteh and Bah30) which design is not meant to measure the point of coverage estimates, but to identify whether an area (lot) has achieved a minimum level of coverage(Reference Danovaro-Holliday, Dansereau and Rhoda32). Although the main outcome of CLQAS is a binary classification of areas (lots) in accepted/rejected, without providing a point of coverage estimate, lot data can be aggregated according to a stratified weighted design to estimate coverage in the area. The main advantage of CLQAS is the small sample size required to classify lots with regard to coverage levels, but despite such advantage, the only reviewed CLQAS(Reference Koroma, Conteh and Bah30) selected 855 units, no more no less of the CCSs. Moreover, WHO discourages the use of this design to measure the point of coverage estimates, as it is not specifically conceived for this goal and uses a priori defined decision rules to classify coverage which contrast with the objective of coverage estimation(9).
Because implementing a coverage survey is resource-intensive, efforts should be made to improve efficiency by measuring annual two-dose VAS coverage, presenting data by the age group (i.e. 6–11 and 12–59 months) and by the collection method. To do this, the survey should be performed during the December–February period, as done by two of the reviewed studies(Reference Sangho, Belemou and Keita23,Reference Hadzi, Asalu and Avedzi28) .
Following-up with children to assess if they receive their second dose, as done in the LCS(Reference Adamu and Muhammad29), may introduce selection biases (e.g. by not considering population movement, including newly arrived children and children who age in or out of the eligible age range between the first and second dose). WHO underlines that an important sampling challenge is ensuring that no populations are missed, especially those that are difficult to reach(Reference Danovaro-Holliday, Dansereau and Rhoda32).
In accordance with ethical standards(35), most studies (n 15)(Reference Bharmal and Omair14–Reference Bendech, Cusack and Konaté16,Reference Gebremedhin, Loha and Abebe19–Reference Clohossey, Katcher and Mogonchi24,Reference Mahajan, Srivastav and Mukherjee26–Reference Kassa, Mesfin and Gebremedhin31) were conducted in accordance with national policies on ethics for surveys involving human subjects, including obtaining verbal informed consent, which is widely accepted by Institutional Review Boards for a standard coverage survey without biological sample collection(9).
It is a limitation of this systematic review that six studies(Reference Bharmal and Omair14,Reference Bendech, Cusack and Konaté16,Reference Sachdeva and Datta18,Reference Hodges, Sesay and Kamara20,Reference Clohossey, Katcher and Mogonchi24,Reference Ouédraogo, Becquey and Wilson27) did not provide information on sample size calculation procedures. Moreover, only five articles(Reference Gebremedhin, Loha and Abebe19,Reference Dhillon C, Subramaniam and Mulokozi21,Reference Sesay, Hodges and Kamara25,Reference Adamu and Muhammad29,Reference Koroma, Conteh and Bah30) were classified as being of high quality.
The review is also limited by the small number of studies focusing on the measurement of routine VAS coverage (n 8)(Reference Masanja, Schellenberg and Mshinda15,Reference Bendech, Cusack and Konaté16,Reference Sachdeva and Datta18,Reference Gebremedhin, Loha and Abebe19,Reference Babaniyi, Siziya and Mukonka22,Reference Mahajan, Srivastav and Mukherjee26,Reference Koroma, Conteh and Bah30,Reference Kassa, Mesfin and Gebremedhin31) and, in particular, two-dose routine VAS coverage (n 4)(Reference Sangho, Belemou and Keita23,Reference Ouédraogo, Becquey and Wilson27–Reference Adamu and Muhammad29) .
While a greater number of studies would have provided a wider evidence base upon which to draw conclusions, the lack of peer-reviewed publications itself demonstrates the need to strengthen methods for measuring the administrative coverage of VAS delivered through routine health services.
Conclusion and recommendations
In the current transition process towards routine health system contacts as the main VAS delivery platform and administrative electronic-based data collection systems, improving routine data quality is the best way to ensure stronger service delivery and monitoring of VAS programmes, as these data provide the most sustainable method for coverage estimation.
However, most VAS priority countries are in the early stages of this process and do not yet have the ability and full capacity to measure routine two-dose VAS coverage(4).
Based on the results of this systematic review, these countries can adopt multistage CCS to measure VAS coverage, using the recommendations included in Table 8.
Table 8. Recommendations to conduct vitamin A supplementation coverage cluster survey
Consistent with WHO guidance(9), the methodological recommendations provided will enable and support countries to collect reliable data for VAS coverage measurement (either after a vitamin A event distribution or via routine health contacts) in order to plan, monitor and evaluate VAS programmes in the current transition period and beyond.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/jns.2021.65.
Acknowledgments
A.H. and A.I. led the design and coordination of the review. A.M. peer-reviewed the search strategies for the review, conducted the literature searches, imported records and removed duplicates. A.M., A.H. and A.I. conducted the screening of the records, extracted the data and appraised the quality of evidence. A.M. led the collection of full-text articles. A.M., A.H. and A.I. led the analysis and interpretation of data. A.M. led the writing of the paper. All authors were responsible for revising the manuscript critically for important intellectual content. All authors read and approved the final manuscript.
Funding for this work was provided to UNICEF by the Government of Canada through Global Affairs Canada as part of the ‘Enhanced Child Health Days’ grant. The funder had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
There are no conflicts of interest.
Open access
