The FFQ is a dietary assessment tool commonly used in epidemiological studies. However, data collection via FFQ is associated with a serious issue. A long, self-administered, mailed questionnaire provides the opportunity of non-response. Missing values directly influence the estimation of dietary intake from the questionnaire. Several approaches have been used to deal with this problem, including deletion of cases with one or more missing values from the analysis (complete-case analysis) or filling in the missing data with single values, such as the mean, median or mode (single imputation method). Although zero imputation is a common strategy for handling missing data in FFQ( Reference Lamb, Olstad and Nguyen 1 ), it is doubtful whether the use of this method is acceptable in obtaining unbiased estimates( Reference Ahn, Paik and Ahn 2 – Reference Parr, Hjartaker and Scheel 5 ).
Two studies reported that missing food items did not represent zero intake when the foods were frequently consumed( Reference Fraser, Yan and Butler 6 , Reference Michels and Willett 7 ). In another study, some food items were left blank because of inattention or carelessness( Reference Kuskowska-Wolk, Holte and Ohlander 4 ). Additionally, because of single imputation methods, such as zero imputation, where the subject’s response and imputed value for non-response are treated in the same way, an underestimation of variance occurs( Reference Schafer and Graham 8 ). Therefore, we believe that although the use of zero imputation is reasonable in some situations, it may give misleading results.
In the last decade, statistical methods for the analysis of missing data and the importance of not ignoring missing values in the analysis have gained attention among researchers. Rubin classified missing data into three categories: (i) missing at random (MAR); (ii) missing completely at random (MCAR); and (iii) missing not at random (MNAR)( Reference Schafer and Graham 8 – Reference Rubin 10 ). Data are considered MAR when the probability that the data are missing depends on the observed data but not on the missing data. MCAR is an important, special case of MAR when the probability that the data are missing does not depend on any data. In contrast, data are considered MNAR when the probability that the data are missing depends on the missing values themselves as well as on the observed data. For instance, MAR is when older subjects tend to skip more questions than younger subjects (age is known) and MNAR is when subjects with a higher or lower salary than the average leave questions pertaining to their salary unanswered because of their reluctance to answer. Some researchers have noted that missing data in FFQ are likely to be MNAR because some missing values might represent zero intake( Reference Lamb, Olstad and Nguyen 1 , Reference Parr, Hjartaker and Scheel 5 ). When considerable missing values due to zero intake are removed from missing data in FFQ, the remnant data might approach MAR. We considered how to predict missing values due to zero intake in the imputation process, because we cannot remove them from data in practice.
In the literature, missing data are indirect MNAR when the observed data can predict the reason for missing data( Reference Schafer and Graham 8 – Reference Rubin 10 ). For example, when subjects with a higher or lower salary than the average are likely to skip a question because of their salary, the missing data are MNAR. When we obtain correlates of their salary, like job and career, subsequently, indirect MNAR data are treated as MAR because the missing salary becomes predictable. In FFQ, missing values for food items that are infrequently consumed tend to represent zero intake; however, this phenomenon may vary with each food item( Reference Hansson and Galanti 3 , Reference Parr, Hjartaker and Scheel 5 – Reference Michels and Willett 7 ). Additionally, more information can help to predict missing values in multiple imputation( Reference Fraser and Yan 11 ). The observed data explain how frequently each food item is consumed or is not. When the missing level is relatively low, the observed data can be used to predict the probability of zero intake to some degree, or can provide support to predict missing values by the distribution and pattern of intake. For these reasons, data may be indirect MNAR.
In the case of MAR or MCAR, multiple imputation is the preferred method to account for missing data and is recommended by several journals and medical research guidelines( Reference Klebanoff and Cole 12 , Reference Ware, Harrington and Hunter 13 ). The use of this method provides several advantages over conventional methods, such as avoiding loss of sample size and considering the uncertainty of imputed values, but only few nutritional epidemiological studies have applied this method for the missing data in FFQ( Reference Barzi, Woodward and Marfisi 14 , Reference Rizzo, Sabate and Jaceldo-Siegl 15 ).
We aimed to examine missing data in FFQ, especially missing values due to zero intake, and to assess the effects on total energy and nutrient estimates. We used data from the Okazaki Japan Multi-Institutional Collaborative Cohort (J-MICC) study to verify the missing data mechanism of FFQ and to compare nutrient estimates obtained with the two imputation methods. We did not conduct a resurvey and treated the baseline values as reference.
Methods
Okazaki J-MICC study
The present study was conducted as a part of the J-MICC study( Reference Hamajima 16 ). We recruited 7580 men and women (aged 35–79 years) between 2007 and 2011 from the Okazaki Public Health Center, Okazaki, Aichi, Japan. The self-administered health and lifestyle questionnaire, which contained an FFQ, was mailed prior to the health check-up at the centre. At baseline, all participants received the health check-up and submitted the questionnaire. The submitted questionnaires were checked three times by the investigators. When participants did not answer completely (e.g. leaving blanks or inconsistent answers), we queried them as possible. Between 2013 and 2017, we conducted the second survey for the 5-year follow-up. Mean period from baseline to the second survey was 5·3 years. At the second survey, some participants received the health check-up and submitted the questionnaire in the same way, whereas others provided the questionnaire only by mail. In this case, the questionnaires were neither checked nor resurveyed. Of the total participants, 5321 completed both the baseline (FFQ1) and second surveys (FFQ2). Data pertaining to 5151 participants (96·8%) who fully completed FFQ1 were included in the present analysis.
The study protocol was approved by the institutional review board of the Nagoya University Graduate School of Medical Science and Medical School. Written informed consent was obtained from each participant.
FFQ
The Department of Public Health, Nagoya City University Graduate School of Medical Science and Medical School, developed and validated the FFQ( Reference Tokudome, Goto and Imaeda 17 – Reference Imaeda, Goto and Tokudome 20 ). Participants were asked how often on average over the previous year they consumed forty-seven foods and beverages. The alcohol item was divided into ten subtypes and had six possible responses ranging from ‘never/rarely’ to ‘every day’. Because energy intake is derived mainly from staple foods in the Japanese dietary style, each staple food item (rice, bread and noodles) was divided into each meal (breakfast, lunch and dinner), and six possible responses similar to those included for alcohol subtypes were also included. For other food and beverage items, there were eight possible responses ranging from ‘never/rarely’ to ‘3 or more times per day’; a portion size was specified for each staple food by meal and for the alcohol subtype (e.g. 1 cup or 1 slice of bread). The nutrient intake was calculated from a total of eighty-one questions (62 frequencies= 10 alcohols+9 staple foods+43 other foods; 19 portion sizes=10 alcohols+9 staple foods) by determining the daily intake frequency (‘never/rarely’=0; ‘1–3 times per month’=0·1; ‘1–2 times per week’=0·2; ‘3–4 times per week’=0·5; ‘5–6 times per week’=0·8; ‘once per day’=1·0; ‘twice per day’=2·0; ‘3 or more times per day’=3·0).
In the present analysis, we calculated energy intake derived from alcohol only for current alcohol drinkers. For nutrient calculations, participants were excluded if answers to more than forty of the eighty-one questions in FFQ2 were missing or if their total energy intake in each FFQ was below 2510 kJ/d (600 kcal/d) or above 14 644 kJ/d (3500 kcal/d), according to previous studies( Reference Parr, Hjartaker and Scheel 5 , Reference Michels and Willett 7 ). The purpose of these criteria was to exclude, for example, those who are unwilling to answer the FFQ or who over-/under-report their dietary intake.
Missing values and imputation
Missing alcohol items were permitted if the participants were not current alcohol drinkers; missing portion size was permitted if the participants answered with the frequency ‘never/rarely’ for the item. These permitted missing values were replaced with zero before imputations.
We filled the missing values in FFQ2 using the following imputation methods.
1. FFQ1 imputation (baseline value): missing values were imputed with the baseline value of the same individual in FFQ1.
2. Zero imputation: missing values were imputed with the frequency ‘never/rarely’ and portion size ‘1 unit’ (standard size).
3. Multiple imputation: we conducted multiple imputation by chained equations( Reference van Buuren 21 , Reference Van Buuren and Oudshoorn 22 ). Missing values for frequency and portion size in the FFQ were imputed as continuous variables (daily intake frequency and portion size per unit, respectively). Sixty data sets were created with a relative efficiency of >0·99 for each variable. Relative efficiency is an indicator of whether a sufficient number of data sets is created( Reference Enders 23 ). Although the usual practice or recommended method is to create ten to twenty data sets, a large number of data sets is expected to achieve stable estimates and to improve the validity of the significance tests. The imputation model included the following known or suspected variables associated with missing values or outcomes( Reference Fraser, Yan and Butler 6 , Reference Michels and Willett 7 ): age at the second survey (year, continuous); sex (men, women); alcohol drinking (current, former, never); smoking (current, former, never); questionnaire submission (at the centre, by mail); work (full-time workers, other workers, non-workers); education (elementary school/junior high school, high-school graduate, college or more); BMI (kg/m2, continuous) and physical activity at leisure time (MET-h/week, continuous; where MET is metabolic equivalent of task). Missing values for these covariates were also imputed. After nutrient calculations, we combined the results into a single estimate with an se.
Statistical analysis
We calculated the mean number of missing values in FFQ2 by category of background variables and evaluated the difference between categories, excluding the missing, by testing with one-way ANOVA. For each food item, the number and proportion of participants who responded with the frequency ‘never/rarely’ and portion size ‘1 unit’ in FFQ2 are presented. For sensitivity analysis, we calculated the mean total energy and nutrient intakes for complete cases and all of FFQ2. We also calculated mean total energy and nutrient intakes by stratifying according to how the questionnaire was submitted (at the centre or by mail). Differences in total energy and nutrient intakes from zero imputation and multiple imputation compared with FFQ1 imputation were tested by paired t tests. We performed all data analyses and imputations using the statistical software package SAS version 9.4. PROC MI and PROC MIANALYZE procedures were used for multiple imputation and combining the results.
Results
Of the 5151 participants who completely answered FFQ1, 3673 participants (71·3%) responded to all the questions in FFQ2 (Table 1). In FFQ2, the mean number of missing values among the 5151 participants was 3·3 per participant (4·1% of data matrix), and 99·4% of them had forty or fewer missing values in the eighty-one questions after allowing for the permitted missing values. The proportion of participants with one missing value was the largest in that of participants with at least one missing value. The number of participants decreased with increasing number of missing values, but a slight increment of participants with nine to twelve missing values was observed. Thirty-one participants were excluded because their FFQ2 had more than forty missing values.
Table 1 Distribution of missing values in FFQ2 of the Okazaki Japan Multi-Institutional Collaborative Cohort (J-MICC) study
Table 2 shows the personal characteristics that might be associated with the number of missing values in FFQ2. The mean number of missing values in FFQ2 increased with age. Similarly, a lower education level and a higher physical activity level indicated more questions being missed in FFQ2 than those indicated by higher education level and lower physical activity. In addition, men, current alcohol drinkers, former smokers and non-workers were more likely to have a larger number of missing values in FFQ2. Obviously, participants who submitted the questionnaires by mail left many more blanks in FFQ2 than those who submitted the questionnaires at the centre.
Table 2 Characteristics related to missing values in FFQ2 of the Okazaki Japan Multi-Institutional Collaborative Cohort (J-MICC) study
† P value when the missing category is excluded.
We investigated the proportion of frequency ‘never/rarely’ in: (i) FFQ2; and (ii) FFQ1 among those who had a missing value in FFQ2 (Table 3). When proportion (i) was over 70%, it was observed that proportion (ii) was more than 80%. Most of the food items had almost the same degree of proportion (i) and (ii), but some food items (e.g. chu-hi, bread at lunch, broccoli) had a higher proportion (ii) than proportion (i).
Table 3 Frequency of intake: response of ‘never/rarely’ in FFQ1 and FFQ2 of the Okazaki Japan Multi-Institutional Collaborative Cohort (J-MICC) study
† Proportions when missing values in FFQ2 are excluded for each item.
‡ Proportions to the number of missing values in FFQ2.
§ Alcoholic drinks are only for current alcohol drinkers at the second survey (n 2702).
We also investigated the proportion of portion size ‘1 unit’ in: (i) FFQ2; and (ii) FFQ1 among those who had a missing value in FFQ2 (Table 4). We excluded the participants who answered ‘never/rarely’ for each item from the analysis of Table 4 because the distribution of portion size was divided into zero intake and others. Most of the participants responded with ‘1 unit’ in FFQ2. When proportion (i) was over 70%, proportion (ii) was observed to be more than 60%. Proportions (ii) of food items excluding rice (bowl at dinner) and whiskey (double) were a little lower than proportions (i).
Table 4 Portion size of intake: response of ‘1 unit’ in FFQ1 and FFQ2 of the Okazaki Japan Multi-Institutional Collaborative Cohort (J-MICC) study
† Excluded when the frequency of each item is ‘never/rarely’. Proportions when missing values in FFQ2 are excluded for each item.
‡ Proportions to the number of missing values in FFQ2.
§ Alcoholic drinks are only for current alcohol drinkers at the second survey (n 2702).
Table 5 shows the estimated dietary intakes of total energy and selected nutrients from FFQ1 and FFQ2. In complete-case analysis, the total energy decreased from 7082 kJ/d (1693 kcal/d) in FFQ1 to 6834 kJ/d (1633 kcal/d) in FFQ2. There were slight changes in the intakes of protein, fat, carbohydrate and dietary fibre. In all participants, zero-imputed FFQ2 presented the smallest total energy, whereas FFQ1 presented the highest total energy. The results of multiple imputation were almost identical to estimates from FFQ1 imputation. Particularly, there were the smaller differences of total energy and carbohydrate (g) from multiple imputation to FFQ1 imputation than those from zero imputation. No significant differences between multiple imputation and FFQ1 imputation were observed for total energy, protein (% energy), fat (% energy), carbohydrate (g and % energy) and dietary fibre (g/1000 kcal). Less than forty participants were excluded from the total of 5120 participants because their total energy intake was <2510 kJ/d (<600 kcal/d) or >14 644 kJ/d (>3500 kcal/d).
Table 5 Estimated dietary intakes of total energy and selected nutrients from FFQ1 and FFQ2 of the Okazaki Japan Multi-Institutional Collaborative Cohort (J-MICC) study
n indicates the number of participants who had forty or fewer missing values and total energy intake between 2510 kJ/d (600 kcal/d) and 14 644 kJ/d (3500 kcal/d). n alc indicates the number of participants who consumed >0 g alcohol/d in n participants. Energy from alcohol was not included in the total energy.
*P<0·05, **P<0·01, ***P<0·001 for paired t test between imputation methods.
As a sensitivity analysis, participants were stratified by questionnaire submission (see online supplementary material). Obviously, the results of participants who submitted a questionnaire at the centre were exactly similar between the imputation methods, because the proportion of missing values was too small (0·3% of data matrix; Supplemental Table 1). In participants who submitted by mail, where the proportion of missing values was 11·5% of the data matrix, multiple imputation presented smaller differences of total energy and nutrient intake except for alcohol to FFQ1 imputation than zero imputation, although these differences were larger than the results of all participants (Supplemental Table 2).
Discussion
From the population of the second survey of the Okazaki J-MICC study, we examined the missing data in FFQ and then compared the results of some imputation methods for total energy and nutrient intakes. Missing food items that are not eaten often were likely to represent zero intake, and the number of missing values was associated with personal characteristics. The estimated total energy and nutrient intakes of the multiple imputation method were found to be slightly different from those of the single imputation method (zero or baseline value). In particular, multiple imputation showed almost the same estimates as FFQ1 imputations (imputing baseline values). Our findings suggest that the application of multiple imputation to missing data in FFQ is a reasonable choice when the proportion of missing data is relatively low.
To the best of our knowledge, only two studies have investigated the potential effects of multiple imputation on missing data in FFQ. Barzi et al. applied repeated measures of the consumption of five food items in the GISSI-Prevenzione study and suggested that the use of multiple imputation is likely to provide valid estimates( Reference Barzi, Woodward and Marfisi 14 ). Fraser and Yan introduced guided multiple imputation to fill in a random sub-sample of initially missing data and to adjust the imputation model using this extra information as a guide( Reference Fraser and Yan 11 ). Compared with other methods (complete-case analysis, zero imputation and multiple imputation), the food intake frequencies showed a moderate difference. Although ascertaining whether the missing data in FFQ were actually MAR was a common problem in these studies, a lower proportion of missing values is likely to have minimized bias.
We investigated the mechanism of missing data in FFQ by comparing with baseline values. Most food items showed that the proportion of zero intake in the observed data was likely to be similar to the probability of zero intake in missing data; whereas for some food items this did not hold. The former indicates that missing values due to zero intake can be predicted from the observed distribution of how rarely a particular food is consumed in the population, and the data can be treated as MAR and not MNAR. The latter seems to indicate that there were other factors that influenced the missingness than just a zero intake; that is, missing data in FFQ may be MNAR. These findings are consistent with the results of previous studies that showed that each food item that is not eaten often is likely to represent zero( Reference Parr, Hjartaker and Scheel 5 – Reference Michels and Willett 7 ). For this reason, it would be reasonable to impute zero at least for rarely eaten foods. However, zero imputation can introduce an element of bias because it is impossible that all missing values are zero.
Additionally, the estimates of total energy and nutrient intakes determined with multiple imputation were equivalent to those determined with FFQ1 imputation, which suggests that multiple imputation can predict the missing values due to zero intake. Even if we determine the ‘true’ value for the missing food items, it might lie between the values determined with zero imputation and FFQ1 imputation. In the present study, the proportion of missing values in our data was relatively low, and a comparison of imputation methods might be much more difficult because of little freedom for any moderate difference to be displayed. To deal with this problem, we carried out stratified analysis and presented that multiple imputation had the smaller differences to FFQ1 values in total energy and nutrient intakes, except for alcohol, than zero imputation. The application of multiple imputation might provide a more accurate estimate when the probability of missing data is less than 11·5%. For alcohol, several possible reasons exist. First, it seemed to be more difficult to predict and impute missing values than for the other items because the alcohol item has detailed questions. Additional covariates for more accurate imputation may be needed. Second, missing alcohol values were more likely to represent zero intake than other food items.
Furthermore, imputing a single value ignores the uncertainty of imputed values and can lead to biased results because of underestimating the variance( Reference Schafer and Graham 8 ). Regarding the current trend in medical research, it is important to appropriately handle missing data and to avoid misleading results. Our findings suggest that multiple imputation may be employed for handling missing data in future studies involving FFQ.
The present study has several limitations. First, we treated baseline values in FFQ1 as a reference and did not perform a resurvey. Some studies have shown that total energy and food intakes decrease with age( Reference Otsuka, Kato and Nishita 24 , Reference Wakimoto and Block 25 ); further, our results of complete-case analysis showed a decrease in total energy and nutrient intakes over a mean follow-up period of 5·3 years. To improve the accuracy of multiple imputation, the investigation of additional variables in the imputation model or of item-level and score-level multiple imputation( Reference Gottschall, West and Enders 26 , Reference Plumpton, Morris and Hughes 27 ) may be needed. However, because of the slight difference in nutrient estimates among imputation methods, this was not a problem in the present study.
Second, we used the complete-case data in FFQ1. Complete-case analysis may lead to biased results unless the mechanism of missing data is MCAR( Reference Schafer and Graham 8 ). However, all questionnaires of the baseline survey were reviewed and the percentage of our participants who completely answered FFQ1 was 96·8%. Thus, the missing data of FFQ1 were considered to be MCAR and had no impact on our results.
Third, the questionnaires used in the second survey were submitted by two different approaches. Approximately 70% of the questionnaires were reviewed at the time of health check-up of the participants, whereas approximately 30% were submitted by mail and did not receive a review. Consequently, a higher proportion of our participants completely answered FFQ2 compared with that in some previous studies (34–41%)( Reference Michels and Willett 7 , Reference Caan, Hiatt and Owen 28 ), which would contribute to a more plausible imputation rather than any undesirable results from multiple imputation( Reference White, Royston and Wood 29 ).
Conclusion
Zero imputation was a reasonable choice for a limited number of food items only when the missing values were due to ‘zero intake’. To appropriately handle missing data in FFQ, we recommended multiple imputation, taking account of the uncertainty of imputed values. The present study confirmed that multiple imputation would be applicable to missing data in FFQ when missing level is relatively low. Particularly, if missing level exceeds about 10% of data, one should carefully consider the effect of the imputation method on epidemiological risk analysis with a disease outcome, at least for dietary intake.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1368980019000168
Acknowledgements
Acknowledgements: The authors would like to thank all the staff and participants for supporting this study. Financial support: This study was supported in part by the Grants-in-Aid for Scientific Research on Priority Areas of Cancer (No. 17015018), on Innovative Areas (No. 221S0001), and JSPS KAKENHI Grant Number JP (No. 16H06277) from the Japanese Ministry of Education, Culture, Sports, Science and Technology. The Japanese Ministry of Education, Culture, Sports, Science and Technology had no role in the design, analysis or writing of this article. Conflict of interest: None declared. Authorship: S.S. was responsible for the design and funding of the present study. T.Y. is the director of the Public Health Center of the Okazaki City Medical Association. N.I. and C.G. developed and provided the calculation program for the FFQ. A.H. took the lead role in collecting data. K.S., S.T., M.I., Y.T., M.W., K.O., H.F., N.O., M.K., F.K., R.W. and T.N. also contributed to data collection. M.I. performed statistical analysis. S.S. and T.I. advised on drafting the manuscript. All authors have checked and approved the final version of the manuscript submitted and any revised version. Ethics of human subject participation: The study protocol was approved by the institutional review board of the Nagoya University Graduate School of Medical Science and Medical School. Written informed consent was obtained from each participant.
