FFQ are widely used in large-scale studies to assess dietary intake. The FFQ is relatively inexpensive when self-administered, less intrusive and less time-consuming for the participants compared with other dietary assessment methods such as 24-h recalls and food records( Reference Ambrosini, de Klerk and Musk 1 – Reference Cullen, Watson and Zakeri 3 ). Due to the design of an FFQ with a list of food items and response categories of how frequently the food items are consumed, the FFQ is appropriate for ranking individuals into groups of habitual dietary intake. As a result, the FFQ is primarily used to study associations between habitual dietary intake and diseases in epidemiological studies( Reference Gibson 4 – Reference Willett 6 ).
To detect possible associations between diet and diseases it is crucial to clarify if the dietary assessment method is valid. One aspect of validity is to evaluate the reproducibility of the method( Reference Araujo, Yokoo and Pereira 7 – Reference Roman-Vinas, Ortiz-Andrellucchi and Mendez 9 ). Reproducibility gives an indication of whether the dietary assessment method will generate similar results independently of researcher and occasion when distributed several times and may reveal problems in instrument design or respondent instructions( Reference Turconi, Celsa and Rezzani 10 ). When evaluating reproducibility the interval between occasions is an important factor to consider as too long intervals may simply reflect true change in dietary intake in addition to normal variation in responses, whereas a short temporal separation may reflect carry-over responses from previous completion of the FFQ( Reference Deschamps, de Lauzon-Guillain and Lafay 11 ).
Even among adults, completing an FFQ can be a difficult cognitive task, as it requires accuracy in terms of memory, ability to identify various types of foods and their preparation methods as well as quantification of the amounts consumed( Reference Chambers, Godwin and Vecchio 12 , Reference Livingstone and Robson 13 ). In children, it is generally assumed that from the age of around 10 years they can recognise and describe quantities necessary for filling out an FFQ( Reference Livingstone and Robson 13 ). However, adolescents are more prone to omit and intrude foods when reporting their diet compared with adults( Reference Livingstone and Robson 13 – Reference Lyng, Fagt and Davidsen 15 ). Even though skills in estimating portion sizes increase with age, focus on body image which seems to affect the accuracy of adolescents’ self-reported dietary intake also increases with age( Reference Bandini, Schoeller and Cyr 16 , Reference Livingstone, Robson and Wallace 17 ). For adolescents, previous studies report that foods consumed on a regular basis and main courses are more accurately reported compared with foods eaten less often( Reference Baxter, Thompson and Davis 14 , Reference Speck, Bradley and Harrell 18 , Reference Wong, Boushey and Novotny 19 ). However, lack of attention among adolescents may increase recall bias( Reference Livingstone, Robson and Wallace 17 , Reference Thompson and Subar 20 ). To increase motivation among adolescents and reduce costs of distribution and processing data, web-based FFQ have been developed by several research groups( Reference Matthys, Pynaert and De 21 – Reference Overby, Johannesen and Jensen 23 ). An acceptable level of reproducibility and validity of such FFQ have been observed among adolescents( Reference Matthys, Pynaert and De 21 , Reference Filippi, Amodio and Napoli 22 , Reference Tabacchi, Filippi and Breda 24 ).
In spite of the limitations and potential pitfalls mentioned, adolescents are generally considered able to recall dietary intake at an acceptable level as they have knowledge of food items, the concept of time, and portion size estimation( Reference Livingstone and Robson 13 ). It is, however, for this age group extremely important to elucidate the validity and reproducibility of the FFQ designed for adolescents to facilitate further development of instruments that can be used to accurately assess the diets of adolescents. The aim of the present study was to evaluate the reproducibility of a web-based FFQ targeting Danish adolescents, aged 13–15 years, within the Danish National Birth Cohort (DNBC).
Methods
Subjects and study design
The DNBC was initiated in 1996 and includes information from 101 042 pregnancies. Various data from gestation week 12 and 30, and 6 and 18 months postpartum are available( Reference Olsen, Melbye and Olsen 25 ). Detailed offspring follow-up was conducted at the ages of 7 and 11 years.
Data for the present study were obtained from a subcohort study within the DNBC, which ran from May 2012 to April 2014. The purpose of this study was to investigate insulin sensitivity, risk of type 2 diabetes after pregnancy, and risk of diabetes in a group of children aged 9–15 years whose mothers had had gestational diabetes mellitus (GDM) and in a similar age group of offspring of women who had had a normal pregnancy( Reference Zhang, Hu and Olsen 26 ). Enrolled adolescents aged 13–15 years of age who completed a web-based FFQ (FFQ1) at a clinic visit were invited for this reproducibility study. No exclusion criteria were included because the invited adolescents were relatively healthy in terms of non-communicable diseases. Invitations were distributed to 100 adolescents born to both GDM and non-GDM mothers and were handed out at the clinic between January and March 2013. The adolescents were asked to complete a second FFQ (FFQ2) at home 4 weeks after the clinic visit. At the clinic they could ask for help from parent or clinical personnel if needed. The FFQ was available online and the adolescents entered the FFQ with a personalised ID number and password. Follow-up on completing FFQ2 was done first by email and second by mobile text message. Time for filling in the FFQ was approximately 40 min.
The GDM study was approved by the National Committee on Health Research Ethics in Denmark (H-4-2011-045).
FFQ
The web-based FFQ was developed from the validated Youth/Adolescent Questionnaire (YAQ) used in the US Growing Up Today Study (GUTS)( Reference Rockett, Breitenbach and Frazier 27 , 28 ). The YAQ was translated into Danish and modified to include typical Danish foods based on reports on National Danish Dietary Habits and Physical Activity( Reference Fagt, Christensen and Groth 29 , Reference Pedersen, Fagt and Groth 30 ). The FFQ is based on the online html program Limesurvey. Screen dumps from the FFQ are available (www.ssi.dk ( 31 )).
The FFQ included frequency questions about 145 food items and the recall period was the previous month. The food items were clustered into the following eight groups (number of food items in parentheses): beverages (18); dairy products (8); bread and cereals including butter on bread (13); spread on bread (14); cold and warm dishes (25); side dishes and condiments (18); fruit and vegetables (29); snacks and desserts (20). Frequency scales used for beverages, milk products, bread, side dishes and condiments, added salt and sugar, fruit and vegetables ranged from ‘did not drink/consume the last month’ to ‘4 times or more per d’. For cereals, spread on bread, cold and warm dishes, snacks and desserts, the frequency scale ranged from ‘did not consume the last month’ to ‘2 times or more per d’. Predefined portion sizes based on standard portions developed by the National Food Institute in Denmark( Reference Ygil 32 ) were applied for beverages and fruit, whereas for the rest of the food items no portion size was given. For calculations of intake in g per d all frequencies were computed into times per d and multiplied with standard portion sizes. Foodcalc v.1.3 combined with the Danish Food Table was used for estimating food and nutrient intake( Reference Lauritsen 33 , 34 ).
Open-ended questions were included to record additional information on food allergy, foods the participant may avoid, and questions about changes in food habits in the last month. Furthermore, the FFQ contained questions on (number of questions in parentheses): age, sex, self-reported height and weight, meal habits (15), physical activity (11) and puberty (6). Small notes and pop-up questions (16) were given throughout the questionnaire in order for the participant to keep the recall period in mind. Written instructions on how to complete the questionnaire were available on the first two pages of the questionnaire together with examples.
Statistics
Descriptive analyses were performed to evaluate participant charateristics. Age- and sex-specific BMI was determined based on the International Obesity Task Force( Reference Cole, Bellizzi and Flegal 35 ). Overweight was defined as BMI > 25 kg/m2 and obesity as BMI > 30 kg/m2( Reference Cole, Bellizzi and Flegal 35 ). The highest level of education of two parents was divided into four levels: more than 4 years of post-secondary education as high level of education (corresponding to a masters or PhD level), 3–4 years of post-secondary education as medium level of education (corresponding to bachelors level), skilled workers (9 years of basic school plus 5 years of vocational training), and unskilled workers.
Deviation from normality for dietary variables was evaluated by QQ-plots and histograms. Because the majority of variables were skewed, all variables are presented as medians with 25th and 75th percentiles. Differences between groups were examined using the non-parametric Wilcoxon sum-rank test. Correlation coefficients estimated between the two FFQ were Spearman's correlation coefficients and intra-class correlation coefficients (ICC). Cohen's weighted κ statistics and misclassification analyses (into thirds) were applied to test the FFQ's ability to rank individuals according to intake of food groups, energy and macronutrients.
To examine how consistently subjects could answer simple questions covering school lunch, supplement use, leisure time physical activity and selected food items, κ statistics were applied. The selection of food items for this analysis was based on the most popular and unpopular food items among 11- to 14-year-olds described in the report Dietary Patterns of Danish Children and Adolescents 2000–2004( Reference Fagt, Christensen and Groth 29 ). The popularity of food items was determined by summarising how frequently food items were consumed over a period of 7 d and thereby comparing high-frequency food items with low-frequency food items( Reference Fagt, Christensen and Groth 29 ). The purpose of this analysis was to examine if the adolescents are able to consistently answer simple questions on their overall dietary habits/lifestyle and still have sufficient knowledge/awareness of the foods even though they have difficulties in estimating frequency of consumption. Thus, with irregular eating habits among adolescents, 100 % agreement between measures of specific food items could not be expected. Sensitivity analyses were performed for children born to mothers with and without GDM, age-specific BMI and supervision from parents.
All statistical analyses were performed in the statistical program SAS version 9.4 (SAS Institute).
Results
Subjects
A total of fifty-two subjects agreed to participate. Four individuals did not complete the second questionnaire although several reminders were provided which resulted in forty-eight subjects completing the study. One subject was excluded from further analyses due to too many questions being omitted in the questionnaire which resulted in extremely low energy intake in FFQ2 (energy intake < 2500 kJ/d). Table 1 shows participant characteristics with mean age being 13·5 (sd 0·58) years and median BMI 19·7 (5th–95th percentiles 17–28) kg/m2. Age- and sex-specific BMI by the International Obesity Task Force categorised eleven subjects as overweight and three subjects as obese( Reference Lauritsen 33 ). The majority of participants were girls (60 %) and 60 % were born to mothers with GDM (61 % girls). Comparing those who completed both FFQ (n 48) with non-participants (n 52), respectively, no substantial difference was found for proportions of girls, age, BMI, born to GDM mothers, or parental educational level (data not shown).
Table 1. Participant characteristics (n 47, 60 % girls)
GDM, gestational diabetes mellitus.
* Overweight: BMI >25 kg/m2.
† Obese: BMI >30 kg/m2.
‡ Highest educational level of two parents during pregnancy presented as percentage.
Dietary intake
Table 2 presents the median intake of food groups, energy and macronutrients in FFQ1 and FFQ2. No significant differences were observed between food groups or nutrients in FFQ1 compared with FFQ2. The percentage difference between FFQ1 and FFQ2 showed that the overall intake reported tended to be higher in FFQ1 compared with FFQ2; however, only beverages, meat/poultry, fish and fat intake reached a difference of more than 20 %. The tendency of higher intakes reported in FFQ1 compared with FFQ2 for food groups was confirmed using Bland–Altman plots where the mean difference between FFQ1 and FFQ2 was above zero in all food groups (data not shown). Crude Spearman correlations and ICC for food groups ranged from 0·39 (beverages) to 0·83 (fruit) and from 0·37 (beverages) to 0·83 (fruit), respectively. Lower ICC compared with Spearman correlations for most food groups except for bread, fish, fruit and carbohydrate showed some within-subject variance. Adjustment by sex and BMI resulted in stronger Spearman correlations for a few food groups and most profoundly for cereals, vegetables and protein (g/d) (data not shown). The proportion of subjects classified correctly into the same tertile for food groups ranged from 45 % (fish) to 77 % (vegetables), whereas the proportion of subjects misclassified (into the opposite tertile) ranged from 0 % (fruit, oils and dressing) to 15 % (beverages). Furthermore, Cohen's weighted κ values ranged from 0·23 (fish) to 0·71 (vegetables) (see Table 2).
Table 2. Differences in and intra-class correlations for food groups and energy and macronutrients between FFQ1 and FFQ2 (n 47)
p25, 25th Percentile; p75, 75th percentile.
* Median and percentiles for all variables except energy percentage from macronutirents: mean and sd.
† Median difference for all variables except energy percentage from macronutirents: mean difference.
‡ P < 0·001 for all food groups.
§ Reliability test P < 0·05 for beverages, dairy products, cereals, energy percentage from protein, P < 0·0001 for all others.
We found no significant difference in food groups or energy and macronutrient intake between children born to mothers with/without GDM. An overall lower reported intake in all food groups was seen in adolescents with age- and sex-specific BMI > 25 kg/m2 (n 14) compared with adolescents with age- and sex-specific BMI < 25 kg/m2 (n 33); however, the difference was only significant for sugar and sweets (mean difference 43·6 (95 % CI 4·08, 83·1) g/d; P = 0·03). FFQ1 was self-reported by 38 % of the participants whereas FFQ2 was self-reported by 81 % (P < 0·0001). The proportion of girls was similar to the proportion of boys who received support from parents in FFQ1 (girls 61 % v. boys 58 %). The same was seen for the proportion receiving parental support in FFQ2 (girls 18 % v. boys 21 %).We found no significant difference in energy intake between levels of support.
Agreement with respect to specific questions about dietary habits and selected food items between FFQ1 and FFQ2 is presented in Table 3. The first questions listed in the table focus on dietary habits whereas the next questions concern frequently consumed, rarely consumed and less healthy food items, respectively. The percentage agreement for dietary patterns (89–94 %) showed that the adolescents to a high degree were able to recall overall dietary behaviour. For regularly consumed food items such as white bread (34 %), whole-grain bread (36 %) and potatoes (34 %), agreement was lower compared with more rarely consumed foods such as fish (62 %), porridge (68 %) and avocado (70 %). However, at the same time a relatively accurate recall was seen for type of milk (83 %) and type of dairy products (79 %). Lower agreement was also seen for less healthy food items such as Nutella (40 %) and jam (32 %).
Table 3. Agreement in questions between FFQ1 and FFQ2 (n 47)
Discussion
Using an FFQ designed to capture dietary habits of Danish adolescents we found that the FFQ categorised adolescents consistently (47–77 %) according to the magnitude of food intake at two different occasions, making the method reproducible at an acceptable level( Reference Masson, McNeill and Tomany 36 ). For beverages, fish and energy percentage from protein, correlation coefficients, classification into tertiles, and Cohen's weighted κ were lower compared with the other variables. This highlights the need for cautious interpretation when considering the validity of future association studies based on these variables. Furthermore, we elucidated that adolescents were able to accurately report overall dietary habits, but had some difficulties reporting regularly consumed food items, which demand a more specific estimation of frequency.
Our findings are comparable with studies including the adolescent population( Reference Overby, Johannesen and Jensen 23 , Reference Xia, Sun and Zhang 37 ). Xia et al.( Reference Xia, Sun and Zhang 37 ) included 168 adolescent girls from North China aged 12–18 years with 9 months between administrations of an eighty-one-item FFQ. They reported higher percentage for correct classification into quartiles for food groups (64·5–83·9 %) compared with our study. However, in a study of fifty-eight adolescents from Norway aged 13–14 years who completed a 131-item FFQ 4 weeks apart, Overby et al.( Reference Overby, Johannesen and Jensen 23 ) found between 36 % (bread and cereals) and 55 % (dairy products) correct classification of subjects into quartiles. The lower level of correct classification compared with our study was not expected because the applied FFQ was developed based on the Danish FFQ and the methodology was somewhat similar between the two studies. Nevertheless, in our study correct classification was above 50 % and Cohen's weighted κ values were above 0·4 which has been suggested to be acceptable for reproducibility studies in adults( Reference Masson, McNeill and Tomany 36 , Reference Shrout 38 ). Additionally, the number of food items in the FFQ, sample size and number of categories in the classification analysis could affect the level of correct classification. Spearman correlation coefficients and ICC for both food groups and nutrients were comparable with or higher than those found in other studies among adults and adolescents( Reference Xia, Sun and Zhang 37 , Reference Labonte, Cyr and Baril-Gravel 39 , Reference Willett, Sampson and Stampfer 40 ). Overby et al.( Reference Overby, Johannesen and Jensen 23 ) showed correlation coefficients between 0·41 and 0·67 for food groups and 0·57 to 0·62 for macronutrients, whereas Xia et al.( Reference Xia, Sun and Zhang 37 ) reported ICC between 0·58 and 0·73 for food groups. However, in a review Cade et al.( Reference Cade, Burley and Warm 8 ) reported that the most common correlations in reproducibility studies are between 0·5 and 0·7 among adults. Overall, the reproducibility found in the present study was acceptable and comparable with other studies including adolescents even though differences in population size, methodology and design of FFQ differ somewhat between studies.
The overall decrease in reported intake of specific food groups from FFQ1 to FFQ2 in our study is similar to what has been reported by others( Reference Filippi, Amodio and Napoli 22 , Reference Overby, Johannesen and Jensen 23 , Reference Xia, Sun and Zhang 37 , Reference Rockett, Wolf and Colditz 41 ) and could partly be due to the fact that FFQ2 to a greater extent was completed without parental support. Learning effects from FFQ1 to FFQ2 could also reduce reported intake in FFQ2( Reference Ambrosini, de Klerk and Musk 1 ). It could also be speculated that completing the second FFQ at home alone would decrease focus and increase omission of some food items. In both the studies by Overby et al.( Reference Overby, Johannesen and Jensen 23 ) and Rockett et al.( Reference Andersen, Bere and Kolbjornsen 2 Reference Araujo, Yokoo and Pereira 7 ), the FFQ was self-administered at both occasions. However, even in the studies by Xia et al.( Reference Xia, Sun and Zhang 37 ) and Filippi et al.( Reference Filippi, Amodio and Napoli 22 ) where the FFQ was administered in class with adult assistance, the intake of most food groups decreased from FFQ1 to FFQ2.
The comparison between specific questions to examine consistency between FFQ1 and FFQ2 showed good agreement for overall dietary habits and certain food products. Similar results were reported in a US study including 415 children aged 11–12 years completing an FFQ with twenty-five food items focusing on foods contributing to fat, fibre, fruit and vegetable intakes( Reference Buzzard, Stanton and Figueiredo 42 ). Their item-to-item comparison revealed higher agreement for single listed and regularly eaten foods such as milk compared with items listed with multiple foods (e.g. doughnuts, sweets rolls or muffins)( Reference Buzzard, Stanton and Figueiredo 42 ). Similar results were reported in another study from the USA including 259 children from the 8th grade who completed food and meal behaviour questions 7–14 d apart. The results showed 79–96 % agreement for school lunch, supplement use, type of milk and physical activity( Reference Hoelscher, Day and Kelder 43 ). The lower level of agreement for specific and regularly consumed food items observed in our study supports the possibility that adolescents have some difficulties registering frequency. Higher correlation coefficients for food items consumed with low frequency indicate that these items are easier to recall. Frank et al.( Reference Frank, Nicklas and Webber 44 ) support this finding as they reported higher correlation coefficients for rarely consumed food items among 1108 American adolescents. On the one hand, this could be because it is easier to answer ‘no’ to consumption rather than estimating a specific frequency. It could also be due to multiple food items listed in one question (e.g. ‘Cornflakes, Bran-flakes, Special-K etc.’) compared with single listed food (e.g. ‘avocado’). The lower level of agreement for less healthy food items could be explained by these food items being ‘not socially acceptable’ among adolescents. However, it could also be that these food items may actually be consumed on a regular basis( Reference Fagt, Christensen and Groth 29 ).
There are a number of potential strengths and limitations in the present study. First, all parental educational groups were well represented, which is rarely seen within these types of studies. In large cohort studies, higher socio-economic status is often over-represented. Second, the design with a web-based FFQ was easily accessible for the participants and the 4-week time interval seemed adequate as the observed degree of reproducibility may reflect variation in responses rather than changes in dietary intake( Reference Deschamps, de Lauzon-Guillain and Lafay 11 ). The size of the study population is a potential limitation to recognise together with that fact that 60 % of the study population were children born to GDM mothers. This resulted in a relatively high proportion of overweight children, which may make the results less representative for the Danish adolescent population( Reference Molgaard, Dele and Froberg 45 , 46 ).
Conclusion
In conclusion, the web-based FFQ for Danish adolescents was reproducible to an acceptable level with observed classification analyses and correlation coefficients comparable with similar studies in adolescents. In addition, the study revealed that the adolescents seemed capable of consistently recalling overall dietary habits and had some difficulties estimating the frequency of consumption of regularly consumed food items. The reported level of reproducibility is important when assessing dietary intake among adolescents to facilitate the development of instruments for accurate assessment of the diet of adolescents.
Acknowledgements
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
In cooperation, all authors formulated the research question and design of the study. A. A. B. was responsible for carrying out the study, analysing the data and writing the article. T. I. H. assisted in analysing and interpreting the data and writing the article. T. I. H., S. F. O. and I. T. critically reviewed the manuscript and all co-authors approved the final version submitted.
There were no conflicts of interest.
Open access
