In nutritional epidemiology, prospective cohort studies constitute the strongest observational design to study associations between diet and health outcomes( Reference Satija, Yu and Willett 1 ). FFQ are the common choice for assessing dietary intake in large observational studies: they are able to capture usual, individual, long-term dietary intake, and participant burden is low( Reference Willett 2 ). However, FFQ rely on long-term memory and are subject to socially desirable answers( Reference Satija, Yu and Willett 1 ). Moreover, certain foods may be neglected because of the fixed food list( Reference Satija, Yu and Willett 1 ). As a result of these types of measurement errors, FFQ may not be useful to detect weak associations between dietary intake and health outcomes( Reference Schatzkin, Subar and Moore 3 , Reference Freedman, Schatzkin and Midthune 4 ).
In order to stimulate standardised assessment of dietary intake in large-scale studies in the Netherlands, a new FFQ has been developed – the FFQ-NL 1.0. This FFQ aims to provide comprehensive and standardised data collection of usual energy, food and nutrient intakes in Dutch adults. The questionnaire has been developed to suit current and future research objectives for epidemiological research in the Netherlands. Moreover, to incorporate changes in dietary habits and food products over time, the most recent Dutch National Food Consumption Survey and food composition table were used. The development of the FFQ-NL 1.0 and its compatibility with other Dutch FFQ will be described separately (SJPM Eussen, MCJM van Dongen, NEG Wijckmans-Duysens, et al., unpublished results). The FFQ-NL 1.0 will become nationally available for use in future Dutch epidemiological studies and as such it may serve as a national reference FFQ. Validation of this newly developed FFQ is essential to provide insight to which extent the measurement errors distort observed diet–disease relationships( Reference Cade, Thompson and Burley 5 ). The objective of this study was to validate the FFQ-NL 1.0 in adults against multiple 24-h dietary recalls (24hR), 24-h N and K excretion in urine, and plasma carotenoids and fatty acids measured in cholesteryl esters.
Methods
Study design and population
The validation study was embedded in the Nutrition Questionnaires plus (NQplus) study, an ongoing longitudinal study in the city of Wageningen and surroundings, the Netherlands( Reference van Lee, Feskens and Meijboom 6 ). Between 2011 and 2013, 2048 men and women aged 20–70 years were included. The NQplus study was approved by the ethics committee of Wageningen University and conducted according to the guidelines laid down in the Declaration of Helsinki. Participants filled out general and health questionnaires at baseline, year 1 and year 2. Moreover, a physical examination including blood and urine collection was performed at baseline, year 1 and year 2, and multiple 24hR were administered throughout the 2-year study period. All participants provided their written informed consent. For the present validation study, a sample of 445 participants aged 25–69 years was invited to fill out the FFQ-NL 1.0, of which 386 (87 %) responded. A random subsample of 150 people agreed to repeat urine and blood tests. Fig. 1 gives an overview of the measurements and the time frame for the purpose of this validation study.
Fig. 1 Time frame and overview of measurements of the FFQ-NL 1.0 validation study.
FFQ-NL 1.0
The National FFQ for the Netherlands, the FFQ-NL 1.0, was developed to obtain comprehensive and standardised data on food intake in large samples of Dutch adults. The FFQ-NL 1.0 was developed with FFQTOOLTM, an online tool to develop tailor-made FFQ, and was designed for use in the general population. The food items in the FFQ covered ≥85 % of the absolute intake of energy and thirty-nine nutrients, ≥88 % of the inter-individual variation in intake of energy and macronutrients and 45–93 % of the inter-individual variation in intake of micronutrients as assessed by two non-consecutive 24hR in the Dutch National Food Consumption Survey 2007–2010( Reference van Rossum, Fransen and Verkaik-Kloosterman 7 ). The FFQ-NL 1.0 consists of 160 food items with questions on frequency and consumed amounts with a 1-year reference period. Average daily energy and nutrient intakes were calculated by multiplying frequency of consumption by consumed amounts and nutrient content per item using the Dutch food composition table of 2011( 8 ). Of the 386 participants, three were excluded on the basis of their energy intakes: <2092 kJ/d (<500 kcal/d) or >14 644 kJ/d (>3500 kcal/d) in women and <3347 kJ/d (<800 kcal/d) or >16 736 kJ/d (>4000 kcal/d) in men( Reference Willett 9 ). A total of 278 persons filled out the FFQ again after about 12 months.
24-h recalls
As part of the overall NQplus study, multiple, telephone-based 24hR were administered. Dates were randomly selected and scheduled evenly across the year and days of the week. The 24hR were administered by dietitians trained in interviewing skills using a five-step, multiple-pass method, which is a validated technique to increase accuracy( Reference Conway, Ingwersen and Moshfegh 10 – Reference Moshfegh, Rhodes and Baer 12 ). Recalls were transcribed into food codes and food groups of the 2011 Dutch food composition table( 13 ). For each person, individually, recalls that were assessed within 12 months before that person filled out the FFQ-NL 1.0 for the first time were selected, resulting in 1038 24hR. For seven persons, no 24hR were available, whereas for the remaining 376 persons, an average of 2·7 (range 1–5) 24hR was included in the current analyses.
Urine collection and analyses
The 24-h urine collections started with the second voiding after waking up and were completed with the first voiding after waking up the next day. Urine samples were handed in at the hospital and transported to the study centre, where they were mixed, weighed, aliquoted and stored at −20°C until further analysis. The participants received three 80-mg para-aminobenzoic acid (PABA) tablets to check for completeness of the urine collections. PABA recovery was measured using the HPLC method( Reference Jakobsen, Ovesen and Fagt 14 ). A recovery of at least 78 % of the ingested PABA was considered as complete urine collection. Within the Observing Protein and Energy Nutrition Study, the exclusion of participants with incomplete urine samples had little or no effect on the correlation with true intake and the attenuation factors derived for the FFQ( Reference Subar, Midthune and Tasevska 15 ). Hence, the analyses were performed on all urine samples. In a sensitivity analysis, persons with a PABA recovery <78 % (n 82) were excluded. Total 24-h N excretion was determined by the Kjeldahl technique (Foss KjeltecTM 2300 analyser; Foss Analytical). Urinary protein content was calculated using the following formula: 6·25×(urinary N/0·81)( Reference Bingham 16 ), accounting for approximately 19 % faecal and skin losses. Urinary K concentration measurements were performed with an ion-selective electrode on a Roche 917 analyser; urinary excretion of 81 % was assumed( Reference Freisling, van Bakel and Biessy 17 ). Urinary N and K were available for 363 persons, of which 139 provided a replicate urine sample to determine urinary N and K.
Blood collection and analyses
After a 10-h overnight fast, 24 ml of blood was drawn from an antecubital vein using venepuncture. Blood was immediately centrifuged, and plasma was stored at −80°C until further analyses. Carotenoids were determined using HPLC and UV-vis detection( Reference Leenders, Leufkens and Siersema 18 ). Fatty acids from plasma cholesteryl esters were quantified by GLC using the solid-phase extraction method to separate the cholesteryl esters with acidified methanol. Peak retention times and area percentages of total fatty acids were determined by using known cholesteryl ester standards and analysed using Agilent Technologies ChemStation software (Agilent)( Reference Lu, Vaarhorst and Merry 19 ). Plasma carotenoids were available for 360 persons and plasma fatty acids for 358 persons, of which fatty acids and carotenoids were determined in replicate blood samples of 141 persons.
Other variables
Height was measured without shoes using a stadiometer (SECA 213; SECA Corp.). Weight was measured without shoes and heavy clothing and with empty pockets on a digital scale (SECA 877; SECA Corp.). Questionnaires assessed educational level, presence of diseases, smoking status and whether the participants followed a diet regimen.
Measurement error models
It was assumed that dietary intake estimated using multiple 24hR as well as protein and K intakes estimated by urinary analysis were the best standards to approximate true intake( Reference Jenab, Slimani and Bictash 20 ). For the replicate FFQ-NL 1.0, a constant bias, intake-related bias and person-specific bias were assumed to be present. The following measurement error models were used:
$$\eqalignno{ & {\rm 24{\hbox -}h}\,{\rm recall}\,\left( R \right){\rm \,\colon\,}\quad \quad \quad \quad R\,{\equals}\,T{\plus}e(R) \cr & {\rm Urinary}\,{\rm biomarker}\,\left( G \right){\rm \,\colon\,}\,\, G\,{\equals}\,T{\plus}e(G) \cr & {\rm FFQ{\hbox -}NL\,1}{\rm .0}\,\left( Q \right){\rm \,\colon\,}\quad \quad \quad Q\,{\equals}\,A(Q){\plus}B(Q){\times}T{\plus}q{\plus}e(Q), $$
where A is the constant bias, B the intake-related bias; e the random error, q the person-specific bias, FFQ-NL 1.0 and T the true (unknown) intake.
Statistical analyses
All statistical analyses were performed using SAS 9.3. Linear mixed models with a random intercept for participants were applied taking into account multiple measurements per person. Macronutrients and alcohol were additionally expressed in energy densities to adjust for energy. Absolute differences between self-reported intakes in the FFQ-NL 1.0 and intakes estimated from the reference methods – that is 24hR, urinary biomarker or replicate FFQ – were expressed as group-level bias: (mean intake FFQ-NL 1.0/mean intake reference method)×100–100; differences larger than 5 % were considered relevant. For protein and K, bias in mean intake was evaluated by comparing the distributions of reported intake and intake based on urinary excretion. The attenuation factor was estimated as the slope in the linear regression of the reference method on the reported intake according to the FFQ-NL 1.0. The validity coefficient or de-attenuated correlation coefficient is defined as the correlation between the observed intake as measured using the FFQ and the ‘true’ intake as measured using the biomarkers and 24hR. The de-attenuated correlation coefficient was estimated as the correlation coefficient between the FFQ-NL 1.0 and the reference instrument divided by the square root of the intraclass correlation coefficient (ICC) of replicates of the reference method( Reference Hankinson, Manson and Spiegelman 21 , Reference Kromrey, Fay and Bellara 22 ). The validity of the FFQ-NL 1.0 was judged based on comparison with other published FFQ and whether the correlation coefficients fell within the range of 0·4–0·7 as mentioned by Willett( Reference Willett 23 ). Validity was first evaluated by attenuation factors and de-attenuated correlation coefficients for protein and K using recovery biomarkers and for other nutrients and foods using 24hR as the reference method, followed by reproducibility and ranking ability. The replicate FFQ-NL 1.0 was used to study reproducibility only. In addition to the unstratified analysis, stratified analyses were performed for men and women, persons aged 25–56 and 57–69 years (median split), according to educational status (low/middle: no, lower or lower vocational education; intermediate: intermediate vocational; and high: higher vocational or university), and for persons with BMI<25 and ≥25 kg/m2.
Results
Timing of the measurements and general characteristics
The FFQ-NL 1.0 was administered throughout the year, with a minor higher frequency during winter (Fig. 1). The FFQ-NL 1.0 was repeated after approximately 1 year. Blood and urine sample collections and 24hR were spread out equally over the seasons. The mean age of the participants was 53·9 (sd 10·4) years and 61 % were women (Table 1). In total, 45 % of the subjects were overweight or obese. Most of the participants were highly educated (60 %), and a few participants had prevalent diseases (13 %).
Table 1 General characteristics of the 383 NQplus participants, aged 25–69 years, included in the FFQ-NL 1.0 validation study (Mean values and standard deviations; numbers and percentages)
Validation of nutrients using 24-h recall
Compared with the 24hR, group-level bias was small for energy, most macronutrients, water and dietary fibre (Table 2). The FFQ-NL 1.0 underestimated the intake of total fat (g), SFA, trans-fatty acids and Ca and overestimated the intake of alcohol, EPA, DHA, haem-iron and most vitamins. For energy, macronutrients, dietary fibre and water, de-attenuated correlation coefficients ranged from 0·26 for trans-fatty acids to 1·18 for dietary fibre. For micronutrients, correlations ranged from 0·38 for vitamin B1 to 0·65 for Mg. For energy, macronutrients, dietary fibre and water, attenuation factors ranged from 0·32 for MUFA to 0·73 for alcohol and for micronutrients from 0·27 for vitamin B1, folic acid and haem-iron to 0·48 for Mg. Group-level bias of nutrients was higher in older than in younger subjects (data not shown). Attenuation factors were similar between age categories and men and women, but higher in highly educated subjects and subjects with a normal BMI (online Supplementary Table S1). De-attenuated correlations were higher in men, in highly educated subjects and in subjects with overweight and obesity (online Supplementary Table S2).
Table 2 Absolute intakes of nutrients in the FFQ-NL 1.0 and telephone-based 24-h recalls (24hR) and the relative difference, correlation coefficients, attenuation factors, de-attenuated correlation coefficients and cross-classification between the FFQ-NL 1.0 and the telephone-based 24-h recallsFootnote * (Mean values with their standard errors; group-level bias, correlation coefficient, attenuation factor, de-attenuated correlation coefficient and 95 % confidence intervals)
ALA, α-linolenic acid; ICC, intraclass correlation coefficient.
* % Group-level bias=(mean intake FFQ-NL 1.0/mean value 24hR)×100–100; attenuation factor (95 % CI) estimated as the slope in the linear regression of the reported intake from 24hR on the reported intake from FFQ; de-attenuated correlation coefficient (95 % CI) estimated as the correlation coefficient/√ICC24h R.
Validation of food groups using 24-h recall
For food groups, group-level bias was small (defined as <5 %) for the intakes of non-alcoholic beverages, bread, fruit, nuts/seeds/snacks, soup and fats/oils/sauces (Table 3). The FFQ-NL 1.0 underestimated the intakes of cake/cookies, vegetables, cheese, composite dishes and sugar/honey/jams/candy and overestimated the intakes of potatoes, eggs, cereals, savoury sandwich fillings, milk and milk products, legumes, soya and vegetarian products, fish and meat. Attenuation factors varied between 0·13 (legumes) and 0·72 (soya and vegetarian products).
Table 3 Absolute intakes of food groups in the FFQ-NL 1.0 and the telephone-based 24-h recalls (24hR) and the relative difference, correlation coefficients and cross-classification between the FFQ-NL 1.0 and telephone-based 24hRFootnote * (Mean values with their standard errors; group-level bias, correlation coefficient, attenuation factors, de-attenuated correlation coefficient and 95 % confidence intervals)
ICC, intraclass correlation coefficient.
* % Group-level bias=(mean intake FFQ-NL 1.0/mean value 24hR)×100–100; attenuation factor (95 % CI) estimated as the slope in the linear regression of the reported intake from 24hR on the reported intake from FFQ; de-attenuated correlation coefficient (95 % CI) estimated as the correlation coefficient/√ICC24hR.
Validation of protein and potassium using urinary biomarkers
Compared with the urinary recovery biomarkers, the FFQ-NL 1.0 underestimated protein intake by 15·9 % and K intake by 4·8 % at the group level (Table 4); this is confirmed in Fig. 2 and 3. The attenuation factors for protein and K were 0·46 (95 % CI 0·35, 0·57) and 0·44 (95 % CI 0·32, 0·55), respectively. De-attenuated correlation coefficients were 0·69 (95 % CI 0·53, 0·83) for protein and 0·58 (95 % CI 0·43, 0·73) for K. Including only subjects with a PABA recovery ≥78 % (n 280) yielded lower validity measures: attenuation factors were 0·42 (95 % CI 0·29, 0·55) for protein and 0·39 (95 % CI 0·25, 0·53) for K, and de-attenuated correlation coefficients were 0·59 (95 % CI 0·42, 0·74) for protein and 0·47 (95 % CI 0·31, 0·63) for K. Attenuation factors for protein and K tended to be higher among younger subjects, in men and in subjects with higher education and normal BMI (online Supplementary Tables S1 and S2). The underestimation of protein and K intakes at a group level was larger in younger persons and slightly higher among subjects with higher BMI and higher education. Group-level bias for protein was higher in men and for K higher among women (data not shown).
Fig. 2 Estimated distribution of protein intake from FFQ-NL 1.0 (g/d, 
) and intake based on excretion (g/d, 
).
Fig. 3 Estimated distribution of potassium intake from FFQ-NL1.0 (mg/d, ) and intake based on excretion (mg/d, ).
Table 4 Validity measures of reported intakes of protein and potassium by FFQ-NL 1.0 as compared with their urinary recovery biomarkersFootnote * (Mean values with their standard errors; estimates and 95 % confidence intervals)
ICC, intraclass correlation coefficient.
* % Group-level bias=(mean intake FFQ-NL 1.0/mean value recovery biomarker)×100–100; attenuation factor (95 % CI) estimated as the slope in the linear regression of the biomarker on the reported intake; attenuation factor adjusted for age, sex, BMI and educational attainment (low/medium/high); de-attenuated correlation coefficient (95 % CI) estimated as the correlation coefficient/√ICCrecovery biomarker.
Validation of (fatty) fish, fruit and vegetable intakes using concentration biomarkers
Correlation coefficients between (fatty) fish intake and plasma n-3 fatty acids were 0·43–0·47 (Table 5). Correlation coefficients between plasma carotenoids and fruit and vegetable intakes ranged between 0·24 and 0·43 (Table 5). Cross-classification showed that more than 60 % of the participants were allocated to the same or adjacent quintile of intake or plasma concentration. The correlations between fruit and vegetable intake and total carotenoids were somewhat higher among younger subjects and women. For fish intake and plasma n-3 fatty acids, correlations were higher among subjects with BMI<25 kg/m2; for fatty fish, correlations were higher among men and subjects with higher educational level (data not shown).
Table 5 Pearson’s correlation coefficients and cross-classification of reported intakes of (fatty) fish, fruits and vegetables in the FFQ-NL 1.0 and the related blood concentration biomarkers (Correlation coefficients and 95 % confidence intervals)
Reproducibility of the FFQ-NL 1.0
At the group level, the replicate FFQ showed comparable intakes of most food groups (Table 6). On average, intakes of cake and cookies, savoury sandwich fillings, legumes, nuts, seeds and snacks, and fish were higher in the replicate FFQ, but intakes of soup and soya and vegetarian products were lower. Correlation coefficients showed good agreement, ranging from 0·43 for legumes to 0·85 for soya and vegetarian products. For nutrient intakes (data not shown), relative differences in intakes between the first and the second assessment of the FFQ were negligible, and correlation coefficients were high for almost all nutrients (range 0·55–0·89).
Table 6 Reproducibility measures of food group intakes between FFQ-NL 1.0 and the replicate FFQ-NL 1.0 in 278 adultsFootnote * (Medians and percentiles 25th–75th (P25–P75); group-level bias, Pearson’s correlation coefficient and 95 % confidence intervals)
* % Group-level bias=(mean intake FFQ-NL 1.0/mean intake replicate FFQ-NL 1.0)×100–100.
Discussion
The present study investigated the validity of the FFQ-NL 1.0, a comprehensive, standardised, semi-quantitative FFQ for Dutch adults. Compared with the 24hR, absolute differences were small (<5 %) for energy and macronutrients; however, the FFQ underestimated the intake of fat and overestimated the intakes of alcohol, EPA, DHA and most vitamins. For food groups, we observed only some underestimation and overestimation. Compared with their recovery markers, the FFQ underestimated protein intake by 16 % and K by 5 %; the attenuation factors showed good agreement. Correlation of fruits plus vegetables and fish intakes with plasma carotenoids and n-3 fatty acids, respectively, was good. Overall, the validity measures were well within the range of agreement that could be expected based on the literature.
The current findings can be compared with a number of other Dutch FFQ that have been previously validated. Within 128 men and women from the Leiden Longevity Study, Streppel et al. ( Reference Streppel, de Vries and Meijboom 24 ) evaluated an FFQ against three 24hR. Correlation coefficients varied between 0·21 and 0·78 for nutrients and for food groups between 0·00 and 0·79. We found slightly higher correlations for macronutrients and fatty acids, but somewhat lower correlations for micronutrients. Correlations for food groups were generally higher, except for cheese, fats/oils/savoury sauces and pastry/cake/biscuits. Goldbohm et al. validated a 150-item FFQ against three 3-d dietary records in a representative subsample of the Netherlands Cohort Study on diet and cancer. For most nutrients, correlations between 0·60 and 0·80 were observed( Reference Goldbohm, van den Brandt and Brants 25 ), which were generally higher compared with the present validation study. Moreover, in a reproducibility study of the FFQ, correlations were found within the range of 0·42–0·80( Reference Goldbohm, van ‘t Veer and van den Brandt 26 ). Furthermore, Ocké et al. validated the FFQ used in the Dutch European Prospective Investigation into Cancer and Nutrition (EPIC) with respect to nutrients( Reference Ocke, Bueno-de-Mesquita and Pols 27 ) and food groups( Reference Ocke, Bueno-de-Mesquita and Goddijn 28 ) against the average of 12 monthly 24hR. For nutrients, the median Pearson’s correlation coefficient was 0·59 in men and 0·58 in women( Reference Ocke, Bueno-de-Mesquita and Pols 27 ); median Spearman’s correlation coefficients for food groups were 0·60–0·64 for men and 0·52–0·58 for women( Reference Ocke, Bueno-de-Mesquita and Goddijn 28 ). De-attenuated correlation between protein intake and urinary N was 0·43 in men and 0·50 in women( Reference Ocke, Bueno-de-Mesquita and Pols 27 ). For the FFQ-NL 1.0, we found lower correlations for both nutrients and food groups, which might be explained by the lower number of repeated 24hR – that is, 2·7/person on average.
Freedman et al. ( Reference Freedman, Commins and Moler 29 ) pooled five large US validation studies, comprising data of 2265 participants. Compared with 24-h urinary N, the FFQ under-reported protein intake by approximately 10–29 %. Furthermore, compared with 24-h K excretion, the FFQ under-reported K intake by 5–6 %( Reference Freedman, Commins and Moler 30 ). Estimation of protein and K intakes in the present study was in line with the findings from these five large US studies. Attenuation factors for reported intake by FFQ were on average 0·17 for protein and 0·25–0·30 for K( Reference Freedman, Commins and Moler 29 , Reference Freedman, Commins and Moler 30 ). Our study showed attenuation factors of 0·46 for protein and 0·44 for K, indicating substantially lower de-attenuation of relative risks.
Plasma carotenoids are considered as biomarkers of the intake of fruits and vegetables during the previous weeks or months( Reference Al-Delaimy, Ferrari and Slimani 31 ). Al-Delaimy et al. investigated the correlation between fruit and vegetable intakes from an FFQ and plasma carotenoids within the EPIC study. The Spearman’s correlation coefficient between reported fruit and vegetable intakes and total carotenoids was 0·38( Reference Al-Delaimy, Ferrari and Slimani 31 ). Burrows et al. ( Reference Burrows, Hutchesson and Rollo 32 ) found a correlation between vegetable intake by FFQ and plasma β-carotene of 0·42, between fruit intake and β-cryptoxanthin of 0·52 and a correlation of 0·30 and 0·26 between vegetable intake and lutein and zeaxanthin, respectively. For fruit and vegetable intakes by FFQ and plasma carotenoids, the present study showed correlation coefficients in the same range.
Although no accurate biomarker for total fat intake exists, EPA and DHA may serve as concentration biomarkers to evaluate fish intake( Reference Saadatian-Elahi, Slimani and Chajes 33 ). Within 3009 participants from EPIC, Saadatian-Elahi et al. ( Reference Saadatian-Elahi, Slimani and Chajes 33 ) showed a correlation coefficient of 0·29 between fatty fish intake and n-3 fatty acids at the individual level. In EPIC-Norfolk, Welch et al. ( Reference Welch, Bingham and Ive 34 ) evaluated an FFQ against n-3 fatty acids in blood plasma and found for reported fish intake a correlation coefficient of 0·17 and for fatty fish intake a correlation of 0·19 in women and 0·23 in men. With correlation coefficients of 0·43 for fish and 0·47 for fatty fish, values in the current study are higher than those found these two studies.
Dietary intake from multiple 24hR was assumed to best approximate true intake. However, correlated errors exist between FFQ and 24hR, including the use of the same food composition table, and they both rely on memory and may be subject to socially desirable answers. As such, the 24hR may be considered an alloyed gold standard reference method. However, it is generally considered the best reference method if no recovery biomarkers are available( Reference Freedman, Schatzkin and Midthune 4 ). Using 24hR may not remove all measurement errors of FFQ, but its use in addition to an FFQ will improve diet–disease associations( Reference Freedman, Schatzkin and Midthune 4 , Reference Geelen, Souverein and Busstra 35 ). Furthermore, the use of multiple 24hR is recommended above a single recall( Reference Freedman, Commins and Moler 29 , Reference Freedman, Commins and Moler 30 ). Compared with the 24hR, the FFQ-NL 1.0 seemed to under-report the intakes of fat and high-fat foods, particularly SFA, trans-fatty acids, cake and cookies, and cheese. The current study population comprised more women and highly educated persons than the general population. As a result, total variation in fat intake might be lower, corresponding with lower validity measures. Furthermore, subjects over-reported their intakes of legumes, soya and vegetarian products, alcohol and most micronutrients in the FFQ-NL 1.0 compared with the 24hR. Subjects also tended to under-report the intake of composite dishes, which may be due to coding differences. In the FFQ-NL 1.0, the covered variance in intake was <80 % for EPA, DHA, trans-fatty acids and vitamins B1, B2, B6, C, D, E and folate equivalents, which is likely reflected in the lower validity measures found for these nutrients. To increase the covered variance of particularly B vitamins, a new version of the FFQ-NL 1.0 is now being developed including a few additional food items. Hence, the validity of the improved FFQ is expected to be similar or higher.
Age and BMI may contribute to intake-related bias in the FFQ as well as correlated errors between FFQ and 24hR( Reference Geelen, Souverein and Busstra 35 ). Freedman et al. observed that a higher BMI was associated with a larger degree of under-reporting; having a lower educational level was also associated with more under-reporting. Furthermore, age above 59 years was associated with less under-reporting( Reference Freedman, Commins and Moler 29 , Reference Freedman, Commins and Moler 30 ). Indeed, the current study showed that attenuation factors were generally higher among older subjects and subjects with normal BMI, as well as among men and subjects with higher education.
The NQplus study population comprises a sample of highly educated and committed participants, who have become familiar with dietary assessment methods and with blood and urine collections. Hence, the participants may have been more diligent and accurate in recording their intakes and collecting their urine samples than the general population. However, the study benefitted from the multiple reference methods to which the FFQ could be validated: urinary recovery biomarkers, blood concentration markers and multiple 24hR, as well as the advanced statistical methods, which were used to assess validity.
To provide a reliable and correct measure of dietary intake in free-living populations is the largest challenge of nutritional epidemiology. Measurement errors in FFQ cannot be prevented, but they can be quantified and accounted for. Therefore, it is always recommended to use objective biological markers of dietary intake or 24hR complementary to an FFQ. Furthermore, an external validation study such as the present one is also essential in identifying and quantifying measurement errors and how diet–disease associations are affected. Attenuation directly affects the observed relative risk as well as the necessary sample size to detect these diet–disease relationships( Reference Kipnis, Subar and Midthune 36 ). Thus, future epidemiological studies using the FFQ-NL 1.0 can apply the attenuation factors presented here to calculate the sample size needed for desired statistical power and to adjust for observed relative risks.
Validity coefficients or de-attenuated correlation coefficients may also be used to quantify the impact of measurement error on diet–disease relationships( Reference Geelen, Souverein and Busstra 35 ), and were estimated as the correlation coefficients between the FFQ-NL 1.0 and the reference instrument divided by the square root of the ICC of replicates of the reference method. It is assumed that if the time between replicate urine collections and the 24hR is not too close, the errors in replicates are independent and they provide a measure of within-person variation within the biomarker( Reference Preis, Spiegelman and Zhao 37 ). However, within-person variation of the 24hR, characterised by a low ICC, was high, which may have led to an overestimation of the de-attenuated correlation coefficient. This may be because of the correlated errors between FFQ and 24hR.
In conclusion, the overall validity of the newly developed FFQ-NL 1.0 was acceptable to good, and the FFQ was able to adequately rank subjects according to their dietary intake. Therefore, the FFQ-NL 1.0 is well suited for future use within Dutch cohort studies among adults. As a future application, the FFQ-NL 1.0 can be used to improve the pooling of results from individual studies using the FFQ.
Acknowledgements
This study was financially supported by the Dutch Biobanking and Biomolecular Research Infrastructure project (BBMRI-NL), a Research Infrastructure financed by the Dutch Government (NWO 184·021·007).
M. C. O., E. J. M. F. and P. C. D. designed the study. A. G., J. H. M. d. V. and E. J. M. F. acquired the data. D. S. carried out the analysis, interpreted the data and drafted the article. All the authors interpreted the data, critically reviewed the article for important intellectual content and gave final approval of the version to be published.
All authors declare no conflicts of interest.
Supplementary Material
For supplementary material/s referred to in this article, please visit http://dx.doi.org/doi:10.1017/S0007114516002749
