Relative validity of the online Meal-based Diet History Questionnaire for evaluating the overall diet quality and quality of each meal type in Japanese adults

Abstract

The aim of this study was to examine the relative validity of the online Meal-based Diet History Questionnaire (MDHQ) for assessing the overall diet quality and quality of each meal type (breakfast, lunch, dinner and snacks). In total, 222 Japanese adults (111 for each sex) aged 30–76 years completed the online MDHQ and then the 4-non-consecutive-day weighed dietary record (DR). The diet quality was assessed using the Healthy Eating Index-2015 (HEI-2015) and Nutrient-Rich Food Index 9.3 (NRF9.3). For the HEI-2015, compared with the DR, the MDHQ provided high median values for breakfast (in women only) and dinner and low median values for snacks. There were no significant differences observed for overall diet and lunch. For the NRF9.3, the MDHQ provided higher median values for breakfast and dinner and a lower median value for overall diet than the DR in women, with no significant differences for lunch and snacks. In men, no significant difference was observed, except for overall diet (the MDHQ providing a lower median value). For the HEI-2015, median Spearman correlation coefficient was 0·43, with a range from 0·12 (snacks in women) to 0·68 (breakfast in men). For the NRF9.3, median Spearman correlation coefficient was 0·47, with a range from 0·26 (snacks in men) to 0·65 (breakfast in men). Bland–Altman plots showed wide limits of agreement and, in some cases, proportional bias. In conclusion, the online MDHQ showed an acceptable ability for ranking individuals according to the quality of overall diet, breakfast, lunch and dinner, but not snacks.

Sub-optimal dietary intake is a widely acknowledged major risk factor for promoting morbidity and premature death, and the improvement of diet quality is now a global priority⁽¹⁾. An accurate assessment of habitual dietary intake is a cornerstone for identifying the diet–disease relationships and for promoting favourable changes in dietary behaviours⁽²⁾. Traditional nutritional epidemiological research has focused mainly on the associations between health outcomes and the intake of individual nutrients or foods, but examining the associations between health outcomes and overall dietary patterns or overall diet quality has gradually been more priority⁽³⁾. An increasing number of studies have evaluated the dietary intake and quality of specific eating occasions or meal patterns^(4–6). Examining the dietary intake and quality at each meal level to assess the overall diet may be more pertinent when considering the synergies and interactions during digestion and metabolism⁽⁷⁾.

However, research in this area has been impeded by the fact that the primary method of dietary assessment currently employed in most cross-sectional and prospective cohort studies is the FFQ, which generally precludes an informed evaluation of the timing of dietary intake and meal-specific dietary intake⁽⁸⁾. This type of information can be derived using more detailed dietary assessment methods^(9–13), such as dietary record (DR) and 24-h dietary recall⁽⁴⁾. However, when using these methods, the collection of dietary data for multiple days is essential for the assessment of habitual intake at the individual level, but it is not always feasible because of its expensive and burdensome nature⁽¹⁴⁾, despite the advancement of technology⁽¹⁵⁾. To our knowledge, there are no purpose-built, dedicated dietary assessment questionnaires to collect data on dietary intake at each meal type, which are also inexpensive to implement and less burdensome for participants. In this context, we recently developed the Meal-based Diet History Questionnaire (MDHQ), a self-administered questionnaire designed to estimate the dietary intake for each meal type (breakfast, lunch, dinner and snacks) separately^(16,17) .

The MDHQ has several advantages. First, the MDHQ is data-driven, so the development of the questionnaire structure, food items and dietary intake calculation algorithms was based on detailed dietary information derived from the 16-d weighed DR obtained from 242 Japanese adults ¹⁶ . Second, because the MDHQ assesses dietary intake for each meal type separately and given that the cognitive tasks required during dietary recall are complex⁽¹⁸⁾, the MDHQ may be easier to complete, facilitating better estimation of food intake. This may be particularly relevant to the Japanese because previous studies of Japanese adults have shown that the selection, amount and combination of foods consumed are markedly different between meal types^(6,19–22) . Third, the MDHQ provides information on various aspects of dietary behaviours, such as breakfast quality and percentage of energy from snacking occasions. However, a rigorous evaluation of the validity of the MDHQ has not been conducted yet, except for food group intake⁽²³⁾.

The primary aim of this study was to examine the relative validity of the web version of the MDHQ for assessing the overall diet quality and quality of each meal type (breakfast, lunch, dinner and snacks). In the real world, not all study participants would complete the questionnaire online. Thus, the secondary aim was to similarly examine the relative validity of the paper version of the MDHQ. The diet quality was assessed using the Healthy Eating Index-2015 (HEI-2015)^(24–27) and Nutrient-Rich Food Index 9.3 (NRF9.3)^(27–31).

Methods

Study procedure and participants

This cross-sectional study was based on the data collected from fourteen (of the forty-seven) prefectures between August and October 2021. Recruitment of participants and data collection were conducted by our research dietitians (n 60) with expertise in collecting DR data^(32,33) . First, healthy women aged 30–69 years who were willing to participate and were living with their husbands were recruited for this study. For each prefecture, two women from each 10-year age category (30–39, 40–49, 50–59 and 60–69 years) were selected. Their husbands were then recruited (irrespective of age), resulting in 112 individuals by sex. The sample size was determined primarily based on the recommendation made by Cade et al. that for validation studies, a sample size of at least 50 and preferably much larger (e.g. 100 or more subjects) is desirable⁽¹⁴⁾. To minimise the dropout rate, the potential participants were restricted to individuals who had full understanding of the procedure and showed willingness to complete the entire survey. Meanwhile, dietitians, individuals living with a dietitian, those who had received dietary counselling from a doctor or dietitian, those taking insulin treatment for diabetes, those undergoing dialysis treatment, those without sufficient Internet access, those who had difficulty answering the web-based questionnaires and pregnant or lactating women were excluded. Only participation in the study as a couple (one woman and one man) was permitted.

The study schedule is shown in Fig. 1. Each participant was asked to answer the web version of the MDHQ (web MDHQ). After an interval of 7–10 d (to ensure the completion of the web MDHQ), a 4-non-consecutive-day weighed DR was conducted for 2 weeks. Finally, after an interval of at least 1 d, the paper version of the MDHQ (paper MDHQ) was completed. We designed this schedule because the main purpose of this study was to evaluate the validity of the web MDHQ; thus, a web MDHQ survey was performed prior to the conduct of DR. A total of 111 women aged 30–69 years and 111 men aged 30–76 years completed the study. As a financial incentive, each couple received a voucher worth 5000 Japanese Yen (31 British Pound as of 1 October 2022) after the study.

Fig. 1. Study schedule. MDHQ, Meal-based Diet History Questionnaire; DR, dietary record.

The study was conducted in accordance with the guidelines of the Declaration of Helsinki, and all procedures involving humans were approved by the Ethics Committee of the University of Tokyo Faculty of Medicine (protocol code: 2020326NI; date of approval: 29 January 2021). Written informed consent was obtained from all participants.

Meal-based Diet History Questionnaire

Details of the MDHQ have been published elsewhere^(16,17) . Briefly, the MDHQ is a self-administered questionnaire designed to estimate dietary intake in the previous month for each meal type (breakfast, morning snack, lunch, afternoon snack, dinner and night snack). The MDHQ comprises three parts. Part 1 of the MDHQ includes quantitative questions on the consumption frequency of generic food groups (Tier 1 food groups) for each meal type, with potential answers of 0–7 d/week. Part 2 of the MDHQ includes questions on the relative consumption frequency of sub-food groups (Tier 2 food group) within one of the generic food groups (Tier 1 food group), with possible answers of ‘always, often, sometimes, rarely, and never’. By combining the information derived from Parts 1 and 2, the number of foods that can be estimated efficiently can be increased but within a limited number of questions. Part 3 of the MDHQ enquires about the general eating behaviours, including the amount of brown rice consumed, the relative consumption frequency of wholegrain bread and whether bread was consumed with jam, honey, etc. or with fat spread. Finally, the MDHQ includes the assessment of basic characteristics (sex, age, body height, body weight, education level and current smoking status).

In the MDHQ, information on portion sizes was not collected (except for alcoholic beverages, for which the overall consumption frequency and portion size were assessed in Part 2). This decision was based on our previous observation that the Brief-type Diet History Questionnaire (BDHQ), which assesses the consumption frequency of fifty-eight food items but does not collect information on portion sizes and applies fixed portion sizes for dietary intake calculation, had a similar efficacy in estimating the food and nutrient intake as the Diet History Questionnaire (DHQ), which assesses not only the consumption frequency but also the portion size of 150 food items^(27,34,35) . The limited usefulness of portion size information has also been supported by several previous studies^(36,37) . All the food groups included in the MDHQ (see online Supplementary Table S1), as well as the sex-specific and meal-type-specific portion sizes, were determined based on the 16-d weighed DR data collected from 121 Japanese women and 121 Japanese men, comprising 206 837 food item entries⁽¹⁶⁾.

In the present study, two delivery modes of MDHQ, which are identical in terms of content, were used: web MDHQ and paper MDHQ. The web MDHQ was prepared using Google Forms. Each question was answered by each participant, with non-response not permitted. All responses to the web MDHQ automatically allocated into a spreadsheet format were downloaded from Google Drive. The paper MDHQ used in this study was an A4 21-page questionnaire. Responses to all questions were checked by the research dietitians and staff at the study centre. If any responses were missing, the participants were asked to answer the questions again in person or by phone. All answers in the paper MDHQ were manually entered into a spreadsheet in duplicate, and any discrepancies were checked and corrected. Data obtained using the web MDHQ and paper MDHQ were converted to a dataset suitable for dietary intake calculation.

On the basis of a series of ad hoc computer algorithms in the MDHQ⁽¹⁶⁾, estimated intakes of Tier 1 and 2 food groups were calculated. Estimated intakes of energy and nutrients were calculated using food intake information and the 2015 version of the Standard Tables of Food Composition in Japan⁽³⁸⁾. Component scores needed for the calculation of HEI-2015 were calculated using the Japanese version⁽²⁷⁾ of the US Food Patterns Equivalents Database⁽³⁹⁾. These calculations were done for each meal type, and the overall intake was calculated as the sum of the intake of each meal type.

Four-day weighed dietary record

The 4-non-consecutive-day weighed DR was selected as the reference method in this validation study. Each recording period consisted of three weekdays (Monday–Friday, except for national holidays) and one weekend day (Saturday, Sunday or national holidays). For each couple, a recording day was allocated within 2 weeks by research dietitians. Each couple was provided with recording sheets and a digital scale (KS-274, Dretec, Japan; ±2 g precision for 0–500 g and ±3 g precision for 500–2000 g). After receiving written and verbal instructions from the assigned research dietitian, as well as an example of a completed diary sheet, each participant was requested to document and weigh all consumed foods and drinks, both inside and outside of their homes, on each recording day. On certain occasions when weighing was inconvenient to carry out (e.g. dining out), they were instructed to document as much information as possible, including the brand name of the food and the consumed portion size (based on typical household measures), as well as the details of the leftovers.

The recording sheets used in each survey day were submitted directly to the research dietitian after the survey was completed, who then reviewed the forms and, whenever necessary, sought additional information or modified the record via phone or in-person interview. All collected records were then reviewed by the research dietitians and trained staff at the study centre. In accordance with a standardised procedure, the portion sizes estimated using household measures were converted into weights, and the individual food items were coded based on the 2015 version of the Standard Tables of Food Composition in Japan⁽³⁸⁾. A total of 1297 food codes were used in the DR.

The structure of the food diary sheet used was based on a typical Japanese eating pattern, which comprised breakfast, lunch, dinner and snacks; these meal types were prescribed in the diary. For DR data, the name of the meal type used in the present analysis was based on this classification. As was the case in the MDHQ, estimated intakes of energy and nutrients and component scores needed for the calculation of HEI-2015 were calculated using the 2015 version of the Standard Tables of Food Composition in Japan⁽³⁸⁾ and the Japanese version⁽²⁷⁾ of the US Food Patterns Equivalents Database⁽³⁹⁾, respectively. These calculations were done for each meal type, and the overall intake was calculated as the sum of the intake of each meal type. For all dietary variables, the mean daily values within the 4-d period were used for each individual.

Healthy Eating Index-2015

As described elsewhere^(24–27), HEI-2015 is a composite measure of compliance with the 2015–2020 Dietary Guidelines for Americans⁽⁴⁰⁾. The HEI-2015 is a 100-point scale, with a higher score indicating a better quality of diet. The HEI-2015 consists of nine adequacy components, namely, total fruits (maximum score: 5), whole fruits (5), total vegetables (5), greens and beans (5), whole grains (10), dairy products (10), total protein foods (5), seafood and plant proteins (5) and fatty acids as the ratio of the sum of PUFA and MUFA to SFA (10), and four moderation components, namely, refined grains (10), Na (10), added sugars (10) and saturated fats (10). For each meal type and overall diet for each participant for each dietary assessment method, we calculated the HEI-2015 component and total scores based on energy-adjusted values of dietary intake, namely, amount per 4184 kJ (1000 kcal) of energy or percentage of energy, except for the fatty acids component⁽²⁷⁾.

Nutrient-rich Food Index 9.3

The overall diet quality was also assessed using the NRF9.3, as described in detail elsewhere^(27–31). The NRF9.3 is a composite measure of the nutrient density of the diet, calculated as the sum of the percentage of reference daily values for nine qualifying nutrients, namely, protein, dietary fibre, vitamin A, vitamin C, vitamin D, Ca, Fe, K and Mg, minus the sum of the percentage of reference daily values for three disqualifying nutrients, namely, added sugars, saturated fats and Na. Reference daily values were determined for sex and age categories, based on the Dietary Reference Intakes for Japanese, 2020⁽⁴¹⁾, namely, the RDA for protein, vitamin A, vitamin C, Ca, Fe and Mg and tentative dietary goal for preventing lifestyle-related diseases for dietary fibre, K, saturated fats and Na. For added sugars, the conditional recommendation advocated by the WHO (i.e. upper limit of 5 % of energy)⁽⁴²⁾ was used because of the lack of a recommended value for added sugars in Japan, as well as their low intake levels⁽⁴³⁾. We calculated the NRF9.3 component and total scores based on the daily intake of each nutrient for each participant, which was adjusted for energy intake by the density method and then normalised for the sex- and age-specific Estimated Energy Requirement for a moderate level of physical activity (from the Dietary Reference Intakes for Japanese, 2020⁽⁴¹⁾) and expressed as a percentage of the reference daily value⁽²⁷⁾. These calculations were done for each meal type and for overall diet. Higher NRF9.3 scores indicated a better quality of the diet. A maximum possible score of 900 indicated a diet in which intakes per given amount of energy were above the reference daily values for the nine qualifying nutrients but below the reference daily values for the three disqualifying nutrients. In this study, dietary supplements were not considered during the nutrient intake calculation in any of the dietary assessment methods because it was our intention to assess nutrient intake from foods and beverages only.

Statistical analysis

Statistical analyses were performed using the SAS statistical software (version 9.4; SAS Institute Inc.). A two-tailed P value of < 0·05 was considered significant. Analyses were stratified by sex and conducted to determine the overall intake and intake for each meal type (breakfast, lunch, dinner and snacks). The dietary variables examined in this study included the total and component scores of HEI-2015 and NRF9.3, in addition to energy intake (MJ/d) and percentage of energy intake from each meal type. The amounts of snacks consumed were combined for analysis due to their relatively low intake in both methods. All dietary data were expressed as median and 25th and 75th percentiles. To assess the estimation ability at the group level, the median values of estimates derived from the MDHQ were compared with those derived from the DR using the Wilcoxon signed-rank test. The Spearman correlation coefficients between the MDHQ and DR estimates were used to assess the ability of the MDHQ to rank individuals in a population. In addition, agreement of the total scores of HEI-2015 and NRF9.3 between the MDHQ and DR was assessed using the Bland–Altman analysis⁽⁴⁴⁾. To examine the proportional bias between the MDHQ and DR, the Bland–Altman analysis was accompanied by the linear regression analysis⁽⁴⁵⁾. Identical analyses were conducted to assess the web MDHQ and paper MDHQ. The findings (tables and figures) on the web MDHQ are provided in the “Results” section, whereas those on the paper MDHQ are provided as online Supplementary Materials.

Results

This study included 111 women and 111 men aged 30–69 years and 30–76 years, respectively (Table 1). The median BMI (in kg/m²) was 22·0 for women and 23·5 for men.

Table 1. Basic characteristics of the study population

P25, 25th percentile; P75, 75th percentile.

* Based on self-report.

† Calculated using the self-reported body height and weight.

Results on the web version of Meal-based Diet History Questionnaire

Median estimations

The median estimates of the total and component scores of HEI-2015 and NRF9.3, energy intake and percentage of energy intake derived from the DR and web MDHQ are shown in Table 2 for women and Table 3 for men, according to the meal type. The number of HEI-2015 components (n 13 in total) showing no significant differences in women was 8 for overall diet, 8 for breakfast, 10 for lunch, 4 for dinner and 6 for snacks. The corresponding number in men was 5, 8, 9, 4 and 5, respectively. The number of NRF9.3 components (n 12 in total) showing no significant differences in women was 6 for overall diet, 3 for breakfast, 8 for lunch, 7 for dinner and 11 for snacks. The corresponding number in men was 7, 9, 6, 4 and 8, respectively.

Table 2. Median estimates of the total and component scores of Healthy Eating Index-2015 (HEI-2015) and Nutrient-Rich Food Index 9.3 (NRF-9.3), energy intake and percentage of energy intake derived from the 4-d weighed dietary record (DR) and those derived from the web version of the Meal-based Diet History Questionnaire (MDHQ) in 111 Japanese women, according to meal type*

P25, 25th percentile; P75, 75th percentile.

* The values derived from the MDHQ were compared with those derived from the DR using the Wilcoxon signed-rank test (a: P < 0·05, b: P < 0·01 and c: P < 0·001; shown in bold).

† Calculated as the sum of all components scores. A maximum score is 100. A maximum score for each component is as follows: 5 for total fruits, whole fruits, total vegetables, greens and beans, total protein foods, and seafood and plant proteins and 10 for whole grains, dairy products, fatty acids, refined grains, Na, added sugars and saturated fats. A higher score indicates a higher diet quality (i.e. a lower intake for refined grains, Na, added sugars, and saturated fats components and a higher intake for other components).

‡ Defined as the ratio of the sum of PUFA and MUFA to SFA.

§ Calculated as the sum of scores for nine nutrients to encourage (i.e. protein, dietary fibre, vitamins A, C and D, Ca, Fe, K and Mg) minus the sum of scores for three nutrients to limit (i.e. added sugars, saturated fats and Na). A maximum score is 900. For each component, a maximum score is 100, except for added sugars, saturated fats, and Na components, for which a maximum score is infinite depending on the intake level. A higher score indicates a higher diet quality, except for added sugars, saturated fats and Na components, for which a higher score indicates an unfavourable dietary intake (i.e. higher intakes of added sugars, saturated fats and Na).

Table 3. Median estimates of the total and component scores of Healthy Eating Index-2015 (HEI-2015) and Nutrient-Rich Food Index 9.3 (NRF9.3), energy intake and percentage of energy intake derived from the 4-d weighed dietary record (DR) and those derived from the web version of the Meal-based Diet History Questionnaire (MDHQ) in 111 Japanese men, according to meal type*

P25, 25th percentile; P75, 75th percentile.

* The values derived from the MDHQ were compared with those derived from the DR using the Wilcoxon signed-rank test (a: P < 0·05, b: P < 0·01 and c: P < 0·001; shown in bold).

† Calculated as the sum of all components scores. A maximum score is 100. A maximum score for each component is as follows: 5 for total fruits, whole fruits, total vegetables, greens and beans, total protein foods, and seafood and plant proteins and 10 for whole grains, dairy products, fatty acids, refined grains, Na, added sugars and saturated fats. A higher score indicates a higher diet quality (i.e. a lower intake for refined grains, Na, added sugars and saturated fats components and a higher intake for other components).

‡ Defined as the ratio of the sum of PUFA and MUFA to SFA.

§ Calculated as the sum of scores for nine nutrients to encourage (i.e. protein, dietary fibre, vitamins A, C and D, Ca, Fe, K and Mg) minus the sum of scores for three nutrients to limit (i.e. added sugars, saturated fats and Na). A maximum score is 900. For each component, a maximum score is 100, except for added sugars, saturated fats and Na components, for which a maximum score is infinite depending on the intake level. A higher score indicates a higher diet quality, except for added sugars, saturated fats and Na components, for which a higher score indicates an unfavourable dietary intake (i.e. higher intakes of added sugars, saturated fats and Na).

For the HEI-2015 total score, the web MDHQ provided higher median values for breakfast (in women only: +5·7; with no significant difference in men: +1·4) and dinner (+3·1 for women and +2·4 for men) and lower median values for snacks (–8·9 for women and –6·1 for men) than the DR, although these differences were relatively small, particularly for dinner. There were no significant differences observed for overall diet (–1·0 for women and –0·9 for men) and lunch (+1·2 for both sexes). For the NRF9.3 total score, the web MDHQ provided higher median values for breakfast (+83) and dinner (+27) and a lower median value for overall diet (–21) than the DR in women, although again the differences were relatively small, except for breakfast. No significant differences were observed for lunch (+41) and snacks (–2). In men, no significant difference was observed, except for overall diet, for which the web MDHQ provided a lower median value (–21) than the DR.

For energy intake, the web MDHQ provided lower median values for overall diet, breakfast, lunch and dinner and higher median values for snacks than the DR in both sexes. When expressed as percentage of energy intake, the web MDHQ provided higher median values for snacks and lower median values for lunch and dinner than the DR, with no significant differences for breakfast in either sex.

Spearman correlations

Table 4 shows Spearman correlation coefficients between estimates of the total and component scores of HEI-2015 and NRF9.3, energy intake and percentage of energy intake derived from the DR and web MDHQ. For the HEI-2015 components, median correlation coefficients were 0·39 for overall diet, 0·54 for breakfast, 0·26 for lunch, 0·28 for dinner and 0·21 for snacks in women. The corresponding values in men were 0·28, 0·55, 0·24, 0·20 and 0·22, respectively. For the NRF9.3 components, median correlation coefficients were 0·47 for overall diet, 0·41 for breakfast, 0·33 for lunch, 0·42 for dinner and 0·27 for snacks in women. The corresponding values in men were 0·45, 0·50, 0·29, 0·32 and 0·27, respectively. For the HEI-2015 total score, median correlation coefficient was 0·43, with a range from 0·12 (snacks in women) to 0·68 (breakfast in men). For the NRF9.3 total score, median correlation coefficient was 0·47, with a range from 0·26 (snacks in men) to 0·65 (breakfast in men). For energy intake variables, median correlation coefficient was 0·45, with a range from 0·27 (percentage of energy from lunch in women) to 0·65 (energy from breakfast in men).

Table 4. Spearman correlation coefficients between estimates of the total and component scores of Healthy Eating Index-2015 (HEI-2015) and Nutrient-Rich Food Index 9.3 (NRF9.3), energy intake and percentage of energy intake derived from the 4-d weighed dietary record (DR) and those derived from the web version of the Meal-based Diet History Questionnaire (MDHQ) in 111 Japanese women and 111 Japanese men, according to meal type*

* Values are Spearman correlation coefficients (a: P < 0.05, b: P < 0.01, and c: P < 0.001; shown in bold). For the whole grains component in snacks, Pearson correlation coefficients were not available because all the participants were non-consumers in the MDHQ.

† Calculated as the sum of all components scores. A maximum score is 100. A maximum score for each component is as follows: 5 for total fruits, whole fruits, total vegetables, greens and beans, total protein foods, and seafood and plant proteins and 10 for whole grains, dairy products, fatty acids, refined grains, Na, added sugars and saturated fats. A higher score indicates a higher diet quality (i.e. a lower intake for refined grains, Na, added sugars, and saturated fats components and a higher intake for other components).

‡ Defined as the ratio of the sum of PUFA and MUFA to SFA.

§ Calculated as the sum of scores for nine nutrients to encourage (i.e. protein, dietary fibre, vitamins A, C and D, Ca, Fe, K and Mg) minus the sum of scores for three nutrients to limit (i.e. added sugars, saturated fats and Na). A maximum score is 900. For each component, a maximum score is 100, except for added sugars, saturated fats and Na components, for which a maximum score is infinite depending on the intake level. A higher score indicates a higher diet quality, except for added sugars, saturated fats and Na components, for which a higher score indicates an unfavourable dietary intake (i.e. higher intakes of added sugars, saturated fats and Na).

Bland–Altman plots

Figure 2 shows Bland–Altman plots assessing the agreement between estimates of the HEI-2015 total score derived from the DR and those derived from the web MDHQ, according to the meal type. As mentioned above, the mean difference (MDHQ − DR) was relatively small in any analysis, with a range of −9 (snacks in women) to +4 (dinner in women). Regardless of sex and meal type, the limits of agreement (mean difference plus-minus 1·96 standard deviation of the difference) were generally wide, indicating poor to moderate agreement at the individual level. There was no indication of proportional bias between the web MDHQ and DR, except for snacks in both sexes, in which the HEI-2015 total score tended to be underestimated by the web MDHQ as the average score increased.

Fig. 2. Bland–Altman plots assessing the agreement between estimates of the Healthy Eating Index-2015 (HEI-2015) total score derived from the 4-d weighed dietary record (DR) and those derived from the web version of the Meal-based Diet History Questionnaire (MDHQ) in 111 Japanese women (a: overall diet, c: breakfast, e: lunch, g: dinner and i: snacks) and 111 Japanese men (b: overall diet, d: breakfast, f: lunch, h: dinner and j: snacks), according to meal type.

Bland–Altman plots for the NRF9.3 total score (Fig. 3) generally provided similar findings. The mean difference (MDHQ − DR) was again relatively small in any analysis, with a range of –30 (overall diet in women) to +66 (snacks in men). Regardless of sex and meal type, the limits of agreement were generally wide, indicating poor to moderate agreement at the individual level. Furthermore, with some exceptions, there was an indication of proportional bias between the web MDHQ and DR. The NRF9.3 total scores for overall diet (men only), dinner (both sexes) and snacks (both sexes) tended to be overestimated by the web MDHQ as the average score decreased, while the total score for lunch in women tended to be underestimated by the web MDHQ as the average score decreased.

Fig. 3. Bland–Altman plots assessing the agreement between estimates of the Nutrient-Rich Food Index 9.3 (NRF9.3) total score derived from the 4-d weighed dietary record (DR) and those derived from the web version of the Meal-based Diet History Questionnaire (MDHQ) in 111 Japanese women (a: overall diet, c: breakfast, e: lunch, g: dinner and i: snacks) and 111 Japanese men (b: overall diet, d: breakfast, f: lunch, h: dinner and j: snacks), according to meal type.

Results on the paper version of Meal-based Diet History Questionnaire

Identical analyses of the paper MDHQ were conducted (online Supplementary Table S2 for median estimations, online Supplementary Table S3 for Spearman correlations, online Supplementary Fig. S2 for Bland–Altman plots for the HEI-2015 total score and online Supplementary Fig. S3 for Bland–Altman plots for the NRF9.3 total score). The results for the paper MDHQ were generally similar to those for the web MDHQ, except for somewhat high Spearman correlation coefficients between the paper MDHQ and the DR.

Discussion

To our knowledge, this is the first study to examine the relative validity of the MDHQ, a novel, purpose-built, dedicated dietary assessment questionnaire to collect data on dietary intake at each meal type, for assessing the overall diet quality and quality of each meal type. Using the 4-d weighed DR as a reference method, this study showed that both the web and paper versions of the MDHQ had an acceptable ability for ranking individuals according to the quality of overall diet, breakfast, lunch and dinner (but not snacks) as assessed using the HEI-2015 and NRF9.3. In contrast, the ability for estimating diet quality measures was generally limited, both at the group and at the individual levels.

For the median or mean estimation of overall diet quality, the present findings are broadly comparable with the results of previous relative validation study of the DHQ and BDHQ⁽²⁷⁾. In both women and men (n 121 for each), the mean total scores of HEI-2015 estimated by the DHQ (57·3 and 54·8, respectively) or BDHQ (58·3 and 56·5, respectively) were slightly higher than those estimated by the 16-d DR (55·4 and 54·3, respectively)⁽²⁷⁾. The mean total scores of NRF9.3 estimated by the DHQ were lower than that by the DR in both women (675 v. 704) and men (674 v. 728), while that estimated by the BDHQ (759) was higher than that by the DR in women, with no difference in men (740)⁽²⁷⁾.

For the ability to rank individuals in a population, the Pearson correlation coefficient of the HEI-2015 total score estimated by the DHQ and BDHQ with that by the DR was 0·57 and 0·52 in women, respectively, and 0·51 and 0·43 in men, respectively⁽²⁷⁾. The Pearson correlation coefficient with regard to NRF9.3 total score was 0·61 for both the DHQ and BDHQ in women and 0·55 for the DHQ and 0·37 for the BDHQ in men⁽²⁷⁾. Only a few other studies have examined the validity (ranking ability) of dietary assessment questionnaires using other diet quality measures. For example, the Pearson correlation coefficient between the Diet Quality Index Revised estimated by an FFQ and that estimated by 2 × 7-d DR was 0·66 in 127 US men⁽⁴⁶⁾. For a modified Mediterranean diet score and a Mediterranean-like diet score calculated from an FFQ, the Pearson correlation coefficients with those derived from 10 or more 24-h dietary recalls were 0·48 and 0·62, respectively, in 107 Spanish adults⁽⁴⁷⁾. The Pearson correlation coefficient between the Dutch Healthy Diet Index calculated using an FFQ and that calculated using 2 × 24-h dietary recalls was 0·48 in 121 adults⁽⁴⁸⁾. For a diet quality score assessing the compliance with the American Cancer Society dietary guidelines for cancer prevention, the Pearson correlation coefficient between an FFQ and 4 or more 24-h dietary recalls was 0·65 for 244 men and 0·54 for 433 women⁽⁴⁹⁾. In a US study, the Spearman correlation coefficients between six diet quality scores (including the Alternate Healthy Eating Index-2010 and the Dietary Approaches to Stop Hypertension Trial score) derived from an FFQ and those derived from 2 × 7-d DR ranged from 0·43 to 0·66 in 652 men and from 0·47 to 0·67 in 742 women⁽⁵⁰⁾. Taken together, the present study suggests that the MDHQ’s ability for ranking individuals according to a measure of overall diet quality is not inferior to that of the FFQ mentioned above, as well as the DHQ and BDHQ.

We are unaware of previous studies in which the validity of diet quality (and energy intake variables) for each meal type was assessed. Generally, we found that the level of concordance between diet quality scores derived from the MDHQ and 4-d DR was similar across all meal types based on the median intake estimation and impressions from Bland–Altman analysis, but the ability to rank individuals according to diet quality level was highest for breakfast, moderate for lunch and dinner and lowest for snacks. This finding may be due to the large between-person variability of food intake patterns at breakfast compared with that at lunch, dinner and snacks^(6,19–22) . Alternatively, this may reflect the complex nature of lunch and dinner in terms of food consumption patterns as well as difficulty assessing snacks due to their low intake^(6,19–22) . To partially support this finding, the median Pearson correlation coefficient between energy intake from twelve food groups estimated using an FFQ and a 7-d DR in a small study of Japanese adolescent girls (n 63) was higher at breakfast (0·71) than that at lunch (0·38) and dinner (0·44); this FFQ was not designed to assess the snack intake⁽⁵¹⁾. Similar results were also observed in a small sample of Japanese adults (twenty-nine men and sixty women)⁽⁵²⁾. Nevertheless, given that the results on energy intake are also satisfactory as well as the lack of this kind of dietary assessment tool, the present findings generally support the appropriateness of the MDHQ for assessing meal-specific diet quality, in addition to overall diet quality.

Irrespective of sex, meal type and diet quality score, the Bland–Altman plots showed poor to moderate agreement between the MDHQ and DR. This is generally consistent with a previous study in Spain mentioned above⁽⁴⁷⁾ and our previous study of the DHQ and BDHQ⁽²⁷⁾. Thus, the absolute score of HEI-2015 and NRF9.3 should be interpreted with considerable caution, particularly at the individual level.

In this study, the findings for the web MDHQ were generally similar to those for the paper MDHQ, although the Spearman correlation coefficients with the DR were somewhat high for the paper MDHQ compared with that for the web MDHQ. This is not surprising given that the paper MDHQ was completed after conducting the DR, while the web MDHQ was completed before conducting the DR. While online questionnaires are preferred for administrative and cost purposes, in the real-world settings, not all study participants may be willing to complete the online questionnaires. Thus, a direct comparison between the web and paper versions of the MDHQ is warranted to assess the comparability or compatibility of these two modes but is beyond the scope of this study.

Several limitations in the present study warrant mention. First, whereas the present study was conducted in diverse regions (fourteen of forty-seven prefectures), the present population consisted of volunteers, not a nationally representative sample of the Japanese population. The participants may have been biased towards greater health consciousness, higher socio-economic status or both. For example, the education level in the present population was higher than that in a national representative sample of women (55·9 % completed junior high school or high school, 27·6 % completed college or technical school and 15·6 % completed a university degree or higher) and men (52·9, 12·9 and 33·7 %, respectively)⁽⁵³⁾. However, the mean HEI-2015 for overall diet derived from the DR in the present population (51·6 for women and 49·4 for men) was somewhat lower than that in a national representative sample (52·9 for women and 51·3 for men)⁽⁵⁴⁾, which appears to be mainly due to lower fruit intake in the present population. Meanwhile, the prevalence of current smokers and mean values of body height, body weight and BMI in the present participants⁽²³⁾ were similar to those in a nationally representative sample (women: 7·6 %, 154·3 (sd 6·7) cm, 53·6 (sd 9·2) kg and 22·5 (sd 3·7) kg/m², respectively; men: 27·1 %, 167·7 (sd 6·9) cm, 67·4 (sd 12·0) kg and 23·9 (sd 3·6) kg/m², respectively)⁽⁵⁵⁾. Ideally, further validation should be conducted using a more representative sample.

Second, the weighed DR, a reference method in this study, is susceptible to measurement errors due to the erroneous recording and potential changes in eating behaviour⁽¹⁴⁾. However, the weighed DR is the first method of choice for validating the dietary assessment questionnaires because the errors in weighed DR are thought to be less correlated with those in dietary assessment questionnaires compared with the errors in 24-h dietary recall or other instruments that rely on memory⁽¹⁴⁾. Additionally, although the dietary recording period was set to 4 d (to avoid lower participants motivation and even alteration of dietary habits potentially caused by a long-term DR⁽⁵⁶⁾), this duration might not be sufficient for capturing estimates of habitual intake. Considering that increasing the number of recording days in the reference method improves the apparent validity of a dietary assessment questionnaire^(14,57) , efforts to increase the duration of recording in the reference method would be important in future validation studies.

Finally, because the data collection was conducted over a narrow time frame (between August and October 2021; late summer and early autumn in Japan) as well as due to the 1-month time reference period used in the MDHQ, potential seasonal differences in dietary intake^(58–60) and thus in the validity of the MDHQ could not be considered in the present study. However, the results of our previous validation study of the DHQ and BDHQ suggested that a single administration of a questionnaire assessing the dietary habits during the previous month may reasonably capture the habitual dietary intake over a longer period (i.e. 1 year)^{(27,34,35,61)} . There is no strong reason to consider that the MDHQ is an exception in this regard.

In conclusion, compared with the 4-d DR, both the web and paper versions of the MDHQ showed an acceptable ability for ranking individuals according to the quality of overall diet, breakfast, lunch and dinner (but not snacks) as assessed using the HEI-2015 and NRF9.3. In contrast, the ability of the MDHQ for estimating diet quality measures was generally limited, both at the group and at the individual levels. Taken together, we consider that the MDHQ, a novel, purpose-built, dedicated dietary assessment questionnaire to collect data on dietary intake at each meal type, might be useful for future nutritional epidemiological research on diet–disease relationships, not only for focusing on overall diet but also with a particular focus on meal quality, meal patterns and time of day of dietary intake, or chrono-nutrition research. Nevertheless, both the strengths and disadvantages of the MDHQ described in this study should be carefully considered in any setting.