In the UK and elsewhere there is an ongoing debate about the most effective methods of promoting healthy diets. Some potential methods involve identifying the foods that are most likely to contribute towards healthy and unhealthy diets, i.e. healthy and unhealthy foods. Some people would argue that an individual food should never be described as ‘healthy’ or ‘unhealthy’, but for many practical purposes there is a need to do so – in which case there is often a need for clear definitions.
For example, it has been suggested – by the UK Government and others – that there need to be restrictions on the promotion of ‘unhealthy’ foods to children, although often an equivalent term such as ‘high in fat, sugar or salt’ is used1. If these restrictions are to be implemented, there needs to be a clear definition of what constitutes an ‘unhealthy’ food. A recent European Union proposal for a regulation on nutrition and health claims seeks to restrict the use of claims on foods high in fat, saturated fat, trans-fat, sugars and salt/sodium2; such foods might be said to be ‘unhealthy’ and so again there needs to be a clear definition of ‘unhealthy’ foods for the purposes of this regulation.
In a previous paper we outlined a method for developing different definitions of ‘healthy’, ‘unhealthy’ and other such descriptors that might be applied to foods – a process now generally called ‘nutrient profiling’Reference Scarborough, Rayner and Stockley3. There may need to be different nutrient profile models for different purposes. For any one purpose a large number of different models with associated definitions are possible. Therefore there need to be ways of testing the validity of different nutrient profile models.
One method of validity testing that is commonly used is to examine the classifications of a number of different foods to assess whether the nutrient profile model performs appropriately. This assessment is generally done subjectively by the team developing the model. In general, ‘anomalies’ are sought and if there are too many of these then the model is rejected or modified. This subjective assessment is open to the bias of those carrying out the assessment. Accordingly, we and others are seeking more transparent and repeatable methods of assessing models.
One method that has been proposed is to use the subjective judgements of qualified experts of a panel of representative foods, but to collect these in a standardised and repeatable wayReference Rayner, Scarborough and Stockley4, Reference Azais-Braesco, Goffi and Labouze5. In line with this proposal, we have carried out a survey of a defined sample of nutritionists and dietitians (nutrition professionals) in the UK to create a standard ranking of the ‘healthiness’ of 120 foods representative of the British diet. The results of the survey can be used to compare the ranking or categorisation of foods by a nutrient profile model, and the results of doing this for different nutrient profile models are provided in the accompanying paperReference Scarborough, Boxer, Rayner and Stockley6.
The results of this survey have also been analysed to assess which factors influenced the nutrition professionals' classifications of foods.
Methods
An online questionnaire to assess nutrition professionals' perception of the relative ‘healthiness’ of individual foods was developed and then administered. The online format was selected to facilitate randomisation of the order in which foods were presented and to simplify administration of the questionnaire.
Development and administration of the questionnaire
The questionnaire – in its final form – consisted of five parts: (1) background information to explain its purpose; (2) questions relating to the age, sex, ethnicity, educational background, field or area of practice of the respondent and qualifications; (3) a worked example of the questions in the main body of the questionnaire; (4) the main body of the questionnaire; (5) a section which enabled the respondent to review their responses in the main body of the questionnaire and to make revisions.
In the main body of the questionnaire the respondents were asked to place 40 foods randomly selected from a master list of 120 foods in one of six positions, ranging from less healthy to more healthy. In the background information potential respondents were told that, for the purposes of the survey, a ‘more healthy’ food was a food which should be eaten frequently and/or in large amounts by a person aiming to meet public health nutrition recommendations and conversely that a ‘less healthy’ food was a food which should be eaten infrequently and/or in small amounts. Respondents were also asked to rate different foods compared with all foods, rather than foods from a similar category. To assist with categorisation, the energy (kcal), protein, carbohydrate, total sugars, fat, saturated fat, non-starch polysaccharide (NSP), sodium, calcium and iron contents per 100 g of the foods were provided.
The 120 foods used for the survey were selected from the McCance and Widdowson database of foodsReference Roe, Finglas and Church7. In selecting the foods the aim was to provide a representative sample of foods consumed in Britain. This was achieved by weighting the selection of foods from different food groups by the percentage of adults who regularly consume foods from that food group as reported in the National Diet and Nutrition Survey (NDNS) for adults aged 19–64 yearsReference Henderson, Gregory and Swan8. For example, approximately 22% of NDNS respondents consumed food from the ‘lamb and dishes’ food group. This proportion corresponded to a weighting of 0.7% of all food groups, which equates to 0.87 food selections for the questionnaire (rounded to one selection: ‘roast lamb chops’).
The development of the questionnaire involved three pilot studies. The pilot studies provided information on the number of foods that could be categorised in a reasonably short period of time, the optimal number of positions provided for categorisation, the nutrition information the nutrition professionals needed to make a categorisation and the size of the sample of nutrition professionals required for reasonable confidence intervals around the average scores for each food. Cronbach's α, a measure of the internal reliability of a questionnaire, was calculated for each pilot study. The α score for the final pilot study, which used the same format as the study questionnaire and a small sample of nutrition professionals recruited from the Food Standards Agency (n = 17), was 0.90.
The questionnaire was administered by sending a password-protected link for the questionnaire to 850 members of the paediatric and community subgroups of the British Dietetic Association (BDA) and all 2667 members of the (British) Nutrition Society (NS).
Analysis of the data
Prior to analysis of the data it was agreed that responses would be excluded if the respondent did not have a degree in dietetics, human nutrition, public health nutrition, human metabolism, nutritional biochemistry, nutritional medicine, international nutrition, sports nutrition, nutritional sciences or nutritional physiology. This was to ensure that only responses from people with a reasonable level of nutritional knowledge were included.
It was also agreed that responses were to be excluded if: (1) the respondent categorised less than 30 foods; (2) the respondent placed more than 80% of foods in any one position; or (3) the respondent placed any one of a set of 10 predetermined ‘less healthy’ foods in a healthier category than any one of a set of nine ‘healthier’ foods. These three types of exclusion were made to ensure that responses from respondents who had completed the questionnaire carelessly or maliciously were excluded.
The questionnaire allowed nutrition professionals to categorise foods in one of six positions on a scale labelled at one end as ‘less healthy’ and at the other ‘more healthy’. The least healthy position was allocated a score of 1 and the healthiest position a score of 6. Foods were then ranked on the basis of the average score awarded by the respondents. The responses to the questionnaire were analysed for statistical differences in the average scores by age, years of experience, sex, ethnicity, membership of the BDA or NS, and type of employer.
Forward stepwise multivariate regression analysis was then carried out, with the average score awarded by the nutrition professionals as the dependent variable and the nutritional and other information available to the respondents as the independent variables. This was to provide an estimate of the extent to which the variation in the average scores could be explained by features of the foods, and hence gain an insight into the decision-making of the nutrition professionals.
The first stage of the regression analysis involved the nutritional data provided to the respondents. For this analysis, the sodium, calcium and iron levels were converted to an ordinal scale: a score of 0 was awarded when no nutrient was present; 1 point was awarded when the food was in the first quartile of the distribution for levels of that nutrient; 2 points when the food was in the second quartile, etc. This was considered to be necessary as very high levels of these nutrients in a small number of foods could potentially distort the analysis. The levels of all the other nutrients were considered on a continuous scale because it was considered that the distortion that was possible for micronutrients (where the amount of nutrient per 100 g is effectively unbounded) was unlikely to be relevant for macronutrients (where the amount of nutrient per 100 g is bounded by 100 g).
A second stage of regression analysis included serving size and frequency of consumption as independent variables. Data on serving sizes and frequency of consumption were derived from NDNS data by matching the 120 foods from the online questionnaire with foods from the ‘nutrient databank’ used to analyse the NDNSReference Henderson, Gregory and Swan8. Frequency of consumption (number of servings consumed per week per 1000 people) and average serving size were then calculated using raw (unweighted) NDNS data. The product of these variables (a measure of total dietary consumption) was also included in the analysis. Frequency of consumption was converted to an ordinal scale (based on quartiles) for reasons similar to those for converting sodium, calcium and iron levels to such a scale.
A final stage of regression analysis included nine binary independent variables: drink, fruit/vegetable (for the foods in this category see Table 1), ‘wholemeal’ in description of the food, ‘takeaway’ in description of the food, ‘fried’ in description of the food, ‘canned’ in description of the food, ‘added sugar’ in description of the food, ‘added salt’ in description of the food, and ‘white’ in description of the food. Further analyses including interaction terms were also conducted, but made little difference to the results reported here.
Table 1 Ranking, average score and standard deviations around average scores of 120 foods, ordered by The Balance of Good Health food groups
R – rank; NS – nutrition professionals' average score; SD – standard deviation for nutrition professionals' average score.
Results
Response to the survey
There were 850 responses to the survey, giving a total response rate of 24%. Five hundred and thirty-four responses were from NS members (a response rate of 20%) and 316 were from members of the BDA (a response rate of 37%). Some members of the NS are members of the BDA and vice versa. Discussions with the administrators from the two organisations suggest that about 15% of potential respondents would have received two emails seeking to recruit them to the survey. If this estimate is correct then the overall response rate was 28%.
One hundred and forty-eight of the 850 responses were excluded from analysis: 136 because the respondent did not have a degree in a relevant subject, three because the respondent categorised less than 30 foods, two because the respondent placed more than 80% of foods in any one position, and seven because the respondent placed one or more of the set of ‘less healthy’ foods in a healthier category than one of the set of ‘healthier’ foods. This left a total of 702 responses for further analysis.
Ranking of foods
The 120 foods from the questionnaire were ranked according to the average score awarded by the nutrition professionals. Table 1 gives the ranking for each of the 120 foods together with the average score and the standard deviation around the average score. The foods have been grouped into six categories: the five food groups of the UK's food guide The Balance of Good Health and composite foods.
The healthiest food according to the nutrition professionals was ‘raw green peppers’ with an average score of 5.91. The least healthy food was ‘clotted cream’ with an average score of 1.21. Standard deviations around the average scores ranged from 0.32 for ‘raw green peppers’ to 1.54 for ‘diet cola’, with an average of 0.95. Standard deviations were largest for middle-ranking foods and smallest for high- and low-ranking foods.
The average rank of the foods from each of The Balance of Good Health groups is in accordance with the message of the food guide. The highest average rank (indicating the healthiest category of foods) is attained by foods in the ‘fruit and vegetable’ group, and the lowest average rank is attained by foods from the ‘foods high in fat, foods high in sugar’ group. The other four categories have broadly similar average ranks. The composite foods showed the highest variance in nutrition professionals' opinions, reflecting their difficulty in categorising such foods.
The respondents were then split into groups on the basis of their age, years of experience, sex, ethnicity, membership of BDA or NS, and type of employer. Table 2 shows the number of respondents in each of these groups and the number of foods where significant differences (P < 0.01) in the average score were found between the groups.
Table 2 Number of foods which displayed significant (P<0.01) differences in average score for different groups
Significance of difference in average scores determined by calculating test statistic that the two groups produced different average scores. Test statistic compared with two-tailed normal distribution.
Since the threshold for statistical significance was set at P = 0.01 and there were 120 foods in the questionnaire, one or two significant differences should be found for each grouping purely by chance. For the sex and ethnicity groupings there were few significant differences, indicating that sex and ethnicity had little effect on the average scores awarded to the foods (although the small number of male and non-white respondents means that this is a weak conclusion). However, the age of the respondent and the number of years of experience (clearly related) did seem to have an effect, with older, more experienced respondents tending to perceive foods as healthier than the younger, less experienced respondents. The foods in question were not from any particular food group, suggesting that the older respondents tended to categorise all foods as slightly healthier and a random selection achieved significance. BDA members also tended to categorise more foods as healthier than NS members and employees of the National Health Service (NHS) tended to categorise more foods as healthier than academic nutrition professionals. But again the foods that were perceived as healthier did not come from any particular food group.
Factors affecting nutrition professionals' categorisations
Prior to the regression analysis an assessment was made to ascertain which of the potential explanatory variables were closely correlated. Inclusion of closely correlated independent variables in a regression analysis is problematic, as their effects on the dependent variable may not be independent of one another and interpretation of the results is therefore difficult. It is sometimes appropriate to remove variables from a regression analysis if they are likely to measure the same attribute as one or more of the other variables. In this case energy was removed from the analysis because of its high correlation with fat (r = 0.86, P < 0.05) and to a lesser extent with total sugars (r = 0.29, P < 0.05). An alternative would have been to remove fat but this would have been contrary to the data collected in the pilot study carried out with nutrition professionals from the Food Standards Agency, where more of the respondents declared that they used data on fat to inform their judgements than said they used the energy data (seven compared with four).
Carbohydrate was also excluded from the regression analysis because of its high correlation with total sugars (r = 0.69, P < 0.05). It was felt that total sugars and NSP fibre were a sufficient measure of the carbohydrate quality of the foods. Again, this decision was supported by the results of the pilot study. There was a high correlation between iron and NSP fibre (r = 0.64, P < 0.05) but this was considered to be accidental, possibly due to the over-representation of breakfast cereals in the sample (the correlation (Pearson's r) between iron and NSP levels in all the foods in the McCance and Widdowson database is 0.47), so neither iron nor NSP fibre was removed.
Table 3 suggests that nearly 50% of the variance in the nutrition professionals' average score can be explained by the nutritional data provided to them and specifically the data for fat, total sugars, NSP and sodium. Including further nutrients as independent variables in the model would increase the proportion of variance explained, but would result in a model with a poorer fit.
Table 3 Results of three stages of multivariate regression analysis
R 2 – explained variance; SE – standard error; NSP – non-starch polysaccharides.
* R 2 and SE calculated for each stepwise regression analysis. Final score (in bold) is for final model including each variable.
† Results for final model including each variable.
‡ Ordinal variable with foods categorised as follows: 0 = no nutrient present; 1 = lowest quartile of nutrient present; 4 = highest quartile of nutrient present.
§ 1 = yes; 0 = no.
Inclusion of serving size data had a small effect on the model, increasing the amount of explained variance by 3%, but also increasing the standard error of the residuals and thus suggesting a less accurate fit. The sign for the standardised coefficient for serving size suggests that the nutrition professionals tended to categorise foods with larger serving sizes as less healthy.
Adding the binary variables to the analysis changed the model dramatically. Serving size was no longer included in the model, and NSP fibre and sodium were also absent. In their place were ‘fruit/vegetable’, ‘takeaway’, ‘fried’ and ‘wholemeal’ (which could explain the absence of fibre from the model). The final model explained 64% of the total variance.
Discussion
The primary purpose of this project was to generate a standard ranking of a set of foods which can be used for comparisons with similar rankings produced by nutrient profile models. A secondary purpose was to explore how nutrition professionals rank the healthiness of foods.
On the whole, the standard ranking of foods generated by the survey of nutrition professionals seems in accordance with general healthy eating advice in the UK, although the relative positioning of some foods is surprising. For instance, ‘wholemeal fruit crumble’ was ranked as the 49th healthiest food, whilst ‘plums, stewed with sugar’ and ‘apple, stewed with sugar’ were ranked less healthy at 51st and 52nd healthiest, respectively. This is despite the fact that the crumble contains more sugar, fat and saturated fat per 100 g than either of the stewed fruits.
Similarly, the ranking of ‘takeaway stir-fry vegetables’ was surprisingly unhealthy (48th healthiest) for a meal with a very low fat and saturated fat content and a high fibre content. Indeed, it was ranked below ‘Lancashire hotpot’ (40th), ‘chilli con carne’ (41st), ‘vegetable cannelloni’ (42nd) and ‘beef bourguignon’ (43rd), all of which have higher fat and saturated fat contents when measured per 100 g.
These results are partially explained by the results of the third stage of the regression analysis. At this stage of the analysis, binary variables identifying the presence or absence of certain words in the descriptions of the foods were included. Three of these variables (‘takeaway’, ‘fried’ and ‘wholemeal’) were included in the model, as well as a binary variable which identified fruits and vegetables. Each of these variables improved the explanatory nature of the model (particularly ‘fruit/vegetables’, which was the first variable to be entered in the model and accounts for 26% of the variance in the nutrition professionals' average scores by itself) without affecting the accuracy of the model (as shown by the standard errors of the residuals). This implies that the respondents were using these descriptive prompts to guide their judgements over and above the nutritional data provided, leading to a situation where ‘takeaway stir-fry vegetables’ received a surprising low score while ‘wholemeal fruit crumble’ received a surprisingly high score.
When nutritional data alone were included in the regression analysis, just under 50% of the variance in the dependent variable was explained. Including the binary variables increased this to 64%, but over 30% of the variance remains unexplained. Part of this variance may be due to the respondents' knowledge of the health effects of foods, which depend on more than nutrient composition; for example, the relative bioavailability of nutrients, the effects of consuming foods in combination with others, etc. It may also be due to other concerns – for example, the freshness of foods or the degree to which they have been processed – factors repeatedly shown to be important in lay views of healthy eatingReference Paquette9. Finally, it may be due to the difficulty – even for nutrition professionals – in using all the nutritional information when categorising foods, e.g. when faced with a product that is both high in saturated fat and high in calcium.
There is an extensive literature on perceptions of healthy eating, reviewed recently by PaquetteReference Paquette9. She shows that most studies on perceptions of healthy eating have been carried out with the ordinary public, finding only one studyReference Keane and Willetts10 carried out with health professionals. The results of her review are concordant with the results of the present survey. For example, she found that fruit and vegetables were consistently regarded as healthy foods, and that fat was regularly regarded as an important nutritional determinant of the healthiness of individual foods.
Older, more experienced nutrition professionals, BDA members and NHS employees tended to categorise foods as healthier than their counterparts. However, it does not appear that these groups were favouring certain types of foods above others; rather that all foods were scored as healthier by these groups. As a consequence, the standard ranking of foods produced by the questionnaire should not be affected by the proportions of each of the groups in the sample. However, the increased perception of the healthiness of all foods by certain groups will lead to greater variance around the average scores than if the questionnaire was restricted to a single group of respondents.
The use of the standard ranking of foods to compare different nutrient profile models is described in the accompanying paperReference Scarborough, Boxer, Rayner and Stockley6. The high level of variance in the categorisations of some foods (particularly towards the centre of the rankings) is likely to influence adversely the ability of tests to distinguish between similar nutrient profile models. This is unfortunate, but unavoidable: the large sample size for this survey suggests that the confidence intervals are a true representation of the variation in opinions among UK nutrition professionals.
Moreover, it should be noted that the standard ranking of foods produced by this survey is likely to be affected by the particular cultural perspectives of nutrition professionals in the UK, e.g. in their views of ‘takeaway’ foods as discussed above. This makes the standard ranking most suitable for testing nutrient profiling models for use in the UK. Nevertheless, the methods used to derive the standard ranking would seem internationally applicable.
Finally it should also be noted that use of a standard ranking of foods derived from a survey of the views of nutrition professionals is not the only, and is unlikely to be the best, way of validating or comparing nutrient profile models. A better way would be to use information derived from diets associated with better health outcomes.
Acknowledgements
Sources of funding: The development and administration of the questionnaire was funded by the Food Standards Agency. M.R. and P.S. are supported by the British Heart Foundation.
Conflict of interest declaration: There are no further conflicts of interest for any author.
Authorship responsibilities: Each author was involved in the development of the questionnaire, analysis of the data and drafting of the final manuscript.
Acknowledgements: We would like to thank the Nutrition Society and the British Dietetic Association for their help with administering this questionnaire; Martin Campbell and all at Baigent Ltd who helped with the design and administration of the online questionnaire; Rosemary Hignett, Mark Browne, Robin Clifford, Jennifer Burns and others from the Food Standards Agency; and Dr Steven Allender of the British Heart Foundation Health Promotion Research Group.
