Preparing dossiers: strength of the evidence and problems of proof

Abstract

In Europe, for authorisation of a health claim, applicants must follow the procedures in the legislation and in the guidelines for submission of a dossier set out by the European Food Safety Authority. The Functional Foods in Europe (FUFOSE) and Process for the Assessment of Scientific Support for Claims on Foods (PASSCLAIM) projects underpinned the laws and provided criteria against which the quality of the totality of the available data could be judged. Whereas the regulations and PASSCLAIM require an assessment of the extent to which cause and effect can be demonstrated between a food category, a food or constituent and a health benefit, the European Food Safety Authority requires conclusive evidence of cause and effect. This latter standard of proof and a focus on randomised controlled trials done on isolated components and using validated physiological biomarkers may not always be appropriate to assess nutrition science. The aims of this paper are to address the strengths and weaknesses of different sources of evidence that contribute to the totality of the available data, to undertake a critical examination of the application of a drug-like assessment model in evidence-based nutrition and to encourage research on new biomarkers of health and homeostatic adaptability. There is a need for (a) a robust and pragmatic scientific framework for assessing the strength, consistency and biological plausibility of the evidence, and (b) consumer understanding research on claims that use qualifying language and/or graphics to reflect the weight of evidence. Such scientific, policy and communication approaches are proportionate and could help stimulate academic research, promote fair trade and product innovation and contribute to consumer education about food and health.

Abbreviations:

EFSA

European Food Safety Authority

EC

European Commission

FUFOSE

Functional Foods in Europe

PASSCLAIM

Process for the Assessment of Scientific Support for Claims on Foods

RCT

randomised controlled trial

SSA

significant scientific agreement

Background to the European legislation on health claims

Consumers should be able to make food choices based on clear and accurate information. Hence, an important objective of the European Regulation on nutrition and health claims is to ensure that claims on foods can be properly justified and scientifically substantiated. The Regulation EC 1924/2006^{( 1 )} applies to claims made in commercial communications, whether with respect to labelling, presentation or advertising, regarding foods and food supplements to be delivered as such to the final consumer. The law sets out the conditions of use, establishes a system for their scientific evaluation and creates European Community lists of authorised claims. All claims have to comply with the general principles that they are not false, ambiguous or misleading, as laid down in Article 3, and claims have to be scientifically substantiated as laid down in Article 6. Health claims and reduction of disease-risk claims can be made for a food category, a food or one of its constituents based on an assessment of the totality of the available data and weighing of the evidence. A health claim is defined as any claim that states, suggests or implies that a relationship exists between a food category, a food or one of its constituents and health. A reduction of disease-risk claim is defined as any health claim that states, suggests or implies that the consumption of a food category, a food or one of its constituents significantly reduces a risk factor in the development of a human disease. Health claims based on ‘generally-accepted scientific evidence’ fall under Article 13.1, whereas those based on newly developed scientific evidence and/or where those claims include a request from the applicant for the protection of proprietary data fall under Article 13.5. Disease risk reduction claims and claims for children's development and health fall under Article 14.

For generally accepted claims under Article 13.1 of the regulations, a full scientific dossier was not required. Instead, each member state had to compile a list of claims and submit them to the European Commission (EC), together with the conditions of use applying to them and references to the relevant scientific justification. In 2006–2007, in the absence of any guidance from the EC, the European Food Safety Authority (EFSA) or the competent authorities in member states, the food and food supplements industries in Europe promulgated the need for a standardised format to help compilation of the information required for the national lists of health claims, and attempted to create a harmonised framework for this complex exercise^{( 2 )}. The European ‘industry list’ compiled by the Confederation of EU Food and Drink Industries, European Botanical Forum, European Federation of Associations of Health Product Manufacturers and European Responsible Nutrition Alliance trade associations resulted in a priority listing of 776 claims. The EC subsequently forwarded 4367 Article 13.1 claims from member states to the EFSA and requested it to provide scientific advice about, and to provide opinions on, the extent to which a cause and effect relationship has been established between consumption of a food/constituent and the claimed beneficial effect in human subjects, whether the magnitude of the effect is related to the quantity consumed, and whether this quantity could reasonably be consumed as part of a balanced diet.

The claims under Article 13.1 are based on diet and health relationships that are documented extensively in the scientific literature and where there is general consensus in the scientific community. Subsequent guidance on the processes^{( 3 – 6 )} and the nature of the scientific evaluation by the EC and EFSA highlighted a considerable mismatch in expectations for all the parties concerned, particularly relating to the sufficiency of characterisation of food categories and foods, the focus by the EFSA on the use of randomised controlled trials (RCT) and isolated or purified components. The application and authorisation procedures under Article 13.1 are now effectively closed and health claims in future applications must be made under Articles 13.5 and 14. Applicants for health claims must now follow procedures set out in the regulation^{( 1 )} and implementing rules^{( 7 )}, which include submission of a comprehensive dossier of scientific evidence, a proposal for the wording of the claim and specific conditions for its use. In all cases, nutrition and health claims shall only be permitted if they are scientifically justified and the average consumer can be expected to understand the beneficial effects as expressed in the claim. (The average consumer is defined as one who is reasonably well informed and reasonably observant and circumspect, taking into account social, cultural and linguistic factors (Recital 16 (EC) 1924/2006).)

For functional foods and food supplements to deliver their potential public health benefits, consumers must have a strong confidence level in the scientific and regulatory processes used to support beneficial health effects and claims. As well as achieving a high degree of consumer protection, any regulatory framework needs to promote fair trade, stimulate academic research and encourage product innovation. Table 1 summarises how health claims could create substantial opportunities for a renaissance in European food biosciences and technology. A good example was the Integrated Project HEALTHGRAIN 2005–2010 (http://www.healthgrain.org). This project included five modules on consumer research, plant breeding and biotechnology, technology and processing, nutrition and metabolism and dissemination and technology transfer. Together with the forty-four project partners, leading European nutritionists and experts in communication to consumers joined the Nutrition Information Network and Consumer Communications Panel with thirty-four members from seventeen countries. The Industrial Platform consisted of fifty-nine member companies of thirteen countries, including twenty-five small/medium-sized enterprises. The objectives of the nutrition and metabolism module included work to explore further and understand the health benefits of whole-grain cereals and dietary fibres and their associations with reduced risk of developing diet-related diseases such as CHD, and to achieve an agreement on a European definition of whole grain as a food category.

Table 1. Opportunities for a renaissance in European food biosciences and technology

Process for the assessment of scientific support for health claims on food

The Functional Foods in Europe and Process for the Assessment of Scientific Support for Claims on Foods

In Europe, much attention has been paid to the effects of foods and food constituents on body functions and health. The Consensus Document on Scientific Concepts of Functional Foods in Europe (FUFOSE)^{( 8 )} and the project Process for the Assessment of Scientific Support for Claims on Foods (PASSCLAIM)^{( 9 )} set out how health-related claims could be linked to solid scientific evidence. These concerted action projects were funded by the EC, managed by the International Life Sciences Institute Europe, and their objectives were to expedite and underpin the development of a harmonised EU regulatory process on health claims. A review of the processes and methodologies for the scientific substantiation of health claims around the world was set out under PASSCLAIM by Richardson et al. ^{( 10 )}. The main objective of this paper was to identify common new ideas, definitions and best practice globally. In addition to the regulatory developments in Europe, the USA^{( 11 – 17 )}, China^{( 18 )} and elsewhere, the PASSCLAIM initiative influenced the Codex Alimentarius guidelines^{( 19 , 20 )} that set out a common approach for substantiation of health claims, which is an important step towards harmonisation globally. In his paper on the Codex recommendations, Grossklaus^{( 21 )} concluded that health claims need to reflect the totality of the available data for emerging as well as consensus science, and that the substantiating evidence should be proportionate to the claim.

The Process for the Assessment of Scientific Support for Claims on Foods criteria for the scientific substantiation of claims

The six criteria are shown in Table 2. These criteria describe the standards by which the quality and relevance of the scientific evidence including new data should be judged, and thus the extent to which claims based on them can be said to be scientifically valid^{( 9 )}. Criterion 6 emphasises that a claim should be scientifically substantiated by taking into account the totality of the available data and by weighing the evidence. PASSCLAIM placed no emphasis on the prioritisation of evidence and studies, and argued for the need to take a broad approach to the evidence base. The PASSCLAIM criteria were intended to offer a standard against which the quality of existing evidence could be transparently assessed. It is the integration of findings from several different types of evidence and the degree of consistency between them that is the scientific standard that needs to be applied to reflect state-of-the-art nutrition science. The assessment of the adequacy of the evidence against the PASSCLAIM criteria was intended not only to permit transparent assessments of the totality of the available data but also to help applicants for health claims from research institutes as well as industry to undertake appropriate research and to prepare better applications and presentation of data in a scientific dossier. The criteria were also intended to facilitate feedback from the scientific assessors to applicants and to help identify gaps, uncertainties, variability and inconsistencies in the evidence base and to enable their clear expression in the scientific opinion given by assessors to the EC food policy managers, legislators and communicators.

Table 2. Process for the Assessment of Scientific Support for Claims on Foods criteria for the scientific substantiation of claims^{( 9 )}

Scientific and technical guidance for the preparation and presentation of an application for authorisation of a health claim

In accordance with Article 31 of Regulation (EC) No. 178/2002^{( 22 )}, the EC requested the EFSA to issue an opinion on scientific and technical guidance for authorisation of health claims. This opinion was adopted on 6 July 2007^{( 3 )}. Commission Regulation No. 353/2008^{( 7 )} of April 2008 established implementing rules for applications, and in June 2011 the EFSA Panel on Dietetic Products, Nutrition and Allergies revised its earlier 2007 opinion related to this scientific and technical guidance with regard to the form to be used for the submission of an application for claims under Articles 13.5 and 14^{( 23 )}. The guidelines outline:

1. i. The information and scientific data that must be included in the application.

2. ii. The hierarchy of different types of data and of study designs reflecting the relative strength of evidence that may be obtained from different types of studies.

3. iii. Instructions for presenting summaries of data so as to highlight the relevant aspects related to the design, results and quality of the studies (only in the 2007 guidelines).

4. iv. The key issues that should be addressed in the application to substantiate the health claim.

The applicant must provide all the available scientific data including data in favour and not in favour that are pertinent to the health claim in a stand-alone dossier. Only a relationship between a food/constituent and a single claimed effect can be the object of each applicant. The data provided in the application should be organised into five parts, as shown in Table 3 ^{( 3 , 23 )}.

Table 3. Representation of the organisation of an application for an Article 13.5 or 14 health claim^{( 3 , 33 )}

Part 3 requires a tabulation of all pertinent studies identified including:

1. i. Human intervention and observational studies dealing with the relationship between the consumption of the food/constituent and the claimed effect, including human studies dealing with the mechanisms by which the food/constituent could be responsible for the claimed effect (mechanistic studies), or studies on bioavailability.

2. ii. Animal studies dealing with, for example, the mechanism by which the food/constituent could be responsible for the claimed effect (mechanistic studies).

3. iii. In vitro studies based on either human or animal biological samples.

The EFSA guidelines provide detailed tabulations for presentation of the study types for experimental intervention studies (such as RCT ranging from full randomisation to non-randomised, non-controlled studies), for observational studies (from prospective cohort studies to case reports) and for undertaking a comprehensive review of scientific literature including appropriate inclusion and exclusion criteria. The guidelines, together with the legislation, are essential reading for scientists in academia and in industry whose research work is related to the provision of evidence for inclusion in an application for authorisation of a health claim. Learnings from successful EFSA opinions are also valuable sources of information for applicants, e.g. the Article 14 health claim relating to oat β-glucan and lowering blood cholesterol and reduced risk of heart disease^{( 24 )}, and the Article 13.5 health claim related to water-soluble tomato concentrate and inhibition of platelet aggregation^{( 25 – 27 )}.

European Food Safety Authority assessments

As specified in the Regulations^{( 1 , 7 )}, the health claims should be substantiated by taking into account the totality of the available scientific data and by weighing the evidence, subject to the specific conditions of use. In particular, the evidence should demonstrate the extent to which:

1. i. The claimed effect of the food/constituent is relevant for human health.

2. ii. A cause and effect relationship is established between the consumption of the food/constituent and the claimed effect in human subjects (such as the strength, consistency, specificity, dose–response and biological plausibility of the relationship).

3. iii. The quantity of the food/constituent and pattern of consumption required to obtain the claimed effect could reasonably be achieved as part of a balanced diet.

4. iv. The specific study group(s) in which the evidence was obtained is representative of the target population for which the claim is intended.

In practice^{( 6 )}, the outcomes of each claim assessment by the EFSA has one of three possible conclusions:

1. i. A cause and effect relationship has been established between the consumption of the food/constituent and the claimed effect. YES

2. ii. The evidence provided is insufficient to establish a cause and effect relationship between the consumption of the food/constituent and the claimed effect, i.e. the evidence of cause and effect is not conclusive because the evidence is emerging and/or conflicting, and the claim is not substantiated by generally accepted scientific evidence. NO

3. iii. A cause and effect relationship is not established between the consumption of the food/constituent and the claimed effect (i.e. where the scientific evidence is limited and is not supported by ‘generally accepted scientific evidence’). NO

There is no doubt of the thoroughness of the assessments carried out by the EFSA within a short timescale, but the vast majority of health relationships have received unfavourable opinions. The reasons for rejection by the EFSA, in many cases, are entirely justified, and Table 4 sets out an analysis of these rejections, which illustrate the approaches used in the process of evaluation. In summary, it is clear that most successful outcomes focus on pure food components/bioactives that are very well characterised and for which physiological benefits can be demonstrated by the use of human intervention studies using validated biomarkers. The EFSA also requires conclusive evidence of cause and effect. This pharmaceutical approach is very difficult, if not impossible, to achieve based on state-of-the-art nutrition science and poses a major challenge to undertake future research that would satisfy the EFSA requirements^{( 28 – 30 )}.

Table 4. Scientific reasons for rejection by the European Food Safety Authority of health claim applications

A legal perspective

Although the processes of authorisation differ, the same scientific principles apply to health claim assessments under Articles 13.1, 13.5 and 14, namely the substantiation should be based on the totality of the available data and by weighing the evidence. The legislation clearly requires the demonstration of the extent to which cause and effect can be demonstrated (and other criteria such as characterisation of the food category, food/constituent) and not conclusive evidence of cause and effect. It is vitally important for the scientific community and policy makers to recognise that the notion of ‘conclusive evidence’ of cause and effect is neither a requirement of the legislation nor of PASSCLAIM. The notion of ‘extent’ refers to that of a ‘degree’, e.g. either small, moderate or large. The extent of cause and effect is determined by the strength, consistency and biological plausibility of the totality of the available data in support of a beneficial nutritional or physiological effect following the consumption of the food category or food/food constituent. The EFSA requirement for demonstration of conclusive evidence of cause and effect is not proportionate legally or scientifically, irrespective of the category of health claim. This drug-like approach had made it very difficult within reasonable timescales to achieve successful outcomes for many food/health relationships that reflect state-of-the-art nutrition science, many of which are cited extensively in the peer-reviewed, high-impact scientific journals. This situation could stifle rather than stimulate research and product innovation^{( 30 )}.

The European legislation^{( 1 )} states that health claims should only be authorised after scientific assessment of the highest possible standard. Although no one would disagree with the basic principles of scientific substantiation, the legal obligation to assess evidence using the highest possible standard cannot be automatically associated with the EFSA interpretations for demonstration of conclusive evidence of cause and effect. The scientific community routinely uses frameworks for assessing the strength, consistency and plausibility of the evidence, e.g. the WHO^{( 31 )}, the World Cancer Research Fund^{( 32 )} and the UK Department of Health Committee on Medical Aspects ^{( 33 )} assess the strength of evidence using categories such as ‘convincing’, ‘probable’, ‘possible’ and ‘insufficient’.

The expectations of applicants for a health claim were for a transparent scientific assessment that considered the strength, consistency and biological plausibility of the totality of the available data that could be sufficient to permit a conclusion by policy makers to draw management conclusions about the probability that a change in the dietary intake of a food category or a food/food constituent would result in a health benefit^{( 2 , 34 )}. As well as being a marketing tool, a health claim has the potential to enhance consumers’ nutrition knowledge and to promote healthy eating patterns, in addition to complementing national agendas in public nutrition education, health protection and improvement. As such, it was anticipated by industry that health claims would have an integral role in risk–benefit management and communication^{( 35 )}.

A Process for the Assessment of Scientific Support for Claims on Foods perspective

PASSCLAIM recognised the different interpretations of conflicting evidence and the potential variations in quality among individual studies. The quality of individual studies may differ, and it is possible that not all research has been, or will be, done to the highest standard, or even to a common standard. These difficulties can be due to the complexities of research in human studies and can also arise because data in support of a claim may have been taken from studies that had different primary objectives. PASSCLAIM stated that despite potential limitations in the research base, there may be complementarity between individual incomplete studies that allows an assessment of the totality of the evidence to substantiate a claim. Conversely, a review of all the studies taken together may reveal evidential inconsistencies that are not apparent from the review of a single study in isolation. PASSCLAIM also stated in the concluding comments on the six criteria, shown in Table 2, that the template needs to be applied intelligently and sensitively to existing and potential claims on a case by case basis with respect to both gaps in knowledge and the development of new knowledge^{( 9 )}.

Although PASSCLAIM provided a scientific framework to facilitate the assessment of scientific support for claims on foods, the project did not specifically address the second part of criterion 6 on the weighing of the evidence. However, it has been emphasised that the evaluation process should be transparent and that the grading of evidence into categories including ‘convincing’, ‘probable’, ‘possible’ and ‘insufficient’ could be considered useful in scientific evaluations, and to monitor the development of scientific substantiation^{( 36 )}. The establishment of an appropriate scientific framework to reflect the extent to which a cause and effect relationship can be demonstrated requires an equal, if not higher, standard for the assessment processes. The EFSA process of scientific assessment would have been strengthened, not weakened, by a clear statement of different levels of supporting evidence.

Evidence-based nutrition

Evidence-based nutrition is being used for three aspects of public health nutrition: (1) the establishment of Dietary Reference Values, including recommended intakes; (2) the development and revision of dietary guidelines; and (3) the validation of health claims on foods and food constituents^{( 37 )}. Guidelines advise people, for example, to eat less saturated fat. Health claims declare a benefit that lowering dietary saturated fat can lower the blood cholesterol level, a risk factor for CVD, connected with a nutrition claim that a food is low in saturated fat according to the criterion in the Annex of Regulation (EC) 1924/2006^{( 1 )}.

The type and extent of the evidence required for a health claim is determined by whether the claim relates to a particular food category, a specific food, a proprietary (product-specific) product or a food constituent. All the regulatory approaches for scientific substantiation of claims usually place human intervention trials at the top of a predefined hierarchy of evidence. For example, the EFSA has published a hierarchy of study designs; studies on human subjects are accorded greater weight than animal and in vitro studies, and human intervention studies have greater weight than observational studies^{( 6 )}. However, the research assessments should reflect the effects of foods/food constituents on the health status of human subjects from different sources of evidence. The beneficial outcome(s), measured in clinical, observational, epidemiological studies and, where possible, RCT, should be the improvement in some indicators of health, wellbeing or reduction of risk of disease. The beneficial effects of foods/food constituents can use true endpoints/outcomes, e.g. fatal or non-fatal cardiac event, as well as sufficiently identified, characterised and validated physiological biomarkers, e.g. lowered LDL cholesterol for CHD. Clearly, the relationship between dietary components and health benefits can be demonstrated by a number of different types of studies and designs, and methodological soundness overrides any hierarchy in study type, but also on the quality of its design, execution and analysis^{( 10 , 34 )}. All sources of scientific data have inherent limitations and strengths, and hence a critical review of the totality of the available data and weighing of the evidence should form the basis of the substantiation of a health claim on a case by case basis. In this respect, Codex Alimentarius recognises the complete body of evidence including health claims based not only on RCT but also on observational studies and on authoritative statements prepared or endorsed by a competent authoritative body and which meet the same high scientific standards^{( 20 )}.

Human intervention studies

Well designed RCT provide the most persuasive evidence of efficacy in human subjects and this investigational design permits strong causal inferences. Most other experimental designs lumped together under the term ‘observational studies’ are unable to distinguish whether the observed difference is due to the intervention or to some other unrecognised and often unmeasured factor. However, appropriate study designs and statistical methods can be used to minimise the effects of confounding variables. Scientists are correct in noting that observational studies only provide an association and cannot provide definitive proof of cause and effect^{( 30 )}. However, much of what we already know about human nutrition and health and the knowledge that underpins national and international dietary recommendations is based on epidemiological evidence. Furthermore, the usefulness of the Bradford Hill criteria^{( 38 , 39 )} for systematically evaluating observational studies and for examining evidence of causation from long-term, well-designed cohort studies has recently been supported in a systematic review of the totality of the available evidence for causal links between dietary factors and CHD^{( 40 )}. Given the complex nature of the disease process such as in CHD and the difficulty of characterising the diet, physical activity and other lifestyle factors over a lifetime, a simple hierarchical approach to evidence on causal links cannot rely on RCT^{( 41 )}. Apart from the obvious inability to mask the differences in dietary interventions based on real foods, in practical terms it is equally impossible to secure sufficiently large or sustained differences in lifestyles including diet between intervention groups. Where studies are undertaken, the isolated component is often used at high amounts in high-risk groups. Extrapolation from such intervention studies to the normal healthy population is therefore not straightforward. Hence, it is not surprising to find discrepant results between cohort studies and RCT. The use of relatively short RCT as primary sources of evidence and drug-based standards of proof as requirements to substantiate health claims on foods need to be challenged by nutrition scientists^{( 29 , 30 , 42 – 44 )}.

The success of RCT in evaluating medical treatments and pharmaceuticals does not mean that this method is always the most appropriate for the evaluation of nutritional effects^{( 42 , 43 )}. It is important to compare and contrast the features of foods and food constituents that do not fit the RCT paradigm. Drugs are intended for, and evaluated in, sick people. Food and food constituents with health claims are aimed at the normal healthy population. Drugs typically have only one, or a few, principal endpoints or outcome measures; the effect of a drug is usually measurably large; drug trials require the elimination or minimisation of co-therapies with other agents that might affect the endpoint, and the response to a drug is typically evaluated relative to its absence. In most cases, drugs act quickly and their endpoints can be measured over relatively short periods of time. Few of these features fit the nutrition context. Nutrients and other substances that contribute to nutritional or beneficial physiological effects tend to manifest themselves in small differences over longer periods of time. Nutrients work together rather than in isolation, and often their effects will not develop when intakes of other dietary components are suboptimal. There is, in effect, rarely a nutrient-free state against which the nutrient effects can be compared. Typically, studies compare a low intake with a high intake, but responses will be influenced by threshold characteristics, e.g. Ca absorptive response to vitamin D or Hb response to Fe. The dilemmas of focusing on pharmaceutical approaches to evidence-based nutrition are highlighted by Heaney^{( 43 )}. The reliance on RCT to assess nutrition questions fails to address the limitations of this pharmaceutical approach to nutrition and may explain in part the heterogenicity of results from different research centres and investigators and the different sources of evidence.

Biomarkers and risk factors

One fundamental and challenging approach to the substantiation of health claims is the identification and validation of relevant biomarkers that can predict potential benefits on risks relating to a target function in the body. For function claims, a beneficial effect may relate to maintenance or improvement of a function^{( 6 )}. A risk factor is a factor associated with a risk of a disease that may serve as a predictor of development of that disease. The EFSA stated in its responses to Frequently Asked Questions^{( 5 )}, which were related to the EFSA assessment of health claims applications, that for reduction of a risk factor to be considered beneficial in the context of a reduction of disease risk claim depends on the extent to which it is established that:

1. i. The risk factor is an independent predictor of disease risk (this may be established from intervention and/or observational studies).

2. ii. The relationship of the risk factor to the development of the disease is biologically plausible.

For some risk factors, there is strong evidence that they meet both criteria, e.g. elevated serum LDL cholesterol is a risk factor for CHD. A reduction in systolic blood pressure may be considered beneficial in the context of a reduction of disease-risk claim for CHD and stroke.

Likewise, the USA Guidance for Industry^{( 14 )} states that studies should identify a measurable disease or health-related condition either by measuring incidence associated with mortality or validated surrogate endpoints that predict risk of a specific disease. Whereas the US legislation includes either measurement of incidence of a disease, associated mortality or validated surrogate endpoints that predict the risk of a specific disease, the European legislation requires that the food or one of its constituents significantly reduces a risk factor in the development of a human disease^{( 1 )}. Currently, the EU law pays sole attention to the regulatory requirement for a reduced risk factor in the case of disease risk reduction claims. The absence of a reduced risk factor, as previously noted, is now interpreted by the EC and EFSA as being outside the scope of Article 14 of Regulation (EC) 1924/2006^{( 1 )}, and if such a claim were made with a reference to a human disease, it would take the claim into Article 2(1)(6) of Directive 2000/13/EC as a form of prevention of a disease claim^{( 45 )}. A medicinal claim on normal foods such as the prevention, cure of alleviation of a disease is illegal. All Article 14 claims therefore must have a reduced risk factor. This particular interpretation is challenging from the regulatory and scientific perspectives. The EFSA has also focused only on physiological risk factors, which are surrogate endpoints that have been shown to be valid predictors of disease risk. In contrast, in its definition of reduction of disease-risk claims, Codex states that risk reduction means significantly altering a major risk factor(s) for a disease or health-related condition^{( 20 )}. Codex goes on to state that diseases have multiple risk factors, and altering one of these risk factors may or may not have a beneficial effect. More importantly, the US guidance^{( 14 )} states that risk biomarkers may be used in place of clinical measurement of the incidence of the disease. It points out that it may not be possible to carry out the study for a long enough period to see a statistically meaningful difference in the incidence of disease among study subjects in the treatment and control groups. The US Food and Drug Administration also recognises that accepted surrogate endpoints that are involved in a single pathway may not be applicable to certain substances that are involved in a different pathway.

Examples of surrogate endpoints of disease risk included serum LDL cholesterol, total serum cholesterol concentration and blood pressure for CVD, bone mineral density for osteoporosis, adenomatous polyps for colon/rectal cancer, elevated blood sugar concentrations and insulin resistance for type 2 diabetes, and mild cognitive impairment for dementia.

Key discussion points relate to what the EC/EFSA consider to be a risk factor and whether or not a disease-risk reduction claim can ever be authorised under the current health claim regulation if the evidence is based on a true outcome of a disease from well-designed observational studies, but without a reduced surrogate biomarker from a human intervention study.

FUFOSE attempted to describe markers of exposure and markers of biological response as either factors that are causally related to the endpoint or indicators that are indirectly related. Markers of intermediate endpoint were stated to be more likely to be factors. FUFOSE stated that reduction of disease risk claims would only be justified if the evidence for the effect of a food or food constituent were based on an intermediate endpoint marker of disease. This marker would also have to be shown to be significantly and consistently modulated by the food/constituent for the evidence to be acceptable^{( 8 )}. FUFOSE also set out criteria for markers, which included biochemical, physiological or behavioural markers that should be feasible, valid, reproducible, sensitive and specific. FUFOSE stated, ‘Markers should represent relatively immediate outcomes, which can be used to assess interventions in a reasonable timescale; they could, therefore, wherever possible, replace later and more remote outcomes as have been used in some epidemiological studies’. This particular statement in FUFOSE is a clear indication that if the claimed effect can be measured directly, these measures should take precedence over the use of a surrogate biomarker of the claimed effect, i.e. risk factors are second best and replace a true endpoint or outcome of a disease, from well designed and executed epidemiological studies.

PASSCLAIM set out to develop consensus criteria for the scientific substantiation of claims^{( 9 )}. Criterion 3 shown in Table 2 states, ‘When the true endpoint of a claimed effect cannot be measured directly, studies should use markers’. Criterion 4 states, ‘Markers should be biologically validated in that they have a known relationship to the final outcome and their variability within the target population is known, and methodologically valid with respect to their analytical characteristics’.

PASSCLAIM also noted that, with respect to disease-risk reduction claims, the true disease endpoint often cannot be measured directly for ethical or practical reasons. Therefore, the identification and validation of suitable markers were considered as an important research objective. FUFOSE^{( 8 )} and PASSCLAIM^{( 9 )} recognised that, wherever possible, the claimed benefit that is the true endpoint should be measured directly. However, even though the ideal or target endpoint for human intervention studies may be identified, it may not be measurable in practice. For example, there could be a long time period between the introduction of the intervention and the desired outcome (e.g. a reduced incidence of a disease as evidence of a reduced risk), and it may not be feasible or ethical to access the appropriate target tissues or biochemical processes (e.g. in the vascular wall). Large-scale intervention studies in the otherwise healthy general population for disease-risk reduction claims are, in many cases, excessively demanding of expertise and resources, impractical and are unlikely to reflect the onset and progression of the disease process. FUFOSE and PASSCLAIM state that, when the definitive endpoint cannot be determined, more easily measured markers may be used as proxies or surrogates for the real or desired outcome. For disease risk reduction, the target endpoint, if possible and if accessible, should be measured in some way (e.g. extent of narrowing of the carotid artery as evidence of CVD or bone mineral density as a marker for risk of bone fracture). The more remote markers are from the endpoint, the less specific and more attenuated and subject to confounding variables they become. In a sense, most if not all biomarkers are correlational and are derived from disease states. The existence of an association between a marker and a disease risk does not necessarily mean that changing the variable changes the disease risk. Such modifications can be effective only if the diet–health relationship is causal, if effects already induced are reversible and dependent on the presence of other risk markers that may have stronger effects. Hence, the appropriateness of a marker needs to be considered on a case by case basis. True outcomes and surrogate biomarkers from different types of human studies do, however, contribute to the totality of the evidence.

Hence, the scientific and regulatory issues relate to the following:

1. i. What is a risk factor?

2. ii. Should behavioural and nutritional risk factors such as low or reduced intake of a food/constituent be included?

3. iii. Are only measurable physiological surrogate biomarkers needed?

4. iv. How many risk factors/surrogate biomarkers are considered to be validated?

5. v. What use is evidence from epidemiological/observational studies where true outcomes of disease and statistical evaluation of relative risk can be estimated?

The claimed physiological or nutritional effect needs to be specific enough to be testable and measurable by generally accepted methods. For example, the EFSA now considers ‘gut health’ or ‘digestive health’ to be too general, whereas ‘transit time’ is specific and measurable by generally accepted methods. However, the number of validated surrogate biomarkers is discouragingly low^{( 46 , 47 )}, and in many health relationships it is possible only to describe or refer to the role of a nutrient or other substance in growth, development and the functions of the body rather than measure specific nutritional effects^{( 30 , 48 )}.

Homoeostasis and normal physiological adaptation

In general, diet-related diseases are caused by chronic exposure to unbalanced diets and not by acute exposures. The body's physiology may cope with variations in diet through feedback mechanisms, the buffering capacity of homeostasis and, if necessary, repair mechanisms. Adaptation to habitual consumption of such a diet or diets with unbalanced composition modulates the acute response and produces less dramatic alterations in molecular and physiological processes^{( 49 )}.

Adaptive responses attempt to keep physiology within an individual's ‘normal’ range. While physiological adaptation will result in maintenance of functional biomarkers within the healthy range, these biomarkers may show a very different response in one healthy individual v. another, more susceptible individual. Nutrition has a function in maintaining homoeostasis in metabolic processes such as oxidative stress and inflammation. Because of the large variation in ‘normality’, the effects of nutritional interventions may remain hidden because of the dynamic and multifactorial nature of the homoeostatic processes^{( 29 , 44 , 49 )}.

Ideally, biomarkers of health should quantify the subtle but relevant effects in the healthy general population that precede the onset of disease, to identify predispositions or predict our capacity to deal with dietary and age-related stresses. Most validated biomarkers currently used in nutrition intervention studies are associated with the diagnostic and prognostic use for chronic disease and since most complex diseases are of late onset, biomarkers are typically associated with surrogate endpoints. Such endpoints would be equivalent to the clinical endpoint. The use of patients with a particular disease or condition is common in nutrition science, and of key importance is to weigh up how representative the clinical studies are for the general population. Health is a continuous process involving multiple organs, metabolic pathways and genes all interacting to maintain homoeostasis^{( 44 )}. Clearly, studies in patients are valuable sources of evidence. However, scientific conclusions on the relevance of patient studies to the normal healthy population need a coherent and transparent approach on a case by case basis. Research is needed to identify physiological responses of adaptation that will expand our knowledge of how health is maintained and optimised, and when homoeostasis is disturbed, leading to the onset of disease^{( 29 )}.

A scientific framework for weighing the evidence

It is necessary to have a transparent framework for commenting on the nature and quality of the totality of the data and for weighing the evidence in order to allow regulators to judge and make risk management decisions about the acceptability and veracity of a health claim submitted by an applicant^{( 34 )}. The development of a scientific framework for weighing the totality of the available data and the determination of the extent to which a cause and effect relationship is demonstrated are both scientifically justified and valid, and they can help to identify gaps in research.

Organisations such as WHO and World Cancer Research Fund have already used various systems to assess the level of evidence from different types of studies^{( 31 , 32 )}, and in the EFSA Scientific Opinion on establishing food-based dietary guidelines, the identification of diet–health relationships was described using the same terminology, namely convincing evidence, probable evidence, possible evidence and insufficient evidence^{( 50 )}. Likewise, the EFSA consultation paper on guidance on human health risk–benefit assessment of foods defines ‘benefit’ as the probability of a positive health effects and/or the probability of a reduction of an adverse health effect^{( 51 )}.

Clearly, the concepts developed by PASSCLAIM, WHO, the World Cancer Research Fund and EFSA could be used further to underpin the regulatory approaches to assessing the totality of the available data and, in particular, the weight of the evidence. Other researchers such as Mente et al. have also proposed scientific criteria to assess the (i) strength, (ii) consistency, (iii) temporality and (iv) coherence of evidence from cohort studies on diet and health relationships, and have examined the consistency of these findings with results from randomised trials^{( 40 )}.

Mente et al. applied the Bradford Hill guidelines^{( 38 )} for causality and a modified set of criteria for assessing the associations between diet and CHD. A causation score was based on whether the four criteria were met or not. If all four criteria were met, a score of four was assigned (i.e. strong evidence), whereas if only one or two criteria were met, a score of one or two (i.e. weak evidence), respectively, was assigned. The evidence for each food/health relationship was then tabulated to show whether it was judged to be ‘strong’, ‘moderate’ or ‘weak’. Although this judgemental classification could be criticised for being arbitrary, the framework illustrates that it is possible to assess the extent of the evidence of causation and to compare the consistency of relative risk from well-designed epidemiological cohort studies with outcomes from RCT. The findings of Mente et al. ^{( 40 )} support the strategy of investigating dietary patterns in cohort studies and RCT, especially for common and complex chronic diseases such as CHD.

Grading the strength of the body of evidence is an accepted scientific practice that allows the assessment of the quality, consistency and quantity of evidence, and the use of such grading is needed for elaborating not only dietary guidelines but also a regulatory framework for the use of health claims. Rarely are there cases where there is only a single piece of evidence for a causal claim. When assessing whether an association is causal, it is necessary to consider all the relevant studies: this is the powerful idea underlying the importance of systematic reviews^{( 39 )}. Howick et al. ^{( 39 )} discuss the evolution of evidence hierarchies and conclude that the Bradford Hill guidelines for causation form a useful tool for the evaluation of the strength of the evidence that mechanistic evidence can provide evidential support for a causal hypothesis, and that current hierarchies of evidence need to be revised.

In evidence-based nutrition there is therefore a need to examine critically and scientifically the current hierarchies of evidence and evidence-based grading systems that can be applied in the area of human nutrition. Assessing the strength, consistency and biological plausibility of the evidence is a prerequisite to the determination of the strength of recommendations for regulatory use by risk managers, and for the wording of the claim to reflect the extent to which, or probability that, a particular food/constituent health benefit is true and will likely be refined (not reversed) by subsequent scientific research^{( 34 )}.

It is now critical from both a European and a global perspective to identify a suitable scientific framework for the weighing of evidence in order to embrace the ‘state-of-the-art’ nutrition science, to stimulate future academic research, to promote product innovation and to communicate accurate and truthful nutrition and health messages to the public.

Consumer understanding and the use of wording to reflect the strength and consistency of the evidence to support health claims

Nutrition and health claims are potentially powerful tools in communication to consumers, and when used correctly, they present one way to improve consumers’ nutritional knowledge and healthy eating patterns, as well as contributing to public health more generally^{( 35 , 52 – 54 )}. Not surprisingly, consumer understanding as well as scientific substantiation are cornerstones of European legislation on these claims^{( 1 )}. This area of consumer research is ripe for development, and a key question is: can a scientific judgement on the strength, consistency and biological plausibility of the totality of the available evidence be communicated to the consumer using claim wording, disclaimers or graphics that are truthful and meaningful to consumers? Clearly, the health benefits must not go beyond the scope of the evidence or confuse or mislead the consumer^{( 55 )}.

In the USA, the Food and Drug Administration authorises health claims (which equate to the European reduction of disease risk claims) on the basis of significant scientific agreement (SSA), or based on the totality of publicly available scientific evidence (including evidence from well-designed studies conducted in a manner which is consistent with generally recognised scientific procedures and principles), that there is SSA among experts qualified by scientific training and experience to evaluate such claims, (and) that the claim is supported by such evidence. Legal action challenged the SSA standard and resulted in an authorisation process for the so-called ‘qualified health claims’, where claims are based on evidence for which the strength and consistency of supporting data are lower than for SSA^{( 56 )}. Subsequent studies indicated that consumers have difficulty understanding the different types of health claims and that using the US system, they found it difficult to distinguish between the four levels of claims proposed by the Food and Drug Administration, categorised as SSA, B, C and D to reflect the scientific evidence that was graded as high, moderate/good, low and lowest, respectively^{( 57 )}. Research in The Netherlands^{( 52 )}, Australia^{( 58 )} and the USA^{( 59 )} has also provided evidence that claims with differing levels of scientific support may not adequately protect public health. However, research observations in the USA relate essentially to a flawed system where consumers had difficulty distinguishing between the four differing evidentiary levels for claims, especially the language-only formats and the use of A, B, C and D grades as opposed to graphic representations^{( 60 )}. The words ‘promising’, ‘inconclusive’ and ‘may’ were also perceived to mean different things to different consumers.

As a result, in Europe there has been some opposition to the use of qualified health claims^{( 61 )}, and this position has been allowed to remain unchallenged. In the PASSCLAIM consensus criteria^{( 9 )}, the text states, ‘The context within which a claim and the case made in its support should be assessed involves considering existing legislation and dietary guidelines; the need for review in the light of evolving science; and the comprehensibility of the claim to consumers. These aspects are not thought to be part of the scientific criteria reviewed by PASSCLAIM. They nevertheless provide a background against which the scientific validity of claims should be justified’. Owing to lack of knowledge, misinterpretation and over-generalisation of nutrition and health claims by consumers in the USA and the limited research on consumer understanding in Europe and other parts of the world, the concept of using qualified health claims has been treated cautiously by some regulatory bodies. In the PASSCLAIM report of the Second Plenary Meeting^{( 61 )}, there was support for the idea that a structured approach to characterising the quality of the data would enable assessors to weigh the evidence, but caution was expressed by participants that this should not lead to a weighted characterisation of the claim itself. Participants also expressed opposition to the idea of ‘qualified claims’ on the grounds that a claim should either be judged as substantiated or not. This ‘yes’ v. ‘no’ approach to the assessment of outcomes for the scientific substantiation of health claims in Europe has been influenced by the low comprehensibility of claims to consumers. These aspects of consumer understanding were not part of the PASSCLAIM initiatives and are considered to be outside the scope of the scientific assessors of health claims. The use of a proper scientific framework for assessing the strength and consistency of the evidence and the appropriate use of graphics or phrases such as ‘supported by strong evidence’, ‘moderate evidence’ or ‘weak evidence’ could be an appropriate way of communicating the level of evidence to consumers and to fulfil the principle of proportionality in European law. The use of the concept of ‘qualified health claims’ is not prohibited in the regulation on health claims, and appropriate wording or graphics could reflect the extent to which a cause and effect relationship has been demonstrated, which is exactly what the law states.

Conclusion

The totality of the available data from all the different sources needs to be considered in the weighing of the evidence. It is not disputed that outcomes based on RCT offer the strongest support for cause and effect relationships. However, RCT are not always feasible or available (now or in the future), and for nutritional studies they are costly and difficult experimentally. The assessment of the totality of the evidence needs greater recognition of the strengths and limitations of the different experimental designs. To date, the reliance on RCT for demonstration of a cause and effect relationship, the focus on pharmaceutical approaches using isolated components, the fact that epidemiological studies do not prove cause and effect, and the lack of validated biomarkers could all conspire to stifle nutrition research. The PASSCLAIM criteria are also being applied in such a way that all the criteria must be achieved, whereas the intention of PASSCLAIM was for the totality of the available data to be compared transparently and scientifically against them. The evidence in most areas of nutrition science do not achieve the PASSCLAIM criteria, and if the WHO/World Cancer Research Fund criteria for grading the evidence are considered, most state-of-the-art nutrition science falls in the probable/possible categories rather than in the convincing or even stronger, conclusive categories.

Clearly, there should be no compromise on the quality of evidence and standards of scientific assessment. PASSCLAIM and the legislation placed no emphasis on the prioritisation of evidence and studies. It argued for a coherent case for causal associations based on intelligent use of the science base. The notion that RCT are needed to establish causality needs to be challenged by nutrition scientists. The demonstrations of causal associations (or the extent to which a causal relationship is established) can be scientifically justified from both single and collective studies that enable a scientific appraisal of the causality of any claimed relationship.

The scientific assessments should reflect the strength of the evidence to allow researchers and innovative manufacturers supporting research to know where they stand on the continuum of research investigation. The determination of the strength of the evidence should draw on best practice around the world^{( 31 , 32 , 39 , 40 )} to describe the evidence as convincing, probable, possible or insufficient or as strong, moderate or weak. Further research is also needed to develop a transparent process for creating well-defined consensus standards and guidelines for the development of biomarkers, their validation and qualification^{( 56 )}.

Much of the available nutrition scientific data are derived from the state-of-the-art published literature, where the onus has been, and still is, on the peer-review system to ensure that an appropriate standard of rigour is applied in the assessment of the quality of the studies. There is ample nutrition science available, but it was not necessarily designed to fit the purpose of substantiating health claims and providing conclusive evidence of cause and effect^{( 30 )}. A major issue relates to the appropriateness of a drug-based standard of proof. The field of nutrition needs to reflect on alternative experimental designs and endpoints. Nutrition policy decisions need to be based on the totality of the available data and on evidence that may be less persuasive than that provided by an RCT. In the future, heightened scrutiny of the design, execution and interpretation of the data from human studies before publication will improve the overall credibility within the nutrition area, particularly in the context of the potential use of these data to establish and sustain food-related health claims^{( 62 )}. Likewise, researchers exploring the benefits of particular food categories, foods, food ingredients and food constituents will have to pay greater attention to the relevant legislation, to available guidance documents and to the scientific quality of their data, both proprietary and in the public domain.

Health claims should assist consumers to make informed choices and help consumers identify particular foods and food constituents as well as encouraging greater consumption of such foods as part of a balanced diet. From an industry perspective, claims are used to identify, market and promote products. The challenge is to translate accurately the scientific wording of the nutritional benefit into consumer language^{( 35 , 55 )}. The wording of a claim involves a careful balance between keeping it simple and understandable v. keeping it serious and scientific. In Europe, a key issue relates to the move from the current use of more generalised claims to more specific, substantiated claimed effects describing discrete physiological functions. The initial research on consumer understanding of health claims reinforces the need to develop new, and refine existing, methods of consumer research, and to conduct academic and market research on the intended consumers in order to ascertain the extent to which the claims are understood in the context of the total diet, and to establish whether consumers can understand the relative strength of the evidence that exists to support the claim^{( 35 )}.

It is essential that the regulation and its interpretation are fit for purpose. What is required is clear guidance to protect the consumer and to guard against dishonest food labelling. The use of words to reflect the strength and consistency of the evidence rather than a ‘yes’ or ‘no’ would be a positive development in that they would assign some intellectual credit to both consumers and to the food industry^{( 28 )}.

If conclusive proof is required, many experts in nutrition are saying that this level of evidence may not be achievable in terms of costs and resources within reasonable timescales, and that very few health claims (other than the well-established nutrient function claims) will ever be approved. Where epidemiological evidence is strong and the human intervention study is not as good, a different approach to the hierarchy of evidence is required, particularly for disease-risk reduction claims. Emerging and possibly superior technologies, including markers of homoeostatic adaptability arising from nutrigenomics, proteomics and metabolomics may provide the intermediate markers envisaged by FUFOSE and PASSCLAIM. In the meantime, the totality of the available data, the weighing of the evidence using a framework to assess the strength and consistency of the evidence, and the communication of the probabilities of benefit should be considered urgently^{( 44 )}.

Although the EFSA believes that it is possible for companies, particularly small/medium-sized enterprises, to use the authorised list of Article 13.1 generic claims (mostly nutrient function claims), the reality is that most companies do not see this as providing them with a competitive advantage. If the only routes to product differentiation are through the submission of full Articles 13.5 and 14 dossiers that require conclusive proof of the claimed effect, many companies will look for a return on their commercial investments in other directions. A lack of motivation for the food industry to use health claims in Europe will ultimately affect research funding, which could have adverse long-term consequences both for nutrition science and for consumers, who would not benefit from new science that could have a positive impact on health^{( 28 )}.

From the consumer protection perspective, there are already concerns: firstly about the increasing use of food fortification with vitamins and minerals that highlights the EFSA-approved benefits to health of micronutrients, and secondly, about the quality of advice about diet and nutrition in the media outside the scope of the regulations. Consumers are often confused by what they see and hear in the media. Scientists interpret emerging science whereas journalists deliver the information, and consumers find it difficult to distinguish between media trivia and information that actually warrants behavioural change. Misreporting of dietary advice by UK newspapers is already widespread and may contribute to public misconceptions about food and health^{( 63 )}.

In conclusion, the clear aims of the European and, indeed, global legislation on the scientific substantiation of health claims are to achieve a high degree of consumer protection, to ensure confidence in claims on foods and food supplements, and to promote and protect innovation.

Developing the scientific criteria and the legislation for the substantiation of health claims have already involved extensive collaboration and discussion among the different stakeholders, including scientists from academia and research institutes, industry, consumer organisations and regulatory bodies. Further work is necessary to elaborate a robust and pragmatic scientific framework for weighing the totality of available data and for expressing the strength, consistency and biological plausibility of the evidence. To provide conclusive proof of a diet and health relationship represents an aspirational scientific standard that most health claims and indeed, most nutritional public health recommendations and dietary goals cannot achieve. To weigh the evidence involves value judgements, but it needs a credible scientific structure that captures the existing knowledge in such a way that policy makers can draw conclusions about the probability that a change in the dietary intake of a food category, a food or a food constituent will result in a health benefit.

European law and other approaches to scientific substantiation of health claims around the world specifically state that it is the extent to which cause and effect of a diet–health relationship should be demonstrated to reflect the available evidence and scientific consensus among experts in a particular field. The creation of a harmonised, scientifically robust, transparent and proportionate framework for the assessment of nutrition and health claims is a critical regulatory and policy issue. Such a framework, when administered soundly, should give a high level of consumer protection and legal certainty for companies and research organisations as well as protect the legitimate expectations of applicants. These scientific and policy issues need to be addressed urgently and advanced through academia, member state representatives, the EC and the European Parliament.