A commentary on current practice in mediating variable analyses in behavioural nutrition and physical activity

Abstract

Objective

To critique current practice in, and provide recommendations for, mediating variable analyses (MVA) of nutrition and physical activity behaviour change.

Strategy

Theory-based behavioural nutrition and physical activity interventions aim at changing mediating variables that are hypothesized to be responsible for changes in the outcome of interest. MVA are useful because they help to identify the most promising theoretical approaches, mediators and intervention components for behaviour change. However, the current literature suggests that MVA are often inappropriately conducted, poorly understood and inadequately presented. Main problems encountered in the published literature are explained and suggestions for overcoming weaknesses of current practice are proposed.

Conclusion

The use of the most appropriate, currently available methods of MVA, and a correct, comprehensive presentation and interpretation of their findings, is of paramount importance for understanding how obesity can be treated and prevented.

There appears to be no better time to evaluate our current practices in analyses of mediators of dietary and physical activity behaviour than now. Following the modest performance of obesity prevention programmes⁽¹⁾ and in recognition of the importance of identifying the most effective mechanisms and strategies for change⁽²⁾, the amount of research output on potential mediators of dietary and physical activity behaviour is increasing rapidly. The promise of analysis of mediating variables is the objective testing of theoretical processes in diet and physical activity prevention programmes. If these critical mediating processes can be identified, programmes may be improved by identifying important components and the cost of the programmes may be reduced by removing ineffective components. Ideally, true theoretical mediating mechanisms will emerge across studies and across behavioural interventions. Although mediation analysis is promising, before potentially going too far in delivering less-than-optimal communications, we need to evaluate the informational value and soundness of our current analytical practices and their interpretation. The aim of the current paper is to present general strategies that will help enhance the current quality of reports on mechanisms of obesity-related behaviour change. The paper starts with a brief overview of Baron and Kenny’s⁽³⁾ method of mediating variables analysis (MVA), then summarizes recent criticisms of their approach, and finally proposes ways to improve the presentation and interpretation of results from MVA in obesity-related behaviour research.

Current practice: using the Baron and Kenny method

Baron and Kenny’s⁽³⁾ seminal paper on mediators and moderators is by far the most cited published piece of work within studies investigating mediators of nutrition and physical activity behaviour. According to their not always correctly interpreted conceptual framework, to function as a mediator, a variable must meet four criteria.

1. 1. An independent variable (e.g. fat-intake intervention) must be significantly associated with the potential mediator (e.g. change in self-efficacy for a low-fat diet); this relationship is operationalized as the α regression coefficient (also termed ‘path coefficient’ in the language of structural equation modelling; see Fig. 1).

   

   Fig. 1 Diagram for analysis of the effect of a fat-intake intervention on changes in fat intake mediated by self-efficacy for a low-fat diet

2. 2. The potential mediator (self-efficacy) must be significantly associated with the outcome (e.g. change in fat intake), after adjustment for the independent variable; this is represented by the β coefficient (see Fig. 1).

3. 3. After adjustment for the potential mediator (change in self-efficacy), a previously significant relationship between the outcome (change in fat intake) and the independent variable (fat-intake intervention) is no longer significant or is significantly attenuated; this is represented by the τ′ coefficient (see Fig. 1).

4. 4. The third condition implies a significant relationship between the outcome (change in fat intake) and the independent variable (fat-intake intervention), which constitutes the fourth criterion; this relationship is represented by the τ coefficient (see Fig. 1). All of these coefficients are best obtained using regression models or structural equation modelling.

In the domain of behavioural prevention programmes, an evaluation of the first relationship (α coefficient) is termed ‘action theory test’ and informs us on the extent to which we were successful in manipulating the hypothesized mechanisms of behaviour change. The second relationship (β coefficient) represents a ‘conceptual theory test’, i.e. the extent to which there is empirical support for our conceptual theory (e.g. Social Cognitive Theory) that changes in the hypothesized mechanisms cause changes in behaviour⁽⁴⁾. The italicized text above highlights important aspects of MVA that are sometimes overlooked by researchers claiming to follow Baron and Kenny’s method.

Misinterpreting Baron and Kenny

It is not uncommon to see studies published in very respectable journals reporting a significant association between the mediator and the outcome, unadjusted for the effects of the independent variable, as one of the criteria for mediation^(5–8). The source of confusion appears to be Baron and Kenny’s unfortunate omission of the words ‘controlling or adjusting for the independent variable’ in the first part of their well-cited article (p. 1176), while later (p. 1177) they clearly explain that the second criterion of mediation (see above) needs to be tested by ‘regressing the dependent variable on both the independent variable and on the mediator’⁽³⁾. The reason for this necessary adjustment is that the mediator and the outcome may be related because they are both caused by the independent variable (in which case, there is no mediation) and not because the independent variable influences the mediator, which then influences the outcome⁽⁹⁾. Consequently, our first take-home message is that to test mediation according to the Baron and Kenny method it is unnecessary to examine the unadjusted relationship between the mediator and the outcome. However, it is necessary to establish that the mediator is related to the outcome independently of the independent variable.

Beyond the four criteria of mediation: in search for a measure of mediated effect

Although a formal statistical test of significance of the mediated effect (sometimes called ‘indirect effect’) is not one of the explicit criteria of mediation proposed by Baron and Kenny, they nevertheless note that a significantly attenuated independent variable–outcome relationship, following adjustment for the potential mediator, is a necessary condition for mediation. The magnitude of the attenuation is a measure of the mediated effect. The question is: what do we mean by significant mediated effect (or attenuation) and why is this information important? We argue that truly ‘significant’ mediators are those that satisfy both requirements of statistical and substantive or clinical significance. While statistical significance ensures that we identify reliable mediators, substantive or clinical significance ensures that we focus our efforts on finding and manipulating mechanisms that produce the largest effects in terms of behaviour or health outcomes, thus maximizing the cost-effectiveness of our interventions. Here is where the vast majority of published work runs into problems.

Let us start with statistical significance. There are many ways in which we can test the statistical significance of a mediated effect^{(10, 11)}. In general, if a variable meets all four criteria of mediation as per Baron and Kenny, we can confidently conclude that it is a reliable mediator. In fact, simulation studies have shown Baron and Kenny’s method to be one the most conservative tests of mediation^{(11, 12)}. However, as explained later in more detail, the downside of this overly conservative approach is that, in some circumstances, it can fail to identify reliable and substantively meaningful mediators⁽¹³⁾. On the other hand, large-scale studies may produce statistically significant but substantively or clinically trivial mediating effects. This is why it is important for studies adopting this particular MVA framework to go beyond the four criteria of mediation and explicitly report the observed mediated effect for each examined mediator. This needs to be done using simple, single-mediator as well as complex, multiple-mediator models evaluating the independent contribution of each mediating variable. Importantly, such practice is essential if solid knowledge about mechanism and strategies of behaviour change is to be swiftly accumulated and synthesized with the aid of meta-analytical procedures. Any alternative course of action is bound to unnecessarily slow down the progress in this area of research.

A measure of a mediated effect can be obtained by computing the product of α coefficient (independent-variable effect on the mediator) and β coefficient (mediator effect on the outcome adjusted for the independent variable; see Fig. 1). This is termed ‘product-of-coefficient estimate’ of a mediated effect^(10–12). When using ordinary least squares regression in single-mediator models, it is algebraically equivalent to computing the reduction in the independent-variable effect after inclusion of the mediator in the regression model^(10–12). This is named a ‘difference-of-coefficients estimate’. These estimates of mediated effect represent measures of the effect of the independent variable on the outcome via the mediator (the proposed mechanism) in the units used to measure the outcome (which implies that unstandardized regression coefficients have been used to compute them); hence their appeal and informative value.

For instance, if an intervention produced a decrease in fat intake of 10 g/d, and after including self-efficacy for a low-fat diet in the model this effect is reduced to 4 g/d, we would state that the effect of the intervention mediated by self-efficacy was a decrease in fat intake of 6 g/d. In other words, the intervention yielded a reduction in fat intake of 6 g/d likely because of its effect on self-efficacy. In terms of product-of-coefficient estimate, if the intervention has an average effect on self-efficacy of 2 units on a 5-point scale (α coefficient) and the independent effect of self-efficacy is a decrease in fat of 3 g/d per unit increase in self-efficacy (β coefficient; conceptual theory test), by multiplying these estimates we obtain a mediated effect of the intervention through self-efficacy of 2 × (−3) g/d=−6 g/d (i.e. a decrease in fat intake of 6 g/d). These values inform us of the extent to which self-efficacy is important in changing the target behaviour (3 g/d per unit change in self-efficacy; conceptual theory test), our ability to change self-efficacy (a 2 unit change on a 5 point-scale; action theory test), and the expected benefit of targeting self-efficacy with a specific type of intervention within a specific target population (a decrease in fat intake of 6 g/d). Hence, the second take-home message of the present paper, especially relevant to intervention studies, is that it is important to report estimates of mediated effects as well as action and conceptual theory tests (see Haerens et al.⁽¹⁴⁾ for an example).

Beyond the four criteria of mediation: a special note on the ‘dichotomous’ case

Dichotomous outcome variables, such as being overweight v. being of normal weight or meeting v. not meeting the physical activity recommendations for general health, are often used in behavioural nutrition and physical activity research. In such cases, assuming that the mediator is a continuous variable, a mediating effect can be computed using a product-of-coefficient test based on the estimate of α from ordinary least squares regression and β from logistic or probit regression^{(10, 11)}. This method has been found to be robust against departures from the logistic or probit assumptions as well as normality assumptions for the distribution of the mediator⁽¹⁵⁾. Importantly, for dichotomous outcomes, the difference-of-coefficient method yields incorrect results and, hence, its use is not recommended^{(11, 15)}. The same applies to the commonly used approach to MVA based on an examination of the presence of a reduction in the regression coefficient of the independent variable (e.g. intervention) after inclusion of the mediator in the logistic regression model (criterion 3 of the Baron–Kenny method). Specifically, when using these methods, larger actual mediated effects may paradoxically result in smaller or nil estimates of the mediated effect or a reduction in coefficients. This effect is due to the difference in scales of the logistic regression coefficients of the outcome on the independent variable adjusting and not adjusting for the mediator, since these coefficients are derived from separate logistic regression equations (with a fixed error variance). Hence, the potential negative consequences of using Baron and Kenny’s method to conduct MVA with a dichotomous outcome are particularly serious. The third take-home message of the paper is to use the product-of-coefficient method when dealing with dichotomous outcome variables.

Beyond the four criteria of mediation: confidence intervals

Any statistical estimate of effect is accompanied by a measure of its accuracy (i.e. a standard error), which allows for the computation of the range of plausible values (i.e. the confidence interval) of the population parameter (i.e. mediated effect) supported by the data. Confidence intervals tell us how confident we can be that the observed estimate of effect corresponds to the effect in the target population. They provide clearer information than the usual P values because the latter are a confounded mixture of effect size and sample size, while the width of the former is determined by sample size and variability in effect of interest (determining accuracy). This also applies to the estimate of the mediated effect. Using the fat-intake example, a 95 % confidence interval of fat intake ranging from −1 to −11 g/d would indicate that the plausible population effects of the intervention on fat intake through self-efficacy are a decrease of fat intake from 1 to 11 g/d.

The information given by the point estimate and confidence limits of the mediated effect is very useful in guiding efforts to enhance theory and practice of behaviour change. For instance, a clinically meaningful mediated effect accompanied by large confidence limits indicates four main possible sources of problems: (i) insufficient sample size; (ii) large measurement error; (iii) collinearity between the independent variable and the mediator affecting the accuracy of the β coefficient estimate⁽³⁾; (iv) large inter-individual variations in the effects of the intervention on the mediator or/and the effects of the mediator on the outcome. In contrast, a clinically trivial but accurate (with a narrow confidence interval) estimate of mediated effect points directly at problems with action (the effectiveness of our intervention in changing the mediator) and conceptual theory (theoretical mechanisms of change). Thus our fourth take-home message is: report confidence intervals of mediated effects and interpret them conjointly with their point estimates.

How do we obtain confidence limits of mediated effects? Baron and Kenny⁽³⁾ provided a formula for the standard error of the mediated effect based on Sobel⁽¹⁶⁾. The use of the formula assumes that the mediated effect is approximately normally distributed. This solution is less than optimal because, unless the ratio of the α and/or β coefficients to their standard errors is equal or greater than 6, mediated effects are not normally distributed^{(12, 17–19)}. However, it is very easy to implement by those less versed in more sophisticated methods of MVA. All it requires are the point estimates and standard errors of α and β coefficients (see Preacher’s (2003) online program at http://www.psych.ku.edu/preacher/sobel/sobel.htm and associated ‘warnings’).

More optimal and recommended solutions involve the computation of asymmetric confidence intervals of mediated effects using a recent program developed by MacKinnon et al.⁽²⁰⁾ (which can be found at http://www.public.asu.edu/~davidpm/ripl/Prodclin) and the use of bootstrapping resampling techniques^{(17, 19, 21)}. Bootstrapping allows the distribution of the mediated effect to be estimated empirically by using information from the original sample (treated as a pseudo population)⁽²²⁾. It is particularly recommended for non-normally distributed variables, such as mediated effects, for which parametric inferential statistics may produce biased estimates. This method consists in drawing with replacement a large number of bootstrap samples (e.g. 1000) from the original sample; estimating mediated effects for each of these samples; averaging the effect estimates across all of the bootstrap samples; and computing the 95 % percentile confidence intervals for the mediated effects across all of the bootstrap samples⁽²²⁾. Mediators whose confidence limits do not include zero are considered reliable (statistically significant). It is also recommended that the more accurate bias-corrected bootstrap confidence intervals, corrected for the difference between the original estimate and the bootstrap mean estimate, be computed⁽²³⁾ (see Cerin and Leslie⁽²⁴⁾ for examples in the field of physical activity).

Is it really necessary to test the association between the independent variable and the outcome in mediating variable analysis?

The ultimate aim of MVA is to establish whether an independent variable influences the outcome through its effect on a mediating variable. If this is the case, do we really need to show that the independent variable is related to the outcome (fourth criterion of mediation as per Baron and Kenny)? The answer is no. We might be legitimately interested in the overall association of, for instance, socio-economic status and physical activity behaviour. However, a significant association between the two is not a requirement for mediation to occur. Actually, this particular criterion can hinder the discovery of substantive mechanisms of influence.

There are three major reasons for this. The first reason, discussed by Judd and Kenny⁽²⁵⁾ and more recently explained by Kraemer et al.⁽²⁶⁾, relates to the fact that the effect of the mediator on the outcome may depend on the values of the independent variable (interaction effect). It is theoretically possible for an intervention to leave unaltered the average level of the outcome (τ coefficient) and mediator (β coefficient) but change the direction of the effect of the mediator on the outcome (see Fig. 2a; β _C and β _T coefficients). In such a case, mediation would occur without observing a significant intervention effect on the outcome. As a matter of fact, there might not even be any significant intervention effects on the mediator or overall independent effect of the mediator on the outcome (Fig. 2a).

Fig. 2 Examples of mediation in the absence of a significant association between the independent variable and the outcome: (a) interaction between an intervention and the mediator; (b) inconsistent mediation

The second reason regards the possibility that the effect of an independent variable on the outcome be explained by multiple competing mechanisms whose influences are of opposite direction and cancel out^{(19, 27)}. For instance, it has been shown that an intervention-induced increase in physical activity (mediator of intervention effect, if the aim is to reduce BMI by increasing energy expenditure) in normal-weight adults does not generally result in a decrease in BMI (outcome) because it is also accompanied by an increase in energy intake (counterproductive or inconsistent mediator of intervention effect)⁽²⁸⁾. The intervention triggers two different mechanisms exerting opposite effects on BMI. It is clear that here mediation occurs even in lack of a significant overall effect of the intervention on the outcome (see Fig. 2b).

Inconsistent mediation, also called suppression, can sometimes be counterintuitive (see MacKinnon et al.⁽²⁷⁾ for a discussion on inconsistent mediators). In such cases, assuming a robust effect, it is a warning that there may be serious problems with: (i) our action and conceptual theories; or (ii) our mediating variables measures. For example, unexpected inconsistent mediation effects have been recently reported in physical activity and nutrition intervention studies^{(14, 29)}. Self-efficacy and perceived barriers were found to be independently associated, in the theoretically predicted direction, with changes in physical activity and fat intake, respectively. However, the intervention was found to have a negative effect on self-efficacy and perceived barriers, resulting in inconsistent mediation effects. In other words, the intervention would have been more successful had not it exerted a negative effect on self-efficacy and perceived barriers as mechanisms of behaviour change. Changes in self-efficacy and perceived barriers suppressed the intervention effectiveness. The authors concluded that their strategies to manipulate mediators of behaviour change in adolescents needed to be reconsidered (problems with action theory) and/or that an intervention by measurement effect might have occurred, whereby exposure to the intervention affected the interpretation of the items in relation to the underlying construct (change of location of the items on the underlying construct). These explanations can be verified by employing differential item functioning (DIF) techniques in combination with having objective behavioural markers of self-efficacy and perceived barriers to contrast with the DIF of the scale items⁽³⁰⁾.

The third reason for not considering a significant relationship between an independent variable and the outcome as a necessary criterion of mediation is low statistical power^{(12, 18, 19)}. This argument is particularly relevant if we believe that the effect of the independent variable on the outcome is temporally or causally distal⁽¹⁹⁾. In their simulation studies, Fritz and MacKinnon⁽¹⁸⁾ have shown that, under conditions of complete mediation and small effect size, we would need a sample size of approximately 21 000 cases to achieve acceptable levels of statistical power. In contrast, the sample size required to detect the same mediated effect using product-of-coefficient tests (e.g. Sobel test, bootstrap estimates and asymmetric confidence intervals) would range between 490 and 660 cases.

Conclusion

To recap, the main points to remember in improving our current practices in MVA are:

1. 1. Examine potential mediating effects even in the absence of a significant association between the independent and outcome variables.

2. 2. Examine whether there is an interaction effect of the independent variable and the mediator on the outcome (applicable to prospective studies).

3. 3. Report point estimates of the mediated effects, preferably in the units of the outcome measure.

4. 4. Report confidence intervals of the mediated effects, preferably using accurate computational methods (asymmetric confidence intervals, bootstrapping).

5. 5. When examining multiple mediators, aim at evaluating the independent contribution of each of them using complex, multiple-mediator models.

6. 6. In intervention studies, test action (the extent to which the intervention affected the mediator) and conceptual theories (the extent to which changes in the mediator likely caused changes in the outcome).

7. 7. Do not use Baron and Kenny’s method and the difference-of-coefficients method of MVA with dichotomous outcome variables.

Technical, methodological and conceptual issues related to MVA are complex and cannot be satisfactorily discussed in a single paper. Our aim was to highlight a few fundamental issues and strategies related to current practice in MVA in the research fields of behavioural nutrition and physical activity that is hoped will help us improve our understanding of how we can combat the obesity problem in our societies.