Evidence of Psychological Targeting but not Psychological Tailoring in Political Persuasion Around Brexit

Abstract

There are numerous associations between psychological characteristics and political values, but it is unclear whether messages tailored to these psychological characteristics can influence political decisions. Two studies (N = 398, N = 395) tested whether psychological-based argument tailoring could influence participants’ decision-making. We constructed arguments based on the 2016 Brexit referendum; Remain supporters were presented with four arguments supporting the Leave campaign, tailored to reflect the participant’s strongest (/weakest) moral foundation (Loyalty or Fairness) or personality trait (Conscientiousness or Openness). We tested whether individuals scoring high on a trait would find the tailored arguments more persuasive than individuals scoring low on the same trait. We found clear evidence for targeting, particularly for Loyalty, but either no evidence or weak evidence, in the case of Conscientiousness, for tailoring. Overall, the results suggest that targeting political messages could be effective, but provide either no, or weak evidence that tailoring these messages influences political decision-making.

Introduction

Adverts targeted and tailored to psychological traits have been shown to be effective in non-political domains (Matz et al., 2017). Given the alleged roles of Cambridge Analytica and AggregateIQ in recent political campaigns, the question of whether psychologically tailored and targeted arguments are effective in political contexts has important social implications ¹.

Research has demonstrated that personal and moral values reliably correlate with political decisions in the US and UK (Gerber et al., 2010; Graham et al., 2009; Sibley et al., 2012) and so targeting and tailoring arguments in line with these values could be effective. In addition to the research showing the effectiveness of personality-based tailoring in advertising (Hirsh et al., 2012; Matz et al., 2017), there is some evidence that Moral Foundations Theory (MFT) can be effective in tailoring political arguments (Feinberg & Willer, 2013; 2015; Voelkel & Feinberg, 2018), although the combined role of tailoring and targeting based on individual differences in Moral Foundations has not been tested. Moreover, while some longitudinal research has suggested an influence of personality on political decisions over time (Ekstrom & Federico, 2019) the exact role personality plays within political decision-making is still under debate (Hatemi et al., 2014).

Overall, the efficaciousness of psychologically tailored arguments and psychological targeting remains largely untested within the political domain while the causal influence of psychological constructs like Personality and Moral Foundations in political decision-making requires further investigation.

Objective

To investigate psychological-based argument targeting and tailoring, and the potential causal influence of personality and moral foundations in political decision-making, this study sought to test four hypotheses ² in terms of the targeting and tailoring of arguments for Leaving the European Union:

1. 1. In-group loyalty tailored arguments will be more effective for Remain-supporting voters who are high on the Moral Foundation of Ingroup Loyalty.

2. 2. Fairness tailored arguments will be more effective for Remain-supporting voters who are high on the Moral Foundation of Fairness.

3. 3. Openness tailored arguments will be more effective for Remain-supporting voters who are high on the personality trait Openness.

4. 4. Conscientiousness tailored arguments will be more effective for Remain-supporting voters who are high on the personality trait Conscientiousness.

Loyalty and Fairness (study 1) were chosen because they had previously been effective in moral framing research in the US, with Loyalty particularly appealing to conservatives, and Fairness to liberals (Voelkel & Feinberg, 2018). Openness and Conscientiousness (study 2) were chosen because, of the big 5 personality traits, they have shown the most robust links to liberal and conservative political attitudes respectively (Gerber et al., 2010).

Methods

Study 1 – Moral Foundations

398 Remain supporting participants ³ undertook an abbreviated ⁴ Moral Foundations Questionnaire (MFQ) and received MFT-based arguments tailored to the foundations of Loyalty or Fairness.

The research design was mixed; there was one independent variable, message tailoring with two levels – tailored or control. To ensure participants were assigned to the condition reflecting their most (or least) dominant trait, participants first completed a Moral Foundations questionnaire measuring their Loyalty and Fairness. Two standardised z-scores – one for each trait – were created for each participant ⁵. Participants were then assigned to read either the Loyalty or Fairness arguments based on their most extreme (highest or lowest) z-score, to maximise the power of the design. Those in the Loyalty condition received arguments tailored to high Loyalty, and a non-tailored control, whilst those in the Fairness condition received arguments tailored to high Fairness and a non-tailored control.

After the Moral Foundation Questionnaire participants read four counter-attitudinal arguments: three MFT tailored arguments and one non-tailored control. The control argument was included to test the causal influence of argument tailoring on decision-making; participants high vs. low on Loyalty and Fairness are not expected to rate a non-tailored control any differently.

After each argument participants were asked three questions about the argument, two measuring their attitude to the argument (argument credibility and argument compellingness) and one measuring behavioural intentions (intention to vote for a party making this argument) on a scale from 0 to 10 ⁶.

Prior to the analysis, and for analysis purposes, participants were assigned to a high or low Loyalty (/Fairness) group based on the average Loyalty (/Fairness) score in the sample. Those with scores less than or equal to the average were considered low.

A similar study with Leave supporting participants was discontinued due to a small sample size (N = 83).

Study 2 – Personality

A second study following the exact same procedure was conducted to assess the effect of personality-based argument tailoring. 395 Remain supporting participants undertook an abbreviated Big Five Inventory questionnaire (B-5) and received personality-based arguments tailored to the traits of Conscientiousness and Openness.

Again, a study investigating Leave supporting participants was discontinued due to a small sample size (N = 65).

Analysis Strategy

Given the mixed design, the primary statistical interest was the interaction effect between the high and low groups across the tailored and control arguments. To analyse these effects, we conducted two-way mixed ANOVA ⁷ models. These models assessed the between impact of moral foundation (/personality) group (high vs. low), the within impact of argument type (tailored vs. control) and the interaction effect on the argument scores. Given paired comparisons were performed separately for the three rating measures (credible, compellingness, vote intention), for simplicity these were combined for the ANOVA models.

The secondary interest was a between-subjects test of whether participants high on a given moral foundation (/personality) trait rated the arguments differently from those low on the same trait. To assess these paired comparisons independent t-tests were conducted across each of the three rating measures ⁸. Given participants rated the arguments on three separate dependent variables – Credibility, Compellingness and Vote – the statistical threshold for the paired comparisons was controlled using the Holm-Bonferroni correction. All p-values reported for the independent t-tests are two-sided.

Whilst the distributions were not normal, the sample sizes were considered sufficiently large to conduct both two-way mixed ANOVAs and independent t-tests.

Results

Study 1 - Moral Foundations

Loyalty condition

Two-way mixed ANOVA results showed a significant main effect of group on argument scores F(1, 1,172) = 302.96, p < .001, 𝜂 _p² = .21. There was also a significant main effect of argument type on argument scores F(1, 1,172) = 14.97, p < .001, 𝜂_p² = .013. However, there was no significant interaction effect for Loyalty, F(1, 1,172) = .0001, p = .99, 𝜂_p² < 0.001.

Walker Fig 1. – Mean argument rating of high and low Loyalty groups across the tailored and non-tailored arguments. (Error bars denote standard errors).

To investigate the between-subjects effects across the arguments independent t-tests were conducted.

Independent t-tests indicated that individuals high on Loyalty significantly differed from those low on Loyalty across the tailored arguments. Those high on Loyalty found the arguments, more credible, t(200.57) = 6.35, p < .001, d = .89, more compelling, t(177.84) = 8.63, p < .001, d = 1.5 and showed a greater intention to vote, t(169.6) = 8.63, p < .001, d = 1.58 than those low on Loyalty.

This pattern of significant differences between those high and low on Loyalty was also evident across the non-tailored control argument. Those high on Loyalty found the argument more credible, t(199.29) = 4.83, p < .001, d = .68, more compelling, t(169.27) = 6.54, p < .001, d = 1.2 and showed a greater intention to vote, t(155.78) = 6.49, p < .001, d = 1.33 than those low on Loyalty.

Fairness condition

Two-way mixed ANOVA results showed a significant main effect of group on argument scores, F(1, 1,058) = 32.28, p < .001, 𝜂_p² = .03. There was also a significant main effect of argument type on argument scores F(1, 1,058) = 64.98, p < .001, 𝜂_p² = .06. There was no significant interaction effect for Fairness, F(1, 1,058) = 2.86, p = .09, 𝜂_p² = 0.003.

Walker Fig 2. – Mean argument rating of high and low Fairness groups across the tailored and non-tailored arguments. (Error bars denote standard errors).

Independent t-tests indicated that individuals high on Fairness significantly differed from those low on Fairness across the tailored arguments. Those low on Fairness found the arguments, more credible, t(110.17) = −2.16, p = .03, d = −.34, more compelling, t(101.92) = −3.33, p < .01, d = −.54 and showed a greater intention to vote, t(99.04) = −3.42, p < .001, d = −.56 than those high on Fairness.

This pattern of significant differences between those high and low on Fairness was also evident across the non-tailored control argument. Those low on Fairness found the argument more credible, t(98.71) = −2.81, p < .01, d = −.46, more compelling, t(90.82) = −3.34, p < .01, d = −.49 and showed a greater intention to vote, t(82.01) = −3.3, p < .01, d = −.47 than those high on Fairness.

Study 2 - Personality

Conscientiousness condition

Two-way mixed ANOVA results showed a significant main effect of group on argument scores, F(1, 1,166) = 16.18, p < .001, 𝜂_p² = .014. There was also a significant main effect of argument type on argument scores, F(1, 1,166) = 12.78, p < .001, 𝜂_p² = .011. There was a marginally non-significant interaction effect for Conscientiousness, F(1, 1,166) = 3.41, p = .065, 𝜂_p² = 0.003.

Walker Fig 3. – Mean argument rating of high and low Conscientiousness groups across the tailored and non-tailored arguments. (Error bars denote standard errors).

Independent t-tests indicated that individuals high on Conscientiousness significantly differed from those low on Conscientiousness across the tailored arguments. Those high on Conscientiousness found the arguments, more credible, t(145.93) = 2.00, p = .048, d = .3, more compelling, t(135.12) = 2.33, p = .021, d = .35 and showed a greater intention to vote, t(131.05) = 2.56, p = .01, d = .39 than those low on Conscientiousness.

There were no significant differences across the non-tailored control argument: credibility, t(137.49) = .81, p = .42, d = .12, compellingness, t(134.97) = .50, p = .62, d = .07 and vote intention t(128.39) = 1.05, p = .30, d = .16.

Openness condition

Two-way mixed ANOVA results showed no significant main effect of group on argument scores, F(1, 1,136) = 3.15, p = .076, 𝜂_p² = .003. There was a significant main effect of argument type on argument scores, F(1, 1,136) = 19.95, p < .001, 𝜂_p² = .02. There was no significant interaction effect for Openness, F(1, 1,136) = 2.45, p = .12, 𝜂_p² = 0.002.

Walker Fig 4. – Mean argument rating of high and low Openness groups across the tailored and non-tailored arguments. (Error bars denote standard errors).

Independent t-tests indicated there was no significant differences across the tailored arguments: credibility, t(165.56) = −.62, p = .53, d = −.09, compellingness, t(176.7) = −.50, p = .61, d = −.07 and vote intention, t(181.73) = −1.18, p = .24, d = −.16.

On the non-tailored control argument there were no significant difference in terms of credibility, t(160.7) = −.62, p = .54, d = −.09 or compellingness, t(138.31) = −2.08, p = .04, d = −.32. There was a significant difference in terms of vote intention with those low on Openness displaying a greater intention to vote, t(140.85) = −3.02, p < .01, d = −.46 than those high on Openness.

Discussions

Across all four traits there was either no or weak evidence for the interaction between targeting and tailoring. There were however clear effects of targeting, such that participants high or low on a trait rated arguments differently in a manner consistent with that traits association with more liberal or conservative politics. The strength and consistency of these differences differed substantially across traits.

The strongest and most consistent difference was between individuals high and low on Loyalty. This difference – showing a receptivity towards the arguments for those high in Loyalty – was a large effect for the arguments tailored towards Loyalty (d = 0.89–1.58) but also large for the non-tailored argument (d = 0.68–1.33), and there was no interaction. The lack of an interaction between targeting and tailoring for Loyalty suggests that whilst targeting might be effective, these results do not enable us to be certain that receptivity to the moral value of Loyalty is critical; this difference could be mediated by other psychological or demographic differences between these groups. It should be noted however that, whilst the non-tailored argument contained less references to Loyalty, it did contain some (for example to ‘British students’), which may have been sufficient to appeal to participants high on Loyalty.

Conscientiousness was the only trait where participants in the high compared to low groups displayed a difference between the tailored and the non-tailored arguments across the pairwise comparisons. Those high compared to those low on Conscientiousness showed a consistent receptivity towards the tailored arguments whilst showing no differences across the non-tailored control argument. This combination of differences is at least consistent with the idea that personality-based argument tailoring can influence political-based decision-making. However, the absence of a significant interaction effect cautions the veracity of this interpretation. Detecting interaction effects often requires more power however, so the pattern of pairwise comparisons suggests this result is worth testing with a larger sample.

Participants high on Fairness consistently rated the Leave arguments as less persuasive than those low on Fairness across the tailored and non-tailored arguments. Participants high on Openness (compared to low) only differed in their evaluation of one outcome (vote intention) for the non-tailored argument – showing less receptivity towards it. For both Fairness and Openness there was no interaction. The lack of hypothesised effects across the Fairness and Openness conditions suggests that argument tailoring is particularly unlikely to prove effective in the absence of thoughtful targeting. In this instance the lack of an effect for Fairness and Openness could reflect the incongruence between the public discourse surrounding the Leave position and these psychological constructs.

Finally, it should be noted, all of these arguments were tested using arguments for Leaving the EU presented to Remain supporting participants. It would clearly be informative to extend this test to Leave supporting participants with arguments in favour of Remaining. Previous literature has highlighted associations between Loyalty and Conscientiousness and conservativism, and Openness and Fairness with liberalism, so the pattern of targeting effects found here should theoretically be reversed when making arguments for Remaining in the EU.

Conclusions

The results reveal that participants high or low on different traits respond differently to arguments for Leaving the EU. Some of these differences are large, with Remain supporters high on Loyalty consistently finding arguments much more persuasive than Remain supporters low on Loyalty. Some of these differences are more marginal, with participants high on Openness finding arguments for Leaving less persuasive than participants low on Openness only in one instance. Across all of the traits however there is no convincing evidence that the combination of targeting and tailoring is effective. The only potential exception is the case of Conscientiousness, in which participants high on this trait (compared to low) found the tailored arguments more persuasive but showed no difference on the non-tailored argument. Even in this case however the interaction between personality trait and message tailoring was not significant. Thus the results show substantive evidence for psychological targeting, but weak or no evidence for psychological targeting and tailoring.