Political Activists are Not Driven by Instrumental Motives: Evidence from Two Natural Field Experiments

Abstract

Are political activists driven by instrumental motives such as making a career in politics or mobilizing voters? We implement two natural field experiments in which party activists are randomly informed that canvassing is i) effective at mobilizing voters, or ii) effective for enhancing activists’ political careers. We find no effect of the treatments on activists’ intended and actual canvassing behaviour. The null finding holds despite a successful manipulation check and replication study, high statistical power, a natural field setting, and an unobtrusive measurement strategy. Using an expert survey, we show that the null finding shifted Bayesian posterior beliefs about the treatment’s effectiveness toward zero. The evidence thus casts doubt on two popular hypothesized instrumental drivers of political activism – voter persuasion and career concerns – and points toward expressive benefits as more plausible motives.

What motivates citizens to become active in politics? Scholars have long contended that political engagement is partly a product of instrumental ¹ considerations. While voting is hard to rationalize on instrumental grounds (Gelman, Silver and Edlin 2012), the case is quite different for engagement in a political campaign. Here, party supporters may well be motivated by their own effect on the election, given that local races are often close. Participation in a campaign is also a quintessential means to advance one’s political career (Black 1972, 146), (Abramowitz, McGlennon and Rapoport 1983, 1008), (Fox and Lawless 2005, 653). But, are such instrumental, outcome-oriented considerations actual drivers of political activism?

This paper studies whether party activists can be motivated on instrumental grounds. We present evidence from two field experiments conducted with a major European party. Study 1 took place during a national election and explored the role of individual effectiveness. We randomized whether 1,184 party activists received true information about the effectiveness of canvassing in mobilizing voters. Study 2 took place during a state-level election and assessed the role of career ambitions. Here, we randomized whether 1,885 party activists received information that key party leaders had once been active canvassers, accelerating their political careers. Our outcomes are respondents’ intended and actual canvassing behaviour, which we unobtrusively measure via the party’s canvassing app.

We find that neither the treatment highlighting the effectiveness of canvassing nor the treatment underlining that canvassing can advance one’s career affected party supporters’ political engagement. Intended and actual canvassing measured over two weeks (Study 1) and four weeks (Study 2) are highly similar across the treatment and control groups. The null findings are informative for six reasons. First, both studies are well-powered. Second, we find convergent evidence for intentions and behaviour. Third, both studies included a successful manipulation check. Fourth, the results are unlikely to be affected by social desirability bias since we use unobtrusive behavioural data coupled with information sent out on behalf of the party. Fifth, we find no meaningful effect heterogeneity. Sixth, we collected expert and lay people’s prior beliefs on the likely effects and showed that our data moves posterior beliefs toward zero. Taken together, the evidence thus casts doubt that political activists can be motivated on instrumental grounds, pointing to expressive motives as the more likely drivers (Hager et al. 2022).

Motivation

Why do citizens become politically active? Scholars of political behaviour argue that political engagement brings about both instrumental as well as expressive benefits (Gordon and Babchuk 1959). Instrumental benefits accrue from the outcome of political engagement (Hansen 1985). Expressive benefits accrue from the process of activism itself (Gerber et al. 2016). Given that political activism provides a public good, scholars commonly reject the notion that instrumental motives drive political engagement (Olson 1965; Gerber, Green and Larimer 2008). This holds particularly true for voting, given that one vote is unlikely to sway an election (Gelman, Silver and Edlin 2012). The empirical evidence, however, remains mixed (Enos and Fowler 2014). Bursztyn et al. (2023), for instance, find that turnout is significantly higher in close races, while Gerber et al. (2020) find no such link.

Whether instrumental motives explain other, more high-stake forms of political engagement remains an open question. The lack of evidence is surprising given that the impact of volunteering in a political campaign is orders of magnitude larger than that of casting a ballot. A good canvasser convinces hundreds of people to vote. In the setting studied in this paper, roughly 40 per cent of races were close (within a 5-point margin). Canvassers may thus well have had a noticeable impact. A second instrumental motive specific to participation in campaigns is the advancement of one’s political career. ² Active engagement on the campaign trail signals commitment to the party, strengthens activists’ networks and affords grassroots credibility, which may translate into being nominated and elected to party-internal or public office.

More so than voting, political activism may therefore be well driven by instrumental considerations. Indeed, Figure A.1 provides tentative support that instrumental considerations spark activism: We show a clear correlation between political activists’ canvassing efforts and their beliefs about i) the effectiveness of canvassing (Figure A.1a) and ii) the importance of canvassing for activists’ political careers (Figure A.1b). ³ What is more, in our setting, survey-evidence confirms that party activists do believe that canvassing is effective at convincing voters as well as advancing one’s political career. ⁴ And, an analysis of 3.5 million Tweets from the country of study further showcases that canvassing is endorsed by key party leaders, pointing to its effectiveness at persuading voters and advancing one’s career (see Section 1). This evidence, however, is not causal. To study both potential instrumental drivers – persuasion and career ambitions – we, therefore, implemented two natural field experiments, which we describe next.

Study 1: Are activists driven by a desire to mobilize voters?

Setting and Sample

To study whether political activists are driven by a desire to mobilize voters, we cooperated with a large European party and implemented a field experiment during a general election campaign. On behalf of the party, we invited the party’s list of core supporters to participate in an online survey via email. ⁵ The population can best be described as ‘activists’, that is, party supporters who were already active in the campaign or aspired to become active (details on the setting are in Section 2). As such, we study both the extensive margin (whether to become active) as well as the intensive margin (whether to increase one’s activism). The email and survey used a party template and no reference was made to the research team: 1,184 activists agreed to participate (response rate of 2.4 per cent). The survey was distributed two weeks ahead of the election. The descriptive statistics of the sample are in Table A2. The sample broadly maps onto the party’s membership statistics, though the sample is younger and includes more men. The sample also, naturally, had high prior levels of political activism with 43 per cent having canvassed before and 25 per cent having canvassed in the current campaign.

Treatment

After administering eight descriptive questions (see Figure A.2), respondents were asked: ‘Imagine a canvasser who talks to 100 non-voters. What do you think: How many of these 100 non-voters can the canvasser convince to go vote?’ Thereafter, half the sample was randomly ⁶ assigned to true information stating that a typical canvasser convinces sixteen out of one hundred non-voters to turn out. ⁷ To ease interpretation, the treatment screen also included a figure, which compared respondents’ estimates to the true number. The control group was not given this information screen (though the question about their prior beliefs was asked). After the treatment group was given the information, all respondents, including the control group, were asked to imagine a typical canvasser ‘in the party’s current campaign’ who had spoken to one hundred non-voters, and asked again how many non-voters the canvasser could convince to vote. The latter question serves as our measure for respondents’ posterior beliefs, that is, our manipulation check.

Outcome

After eliciting respondents’ posterior beliefs, we asked them whether they planned to canvass at all (extensive margin), and, if so, on how many days (intensive margin). After the survey, we also measured respondents’ actual canvassing behaviour until the election using the party’s canvassing smartphone application, which respondents used to register knocked doors (the details are in Section B). The app data allows us to unobtrusively measure if respondents went canvassing and, if so, on how many days as well as on how many doors they knocked on. Importantly, the behavioural data was collected before and after the treatment was administered during the entire campaign. We thus include all canvassing activity from the day of the survey until the election and also control for prior canvassing activity. As Table A2 shows, 15 per cent of respondents went canvassing, knocking on an average of thirty-two doors. Finally, we also combine the five outcomes into a standardized canvassing index. ⁸

Model

To estimate the effect of the treatment on respondents’ political engagement, we pre-registered the following model: ${y_i} = {\pi _0} + {\pi _1}{{\rm{T}}_i} + {\zeta ^T}{X_i} + {\varepsilon _i}$ . ⁹ Where ${y_i}$ is the canvassing outcome of interest. ${{\rm{T}}_i}$ is a dummy taking the value one treated individuals (effectiveness information) and zero otherwise. ${X_i}$ is the set of pre-registered controls, which are reported in Table 4. ${\varepsilon _i}$ is the error term. We report robust standard errors.

Manipulation check

Column 2 in Table 1 demonstrates that the treatment significantly affected respondents’ posterior beliefs. In the aggregate sample, the treatment group reports a posterior belief that is five percentage points higher compared to the control group. More importantly, Table 1 demonstrates that treated respondents who initially underestimated the effectiveness of canvassing, shifted their posterior belief upward by an average of 3 points. By contrast, treated respondents who overestimated the effectiveness of canvassing, shifted their posterior belief downward by an average of 14 points.

Table 1. Impact of effectiveness treatment on canvassing intentions and behaviour

Notes: Table 1 reports results from an OLS regression of the indicated outcome on the effectiveness treatment dummy. Robust standard errors are given in parentheses. All pre-registered control variables are included * $p \lt 0.10$ , ** $p \lt 0.05$ , *** $p \lt 0.01$ . Intentions capture whether a respondent plans to do any canvassing and, if so, on how many days they plan to go canvassing. Behaviour refers to actual canvassing behaviour measured through a smartphone app, capturing whether a respondent engaged in any canvassing and, if so, how many days they went canvassing and on how many doors they knocked. The Index combines the intentions and behaviour-based data into a standardized index.

Results

Did the effectiveness treatment change respondents’ political engagement? Panel A in Table 1 shows that treated individuals did not show different canvassing intentions or behaviours. ¹⁰ Given that we specifically shifted respondents’ beliefs, in Panels B and C in Table 1 we analyze the behaviour of under-estimators and overestimators separately. The Table confirms our null finding. Underestimators – who learned that canvassing is more effective than previously thought – are not more likely to canvass, and vice versa. ¹¹

Heterogeneity

To assess heterogeneity, we focus on the combined canvassing index and use all available pre-specified control variables to construct subgroups. Figure A.7a plots the coefficients of the treatment dummy and the interaction of the treatment with the respective covariates. Instrumental motives should be particularly strong for individuals who expect a close election as they are more likely to be pivotal. However, we do not find any heterogeneity in expected election closeness. Another potential source of heterogeneity is whether supporters expect their own party or the main competing party to knock on more doors. Again, however, we do not find any heterogeneity along this dimension. More broadly, Figure A.7a shows that there is no significant effect heterogeneity for any of the subgroups.

Study 2: Are activists driven by a desire to advance their careers?

Setting and Sample

A year later, we implemented a second field experiment with the same party to study the question of whether activists are motivated by advancing their political careers. This time, the experiment took place during a state-level electoral campaign for the state parliament (further details are in Figure A.3 and in Section 2). As in the effectiveness study, we sent out a survey on behalf of the party to its list of supporters, using the same unobtrusive template, and invited (potential) activists to participate in an online survey. 1,885 activists agreed to participate (response rate of 4.4 per cent). The survey was distributed four weeks before the election. The descriptive statistics of the sample are given in Table A.2. Compared to the first study, the sample of the career study is older and was, on average, less active during the campaign.

Treatment

After administering a similar set of background questions, respondents were randomly ¹² assigned to information that key party leaders are proud canvassers, which had accelerated their political careers. The specific script read as follows: ‘Many members of the [Party] board are proud canvassers. [Name of party leader 1], for instance, was an active canvasser during his youth, which allowed him to make valuable experiences for his political career. [Name of party leader 2], too, was one of the party’s first canvassers’. The control group was not given any information. In this study, we did not elicit individuals’ prior and posterior beliefs, which did not prove sensible without a specific estimate of the form elicited in the effectiveness study. Rather, we included a distinct manipulation question at the end of the survey. We asked: ‘What do you think: How useful is canvassing to make a career in politics?’ The answer choices – scored on a 6-point scale – ranged from ‘not useful at all’ to ‘very useful’.

Outcome

After administering the treatment, we collected the exact same outcomes as in the effectiveness study, that is, canvassing intentions and, after the survey, the behavioural canvassing data until the end of the campaign.

Manipulation check

Using the same model as in Study 1, Table 2 demonstrates that the treatment increased respondents’ beliefs that canvassing is an essential tool to advance one’s political career: treated respondents score 0.11 points higher on the 6-point scale. ¹³

Table 2. Impact of career treatment on canvassing

Notes: The Table reports results from an OLS regression of the indicated outcome on the effectiveness treatment dummy. Outcomes are defined in Table 1. Robust standard errors are given in parentheses. Demographic control variables are included. * $p \lt 0.10$ , ** $p \lt 0.05$ , *** $p \lt 0.01$

Results

Did the career prime increase respondents’ political engagement? Table 2 shows that respondents did not report different canvassing intentions and also did not change their behaviour relative to the control group. All estimated coefficients are close to zero. The null finding holds across both the extensive margin and the intensive margin. The coefficients are also highly similar across the self-reported survey data and the behavioural data. The evidence thus casts doubt on whether career ambitions are a meaningful driver of political engagement – despite activists stating that they are motivated by advancing their careers (more below; and in Figure A.8). Importantly, Study 2 has a minimum detectable effect size of 0.10 SD for the combined canvassing index, which allows us to rule out a substantively meaningful effect. Again, results are robust to omitting covariates (see Table A11).

Heterogeneity

Do we observe treatment effects for subgroups where the manipulation check is particularly strong or for other theoretically plausible subgroups? Tables A12 and A13 show that the manipulation check worked very well among men and respondents with initially lower career concerns (though the difference in treatment effects is not significant for the latter). But, the Tables also demonstrate that even for these sub samples we do not observe significant treatment effects. More broadly, Figure A.7b assesses treatment effect heterogeneity across all pre-registered covariates for the standardized canvassing index. The Figure confirms that there is no significant treatment effect heterogeneity for any subgroup.

Informativeness of the Null Findings

Statistical power

Are our experiments well-powered to detect reasonable effect sizes? To tackle this question, we compare estimated effects and minimum detectable effect sizes in other studies using survey-administered experiments to study political behaviour. Table A17 shows that the statistical power in our study compares well to the overall literature and to studies in the same context. The minimum detectable effect sizes on the canvassing index are 0.129 standard deviations (Study 1) and 0.101 standard deviations (Study 2). This compares favourably with an average detectable effect size of 0.183 across estimates identified in Table A17. The Minimum detectable effect sizes are also smaller than the average estimated effect size in the literature (0.189 standard deviations) even though this also includes insignificant effects. ¹⁴

Learning

Do our null results provide new information about the motives of political activists? To answer this question, we implemented surveys with three relevant samples: 1,107 supporters of the party, 600 lay people, and fifty-four experts (political scientists working on activism). ¹⁵ We first asked all respondents why they believe party supporters canvass. We asked this question to explore if the relevant samples believe that the primed instrumental motives could, conceivably, be increasing activism. As Figure A.8 shows, 71 per cent of activists, 86 per cent of lay people, and 63 per cent of experts believe that canvassers are, indeed, driven by the desire to persuade voters. Similarly, 43 per cent of activists, 51 per cent of lay people, and 54 per cent of experts believe that canvassers are motivated by the desire to improve their careers (multiple choices were possible).

Even if activists report that they are motivated by persuasion and career considerations, a critic might object that one would still not expect our specific treatments to affect canvassing. ¹⁶ To address this concern, we collected expert and lay people’s beliefs about the likely treatment effects in order to assess to what degree our results provide new information. In particular, we explained both experimental designs to both samples, provided them with the respective control group means, and then asked them what treatment effects they expected. We then integrated the resulting distribution of prior beliefs with the experimental estimates using Bayesian statistics (details are in Section D).

The results of the Bayesian analysis for the expert sample are presented in Table A19 and Figure 1. Two results stand out. First, our evidence provided new information about the expected effect size for the career experiment. The expected treatment effects based on the posterior belief distribution are much closer to zero than those based on the prior belief distribution. ¹⁷ For instance, experts believed that the career treatment would raise intentions by 5.2 percentage points, on average. Updating these prior beliefs with the experimental estimates leads to a posterior belief about the ATE of just 0.9 percentage points. Second, the posterior distributions of beliefs about the treatment effects are much tighter than the prior distributions for both studies. For example, the probability that the treatment effect on the actual behaviour of the effectiveness experiment falls between −2 and 2 percentage points increased from 46.8 per cent to 75.8 per cent for canvassing behaviour in the effectiveness experiment. This underlines that our results not only provided information about the mean but also increased experts’ and lay people’s certainty about the effect of the persuasion and career primes on political activism.

Figure 1. Distribution of expert beliefs about average treatment effects (ATEs).

Notes: The Figures display averaged expert beliefs about treatment effects. Grey bars indicate the averaged prior beliefs calculated by averaging the probability mass experts put on each interval. Dashed lines indicate the averaged posterior beliefs obtained by updating averaged prior beliefs using Bayes’ rule with a distribution of treatment effects obtained through bootstrapping (10,000 repetitions). Panel A displays beliefs about the effect of the career treatment. Panel B displays beliefs about the effect of the effectiveness treatment. Both panels show effects on canvassing intentions and behaviour.

Conclusion

What can the two null findings teach us about political engagement? If taken at face value, scholars are well-advised to continue to direct their focus on ‘expressive’ and social motives. Instrumental motivations that are closely linked to the outcome of political activism are seemingly less relevant. Importantly, this finding held across two very different ‘instrumental’ treatments, which – though both primed outcomes (the election or one’s career) – differed in the degree to which the outcome was a public or private good. That is, Study 1 primed a public good (affecting an election), while Study 2 primed a private good (affecting one’s career). The degree to which the good is public (and thus plagued by free-riding) therefore did not seem to play a role. In both studies, we found no effect for the instrumental treatments.

We must caution, however, that our studies only illuminated the drivers of engagement in one specific political context for one form of activism: canvassing. A skeptic may therefore ask whether instrumental benefits do matter when one studies different parties, different forms of engagement or more local elections. Reassuringly, we found no evidence that the treatments affected other forms of engagement, namely, social media activity (see Table 18). What is more, the fact that we do not find any effect heterogeneity (including for perceived election closeness), makes it difficult to put much trust in instrumental explanations. We should also underline that we found no effects across two different electoral campaigns, in a large Western-European democracy, which uses both majoritarian single-member districts as well as proportional party lists. Our evidence thus arguably offers a moderate degree of generalizability.