Email Mobilization Messages Suppress Turnout Among Black and Latino Voters: Experimental Evidence From the 2016 General Election

Abstract

Email can deliver mobilization messages at considerably lower cost than direct mail. While voters’ email addresses are readily available, experimental work from 2007 to 2012 suggests that email mobilization is ineffective in most contexts. Here, we use public data to reexamine the effectiveness of email mobilization in the 2016 Florida general election. Unsolicited emails sent from a university professor and designed to increase turnout had the opposite effect: emails slightly demobilizing voters. While the overall decrease in turnout amounted to less than 1 percent of the margin of victory in the presidential race in the state, the demobilizing effect was particularly pronounced among minority voters. Compared to voters from the same group who were assigned to control, black voters assigned to receive emails were 2.2 percentage points less likely to turn out, and Latino voters were 1.0 percentage point less likely to turn out. These findings encourage both campaigns and researchers to think critically about the use and study of massive impersonal mobilization methods.

Email mobilization

What is the effect of an unsolicited email encouragement on voter turnout? Campaigns currently employ a broad strategy of contact that includes both text messages (Roose 2018) and email (Astor 2019) to generate financial and electoral support. Voters opt-in to provide their email addresses to issue-advocacy groups, and national campaigns spend considerable effort to manage, merge, and then re-distribute these lists for use by political allies (Evers-Hillstrom and Erickson 2019). The consequence of these strategies is that voters receive electronic communication from campaigns, though they may not have explicitly “opted in” to receive these messages. Experiments conducted between 2007 and 2012 are mixed in their findings about email’s ability to generate votes (see Table 1). In this study, we re-examine email mobilization because, despite little experimental evidence in support of its effectiveness, campaigns continue to spend considerable capital to send emails to voters for support.

Table 1 Mixed Evidence for Email Mobilization

When email has been effective the senders were a trusted, official source (Malhotra, Michelson, and Valenzuela 2012). Outside of email mobilization, there is evidence that some efforts meant to increase turnout can inadvertently have the opposite effect (e.g., Cornwall and Kessler 2012; Grose and Russell 2008; Kousser and Mullin 2007; McCabe and Michelson 2015).

Here, we test the impact of email messages that prime social norms about voting (Gerber and Rogers 2009). The messages we send emanate from a publicly identifiable political science professor who is a member of our research team. We contact voters using a real identity to forestall concerns about deceptive messaging. While this experiment was designed to test the differential impact of injunctive and descriptive norms on turnout (Rivera, Hughes, and Gell-Redman 2016), we find no clear evidence that voters react differently to different treatment messages. Instead, we were very surprised to observe a small, but persistent decrease in turnout among voters assigned to receive a treatment message.

Experimental setting, approach, and data

We target voters who provide a valid email in the publicly available October 10, 2016, Florida Division of Elections voter roll. Random assignment to receive an email message was blocked on congressional district and self-reported race (See Table 2 in the SI). Of the more than 12 million registered voters in Florida, 503,859 provide a valid email and are assigned to a condition (Rivera, Hughes and Gell-Redman 2020).

Table 2 Comparison of Email Providers

Notes: SEs are omitted here but reported in the SI. Proportion Democrat is the proportion of registered Democrats among those registered as a Democrat or Republican. Column 2, Provide Email includes registered voters with email addresses that could not be reconciled to be valid, Assigned A Condition, includes only voters with a valid email. The Analytic Sample, used throughout the reported analysis, excludes voters who voted early in-person.

Of all votes cast during the 2016 general election in Florida, 39.7% were cast early in-person, 29.4% were by mail, and 30.5% were cast in-person on election day. Throughout the reported results, the analytic sample is the group of $328,181$ Florida voters who provide a valid email address and did not vote early in-person. We include those who voted by mail, because some of these voters could have sent in their ballot after receiving a message. We exclude voters who cast a ballot early in-person, because these votes were cast before the distribution of our treatment messages. The analytic sample is notably younger than the whole set of registered voters, but otherwise broadly similar in terms of party affiliation, gender identification, and ethnic/racial identification (Table 2, columns 1 & 4).

We contact voters via a single email sent from an active email address with a.edu top-level domain. All emails contain the subject heading “Please Remember to Vote Tomorrow,” begin with a greeting, and close with a brief paragraph that directs questions about the voting process to the Florida Department of State. ¹ In line with current campaign practices, we sent messages the morning of November 7, 2016, at 10am EST, one day before the election. Limitations on the sending infrastructure precluded the ability to measure whether a voter received or opened our emails (Hughes et al. 2019). All results are intent-to-treat effects.

Results

Outcome data are drawn from the June 14, 2017, official voter roll. Voters assigned to receive no contact – that is, the control group – turned out to vote at a rate of 78.7 percent (see SI Section 4.1.1). ² Our main experimental finding, shown in Figure 1, is that those who were randomly assigned to receive any email message turned out at a 0.53 percentage point lower rate ( ${\overline{\tau}} = - 0.53$ , robust $SE = 0.17$ ). ³

Figure 1 Reduction in rate of turnout among registered voters assigned to receive an email, compared to voters of the same race/ethnicity in the control group. All points are estimated in separate models, on data partitions noted on the x-axis. Thick lines are 1.64 times the robust standard error; thin lines are 1.96 times the robust standard error. See Table 4 and Table 6 in the SI.

We further examine the effect of email messages within groups of self-identified black, Latino, and white voters. ⁴ Among the 36,518 black voters in the analytic sample, 64.0 percent assigned to the control group voted, compared to 61.8 percent in the treatment group ( ${{\overline{\tau}}_b} = - 2.168$ , robust $SE = 0.585$ ). Among the 63,404 Latino voters in the analytic sample, 73.9 percent assigned to the control group voted, compared to 73.0 percent in the treatment group ( ${{\overline{\tau}}_l} = - 0.970$ , robust $SE = 0.414$ ). Finally, among the 204,054 white voters in the analytic sample, 84.0 percent assigned to the control group voted, compared to 83.7 percent in the treatment group ( ${{\overline{\tau}}_w} = 0.309$ , robust $SE = 0.193$ ). Plainly stated, these messages may have kept as many as 1,389 voters from going to the polls ( $95\%\, robust \,CI = [ - 2249, - 531]$ ).

Discussion

Email mobilization has attracted interest in part because of its potential to deliver results at very low cost. Our results show that this low-cost investment for campaigns may yield zero or even negative returns for some groups of voters. Like all field experiments, ours has idiosyncratic context and design features that require care in extrapolating to both existing findings and future contexts (Bates and Glennerster 2017). Specifically, our treatment messages were sent in the competitive 2016 election by a university professor with no personal or institutional connection to voters in the study. The African American voters who received our messages live in the shadow of twin legacies of exclusion. Not only have black voters been shut out of the political process (Dawson 1995), but the black community has also been excluded from, and in the extreme, exploited by the research community (Brandt 1978). In fact, one explanation for the particularly sharp depressive effect of our message among these voters is a lingering mistrust of academic researchers, especially those who lack a personal connection to the community.

Ultimately, our design cannot support direct claims about why turnout was reduced. However, our preferred conjecture is that receiving a treatment message induced stress in some subjects. Approximately 700 voters chose to respond to our message; many of these people used worried or anxious language. Voters’ decisions to reply occur after exposure to treatment, and so we are circumspect about the causal inferences permitted by these responses (Coppock 2019). It is possible, however, that when voters perceive the political environment to be hostile or unfavorable the circumstances of voting cause stress, and unsolicited communication may reinforce those negative impressions and lower voters’ internal motivation to turnout (Hassell and Settle 2017). This perspective is consonant with careful observational work that demonstrates that black voters who perceive a lower chance of electing their preferred candidate are less responsive to mobilization efforts (McGowen 2010).

There is no clear consensus within political science over the ethical norms that should guide experimentation (e.g., Desposato 2015). Our own view is that the guiding principle of experimentation should be to avoid doing harm wherever possible and to balance any potential harm against the knowledge to be gained. We expected that email messages that prime social norms might provide a low cost means to encourage turnout, and we reasoned that for at least some voters this would occur by increasing negative emotions (Gerber and Rogers 2009). On the one hand, priming norms may lower voters’ internal motivation (Hassell and Settle 2017) by increasing shame, stress, or anxiety (Marcus, Neuman, and MacKuen 2000; Panagopoulos 2010); on the other hand, priming norms may increase internal motivation by increasing pride (Panagopoulos 2010). The net effect of these and other forces cannot be known in advance. Because our messages decreased turnout, we must ask whether the study’s scientific benefits outweighed the harm done. Our answer is that the learning strongly reinforces findings in the literature that emphasize the importance of personal contact and connection to the community, particularly among minority voters.