Post-election surveys measure voter turnout in a variety of ways. The Canadian Election Study (CES) simply asks respondents whether or not they voted. However, existing research shows that some abstainers report having voted when they in fact did not (Granberg and Holmberg, Reference Granberg and Holmberg1991; Selb and Munzert, Reference Selb and Munzert2013). If this misreporting is correlated with other traits, analysis based on the data can be biased. One possible solution to reduce the incentive to overreport is to reframe the turnout question.
The British Election Study presents respondents with a short preamble (SP) before asking them if they voted or not. Such SPs state that some people abstain at elections, and that they do so for a variety of reasons. No published study has ever tested if including a SP impacts on reported behavior. The CES fielded an experiment in its online survey following the 2015 federal election to answer this very question.
The survey was conducted by the firm Survey Sampling International (SSI). Small monetary incentives were used to encourage participation. Unfortunately, SSI's recruitment methods make it impossible to compute a response rate.Footnote 1 7,557 Rs participated in the pre-election telephone survey, and 4,408 also completed the online post-election survey (attrition rate : 41.7%). A random half of Rs was exposed to a SP before being asked whether or not they voted (see Table 1). The SP mentions: “some people are not able to vote because they are sick or busy, or for some other reason. Others do not want to vote.” Experimental groups are balanced on observable characteristics. Non-response items are recoded as missing data (≈2%).
Table 1 Experimental Conditions in the Short Preamble Experiment
Table 2 presents the results. Subgroup analyses show that the SP reduces reported turnout among people with disabilities (−5.8 percentage points). However, this effect is largely diluted when all Rs are combined. In the whole sample, reported turnout is 87.9% in the control group, against 87.0% in the treatment group. This difference is not statistically significant.Footnote 2
Table 2 Results for the Short Preamble Experiment
The American National Election Studies measure voter turnout using a different approach: they combine a SP with face-saving response items (FSRIs). FSRIs allow respondents to report that they abstained while simultaneously justifying why they did not vote (Duff et al., Reference Duff, Hanmer, Park and White2007; Belli et al., Reference Belli, Moore and VanHoewyk2006).
In a recent publication, Morin-Chassé et al. (Reference Morin-Chassé, Bol, Stephenson and St-Vincent2017) present the results of 19 experiments testing the efficacy of replacing yes or no options with FSRIs (see Table 3). In Figure 1, Case IDs 1 to 5 present five additional experiments fielded as part of the same project.Footnote 3 The lines below report the same results as those published before by Morin-Chassé et al. Finally, the two bottom lines present cumulative effect estimates. The first is the average effect size of (−7.27 pp); the second is based on the combination of all individual survey responses (−6.82 pp).
Table 3 Experimental Conditions in the Face-Saving Response Items Experiment
Figure 1 Using Face-Saving Response Items to Reduce Vote Overrepporting: Effects Measured in the 24 Survey Experiments Fielded as Part of the Making Electoral Democracy Works Project (95% CI).
Overall, the findings reported in this Short Report suggest that combining a SP with FSRIs is a valuable approach to reduce vote overreporting. The main limitation of these experiments is the impossibility to validate self-report data with official voting records.
SUPPLEMENTARY MATERIALS
The appendix is available online as supplementary material at https://doi.org/10.1017/XPS.2018.1
