Hostname: page-component-745bb68f8f-b6zl4 Total loading time: 0 Render date: 2025-01-24T23:32:51.719Z Has data issue: false hasContentIssue false
  • English
  • Français

Do Get-Out-the-Vote Calls Reduce Turnout? The Importance of Statistical Methods for Field Experiments

Published online by Cambridge University Press:  08 June 2005

KOSUKE IMAI
KOSUKE IMAI
Affiliation:
Princeton University
Get access Rights & Permissions [Opens in a new window]

Abstract

In their landmark study of a field experiment, Gerber and Green (2000) found that get-out-the-vote calls reduce turnout by five percentage points. In this article, I introduce statistical methods that can uncover discrepancies between experimental design and actual implementation. The application of this methodology shows that Gerber and Green's negative finding is caused by inadvertent deviations from their stated experimental protocol. The initial discovery led to revisions of the original data by the authors and retraction of the numerical results in their article. Analysis of their revised data, however, reveals new systematic patterns of implementation errors. Indeed, treatment assignments of the revised data appear to be even less randomized than before their corrections. To adjust for these problems, I employ a more appropriate statistical method and demonstrate that telephone canvassing increases turnout by five percentage points. This article demonstrates how statistical methods can find and correct complications of field experiments.

Type
FORUM
Information
American Political Science Review , Volume 99 , Issue 2 , May 2005 , pp. 283 - 300
Copyright
© 2005 by the American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Achen Christopher H. 1986. The Statistical Analysis of Quasi-experiments. Berkeley: University of California Press.
Adams Williams C., and Dennis J. Smith. 1980. “Effects of Telephone Canvassing on Turnout and Preferences: A Field Experiment.” Public Opinion Quarterly 44: 389– 95.Google Scholar
Angrist Joshua D., and Guido W. Imbens. 1995. “Two-stage least squares estimation of average causal effects in models with variable treatment intensity.” Journal of the American Statistical Association 90: 431– 42.Google Scholar
Angrist Joshua D., Guido W. Imbens, and Donald B. Rubin. 1996. “Identification of Causal Effects Using Instrumental Variables (with Discussion).” Journal of the American Statistical Association 91: 444– 55.Google Scholar
Balke Alexander, and Judea Pearl. 1997. “Bounds on treatment effects from studies with imperfect compliance.” Journal of the American Statistical Association 92: 1171– 76.Google Scholar
Bound John, David A. Jaeger, and Regina M. Baker. 1995. “Problems with instrumental variables estimation when the correlation between the instruments and the endogenous explanatory variable is weak.” Journal of the American Statistical Association 90: 443– 50.Google Scholar
Campbell Donald T., and Julian C. Stanley. 1963. Experimental and Quasi-experimental Designs for Research. Chicago: RAND McNally.
Dao James. 2000. “The 2000 Campaign: The Voters; Ringing Phones, Chiming Doorbells, Stuffed E-Mailboxes: The Great Voter Roundup.” New York Times.
Dehejia Rajeev H., and Sadek Wahba. 1999. “Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training Programs.” Journal of the American Statistical Association 94: 1053– 62.Google Scholar
Eldersveld Samuel J. 1956. “Experimental Propaganda Techniques and Voting Behavior.” American Political Science Review 50: 154– 65.Google Scholar
Fisher Ronald A. 1938. “Presidential Address: The First Session of the Indian Statistical Conference, Calcutta, 1938.” Sankhya 4: 14– 17.Google Scholar
Gerber Alan S., and Donald P. Green. 2000. “The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment.” American Political Science Review 94: 653– 63.Google Scholar
Gerber Alan S., and Donald P. Green. 2001. “Do Phone Calls Increase Voter Turnout? A Field Experiment.” Public Opinion Quarterly 65: 75– 85.Google Scholar
Gerber Alan S., Donald P. Green, and Edward H. Kaplan. 2002. “The Illusion of Learning from Observational Research.” Presented at the Economic Science Association.
Gosnell Harold F. 1927. Getting-Out-the-Vote: An Experiment in the Stimulation of Voting. Chicago: University of Chicago Press.
Green Donald P., and Alan S. Gerber. 2001. “Getting Out the Youth Vote: Results from Randomized Field Experiments.” Technical Report. Yale University.
Green Donald P., and Alan S. Gerber. 2002. “Reclaiming the Experimental Tradition in Political Science.” In Political Science: State of the Discipline. Vol. III, ed. I. Katznelson and H. V. Milner. New York: W. W. Norton, 805– 32.
Heckman James, Hidehiko Ichimura, Jeffrey Smith, and Petra Todd. 1998. “Characterizing Selection Bias Using Experimental Data.” Econometrica 66: 1017– 98.Google Scholar
Ho Daniel E., Kosuke Imai, Gary King, and Elizabeth A. Stuart. 2004. “Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference.” Technical Report, http://GKing.Harvard.Edu/.
Holland Paul W. 1986. “Statistics and Causal Inference (with Discussion).” Journal of the American Statistical Association 81: 945– 60.Google Scholar
Horiuchi Yusaku, Kosuke Imai, and Naoko Taniguchi. 2005. “Designing and Analyzing Randomized Experiments.” Technical Report, http://www.princeton.edu/~kimai/research.
Howell William, and Paul E. Peterson. 2002. The Education Gap: Vouchers and Urban Schools. Washington, DC: Brookings.
Imai Kosuke. 2003. “Essays on Political Methodology.” Ph.D. dissertation, Department of Government, Harvard University.
Imai Kosuke, and David A. van Dyk. 2004. “Causal Inference with General Treatment Regimes: Generalizing the Propensity Score.” Journal of the American Statistical Association, Theory and Methods 99: 854– 66.Google Scholar
Kinder Donald R., and Thomas R. Palfrey, eds. 1993. Experimental Foundations of Political Science. Ann Arbor: University of Michigan Press.
King Gary, and Langche Zeng. 2003. “When Can History Be Our Guide? The Pitfalls of Counterfactual Inference.” Technical Report, http://GKing.Harvard.Edu/.
McCullagh Peter, and John A. Nelder. 1989. Generalized Linear Models. 2nd Ed. London: Chapman & Hall.
Mill John Stuart. 1930/1843. A System of Logic, Ratiocinative and Inductive: Being a Connected View of the Principles of Evidence and the Methods of Scientific Investigation. London: Longman.
Miller Roy E., David E. Bositis, and Denise L. Baer. 1981. “Stimulating Voter Turnout in a Primary: Field Experiment with a Precinct Committeeman.” International Political Science Review 2: 445– 60.Google Scholar
Rosenbaum Paul R., and Donald B. Rubin. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika 70: 41– 55.Google Scholar
Rosenbaum Paul R., and Donald B. Rubin. 1985a. “The Bias Due to Incomplete Matching.” Biometrics 41: 103– 16.Google Scholar
Rosenbaum Paul R., and Donald B. Rubin. 1985b. “Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score.” American Statistician 39: 33– 38.Google Scholar
Wantchekon Leonard. 2003. “Clientelism and Voting Behavior: Evidence from a Field Experiment in Benin.” World Politics 55: 399– 422.Google Scholar
Submit a response

Comments

No Comments have been published for this article.