Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-13T02:16:05.507Z Has data issue: false hasContentIssue false
  • English
  • Français

Treatment Spillover Effects across Survey Experiments

Published online by Cambridge University Press:  04 January 2017

John E. Transue ,
Daniel J. Lee  and
John H. Aldrich
John E. Transue*
Affiliation:
Department of Political Science and Institute for Legal, Legislative, and Policy Studies in the Center for State Policy and Leadership, University of Illinois at Springfield, Mail Stop PAC 350, Springfield, IL 62703
Daniel J. Lee
Affiliation:
Department of Political Science, Michigan State University, 303 South Kedzie Hall, East Lansing, MI 48824. e-mail: leedan@msu.edu
John H. Aldrich
Affiliation:
Department of Political Science, Duke University, 326 Perkins Library, Box 90204, Durham, NC 27708-0204. e-mail: aldrich@duke.edu
*
e-mail: jtran8@uis.edu (corresponding author)
Get access Rights & Permissions [Opens in a new window]

Abstract

Embedding experiments within surveys has reinvigorated survey research. Several survey experiments are generally embedded within a survey, and analysts treat each of these experiments as self-contained. We investigate whether experiments are self-contained or if earlier treatments affect later experiments, which we call “experimental spillover.” We consider two types of bias that might be introduced by spillover: mean and inference biases. Using a simple procedure, we test for experimental spillover in two data sets: the 1991 Race and Politics Survey and a survey containing several experiments pertaining to foreign policy attitudes. We find some evidence of spillover and suggest solutions to avoid bias.

Type
Research Article
Information
Political Analysis , Volume 17 , Issue 2 , Spring 2009 , pp. 143 - 161
Copyright
Copyright © The Author 2009. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Authors' note: Replication materials and an Appendix are available on the Political Analysis Web site. We would like to thank Richard Herrmann for sharing data and helping us understand the data's structure. We also would like to thank Paul Sniderman and Eric Lawrence for their advice, as well as presentation participants at Public Institutions and Public Choice at Duke University, University of Illinois at Urbana-Champaign, and Essex University for helpful comments on earlier drafts of this paper. Conflict of interest statement. None declared.

References

Brambor, Thomas, Clark, William R., and Golder, Matt. 2006. Understanding interaction models: Improving empirical analyses. Political Analysis 14: 63–82.Google Scholar
Gaines, Brian J., Kuklinski, James H., and Quirk, Paul J. 2007. “The logic of the survey experiment reexamined”. Political Analysis 15: 1–20.Google Scholar
Gilens, Martin. 1999. Why Americans hate welfare: Race, media, and the politics of antipoverty policy, studies in communication, media, and public opinion. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
Herrmann, Richard K., Tetlock, Philip E., and Visser, Penny S. 1999. “Mass public decisions on going to war: A cognitive-interactionist framework”. American Political Science Review 93: 553–74.CrossRefGoogle Scholar
Krosnick, Jon. 1999. “Survey research”. Annual Review of Psychology 50: 537–67.Google Scholar
Marcus, George E., Sullivan, John L., Theiss-Morse, Elizabeth, and Wood, Sandra. 1995. With malice toward some: How people make civil liberties judgments. New York: Cambridge University Press.Google Scholar
Sniderman, Paul M., and Carmines, Edward G. 1997. Reaching beyond race. Cambridge, MA: Harvard University Press.Google Scholar
Tourangeau, Roger, Rips, Lance J., and Rasinski, Kenneth. 2000. The psychology of survey response. Cambridge: Cambridge University Press.Google Scholar
Wilcox, Nathaniel, and Wlezien, Christopher. 1993. “The contamination of responses to survey items: economic perceptions and political judgments”. Political Analysis 5: 181–213.Google Scholar
Zaller, John. 1992. The nature and origins of mass opinion. Cambridge: Cambridge University Press.Google Scholar