Hostname: page-component-78c5997874-xbtfd Total loading time: 0 Render date: 2024-11-10T11:07:00.017Z Has data issue: false hasContentIssue false
  • English
  • Français

Considering Network Effects in the Design and Analysis of Field Experiments on State Legislatures

Published online by Cambridge University Press:  01 January 2021

Sayali Phadke  and
Bruce A. Desmarais
Sayali Phadke*
Affiliation:
The Pennsylvania State University, State College, PA, USA
*
Bruce A. Desmarais, Department of Political Science, The Pennsylvania State University, Pond Lab, State College, PA 16803, USA. Email: bdesmarais@psu.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Recent work on legislative politics has documented complex patterns of interaction and collaboration through the lens of network analysis. In a largely separate vein of research, the field experiment—with many applications in state legislatures—has emerged as an important approach in establishing causal identification in the study of legislative politics. The stable unit treatment value assumption (SUTVA)—the assumption that a unit's outcome is unaffected by other units' treatment statuses— is required in conventional approaches to causal inference with experiments. When SUTVA is violated via networked social interaction, treatment effects spread to control units through the network structure. We review recently developed methods that can be used to account for interference in the analysis of data from field experiments on state legislatures. The methods we review require the researcher to specify a spillover model, according to which legislators influence each other, and specify the network through which spillover occurs. We discuss these and other specification steps in detail. We find mixed evidence for spillover effects in data from two previously published field experiments. Our replication analyses illustrate how researchers can use recently developed methods to test for interference effects, and support the case for considering interference effects in experiments on state legislatures.

Keywords

field experimentexperimental methodsmethodologynetworkanalysisquantitative methodssimulationslegislative behaviorlegislativepoliticslegislator preferenceslegislative politics
Type
Research Article
Information
State Politics & Policy Quarterly , Volume 19 , Issue 4 , December 2019 , pp. 451 - 473
Copyright
Copyright © The Author(s) 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ansari, Asim, Koenigsberg, Oded, and Stahl, Florian. 2011. “Modeling Multiple Relationships in Social Networks.” Journal of Marketing Research 48 (4): 713–728.CrossRefGoogle Scholar
Aral, Sinan, and Walker, Dylan. 2014. “Tie Strength, Embeddedness, and Social Influence: A Large-Scale Networked Experiment. Management Science 60 (6): 1352–1370.CrossRefGoogle Scholar
Bapna, Ravi, and Umyarov, Akhmed. 2015. “Do Your Online Friends Make You Pay? A Randomized Field Experiment on Peer Influence in Online Social Networks.” Management Science 61 (8): 1902–1920.CrossRefGoogle Scholar
Basu, Debabrata. 2011. “Randomization Analysis of Experimental Data: The Fisher Randomization Test.” In Selected Works of Debabrata Basu, eds. DasGupta, Anirban. New York: Springer, 305–325.CrossRefGoogle Scholar
ben Aaron, James, Denny, Matthew, Desmarais, Bruce, and Wallach, Hanna. 2017. “Transparency by conformity: A field experiment evaluating openness in local governments.” Public Administration Review 77 (1): 68–77.CrossRefGoogle Scholar
Bergan, Daniel E. 2009. “Does Grassroots Lobbying Work? A Field Experiment Measuring the Effects of an E-mail Lobbying Campaign on Legislative Behavior.” American Politics Research 37 (2): 327–352.CrossRefGoogle Scholar
Bergan, Daniel E., and Cole, Richard T.. 2015. “Call Your Legislator: A Field Experimental Study of the Impact of a Constituency Mobilization Campaign on Legislative Voting. Political Behavior 37 (1): 27–42.CrossRefGoogle Scholar
Bernhard, William, and Sulkin, Tracy. 2013. “Commitment and consequences: Reneging on cosponsorship pledges in the US House.” Legislative Studies Quarterly 38 (4): 461–487.CrossRefGoogle Scholar
Bond, Robert M., Fariss, Christopher J., Jones, Jason J., Kramer, Adam D. I., Marlow, Cameron, Settle, Jaime E., and Fowler, James H.. 2012. “A 61-Million-Person Experiment in Social Influence and Political Mobilization.” Nature 489 (7415): 295–298.CrossRefGoogle ScholarPubMed
Bowers, Jake, Desmarais, Bruce A., Frederickson, Mark, Ichino, Nahomi, Lee, Hsuan-Wei, and Wang, Simi. 2018. “Models, Methods and Network Topology: Experimental Design for the Study of Interference.” Social Networks 54:196–208.CrossRefGoogle Scholar
Bowers, Jake, Fredrickson, Mark M., and Aronow, Peter M.. 2016. “Research Note: A More Powerful Test Statistic for Reasoning about Interference between Units.” Political Analysis 24:395–403.CrossRefGoogle Scholar
Bowers, Jake, Fredrickson, Mark M., and Panagopoulos, Costas. 2013. “Reasoning about Interference between Units: A General Framework.” Political Analysis 21:97–124.CrossRefGoogle Scholar
Bratton, Kathleen A., and Rouse, Stella M.. 2011. “Networks in the Legislative Arena: How Group Dynamics Affect Cosponsorship.” Legislative Studies Quarterly 36 (3): 423–460.CrossRefGoogle Scholar
Broockman, David E. 2013. “Black Politicians Are More Intrinsically Motivated to Advance Blacks? Interests: A Field Experiment Manipulating Political Incentives.” American Journal of Political Science 57 (3): 521–536.CrossRefGoogle Scholar
Burt, Ronald S. 1987. “Social Contagion and Innovation: Cohesion versus Structural Equivalence.” American Journal of Sociology 92 (6): 1287–1335.CrossRefGoogle Scholar
Butler, Daniel M., and Nickerson, David W.. 2011. “Can Learning Constituency Opinion Affect How Legislators Vote? Results from a Field Experiment.” Quarterly Journal of Political Science 6 (1): 55–83.CrossRefGoogle Scholar
Cabin, Robert J., and Mitchell, Randall J.. 2000. “To Bonferroni or not to Bonferroni: When and how are the questions.” Bulletin of the Ecological Society of America 81 (3): 246–248.Google Scholar
Coppock, Alexander. 2014. “Information Spillovers: Another Look at Experimental Estimates of Legislator Responsiveness.” Journal of Experimental Political Science 1 (2): 159–169.CrossRefGoogle Scholar
Coppock, Alexander. 2016. “Information Spillovers: Another Look at Experimental Estimates of Legislator Responsiveness—CORRIGENDUM.” Journal of Experimental Political Science 3 (2): 206–208.CrossRefGoogle Scholar
Desmarais, Bruce A., Moscardelli, Vincent G., Schaffner, Brian F., and Kowal, Michael S.. 2015. “Measuring Legislative Collaboration: The Senate Press Events Network.” Social Networks 40:43–54.CrossRefGoogle Scholar
Dodds, Peter Sheridan, and Watts, Duncan J.. 2004. “Universal Behavior in a Generalized Model of Contagion.” Physical Review Letters 92 (21): 218701.CrossRefGoogle Scholar
Ferejohn, John. 1986. “Logrolling in an Institutional Context: A Case Study of Food Stamp Legislation.” In Congress and Policy Change. Vol. 223, eds. Wright, Gerald C., Rieselbach, Leroy N., and Dodd, Lawrence C.. New York: Agathon Press, 224–225.Google Scholar
Fowler, James H. 2006. “Connecting the Congress: A Study of Cosponsorship Networks.” Political Analysis 14:456–487.CrossRefGoogle Scholar
Gerber, Alan S., Green, Donald P., and Larimer, Christopher W.. 2008. “Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment.” American Political Science Review 102 (1): 33–48.CrossRefGoogle Scholar
Grose, Christian R., Malhotra, Neil, and Van Houweling, Robert Parks. 2015. “Explaining Explanations: How Legislators Explain Their Policy Positions and How Citizens React.” American Journal of Political Science 59 (3): 724–743.CrossRefGoogle Scholar
Kim, Jang Hyun, and Barnett, George. 2012. “Comparing the Influence of Social Networks Online and Offline on Decision Making: The U.S. Senate Case.” In E-politics and Organizational Implications of the Internet: Power, Influence, and Social Change. Hershey, PA: IGI Global, 198–219.CrossRefGoogle Scholar
Kirkland, Justin H. 2011. “The Relational Determinants of Legislative Outcomes: Strong and Weak Ties between Legislators.” The Journal of Politics 73 (3): 887–898.CrossRefGoogle Scholar
Kirkland, Justin H. 2013. “Hypothesis Testing for Group Structure in Legislative Networks.” State Politics & Policy Quarterly 13 (2): 225–243.CrossRefGoogle Scholar
Kirkland, Justin H., and Gross, Justin H.. 2014. “Measurement and Theory in Legislative Networks: The Evolving Topology of Congressional Collaboration.” Social Networks 36:97–109.CrossRefGoogle Scholar
Lin, Yu-Ru, Margolin, Drew, and Lazer, David. 2016. “Uncovering Social Semantics from Textual Traces: A Theory-Driven Approach and Evidence from Public Statements of U.S. Members of Congress.” Journal of the Association for Information Science and Technology 67 (9): 2072–2089.CrossRefGoogle Scholar
Little, Roderick J., and Rubin, Donald B.. 2000. “Causal Effects in Clinical and Epidemiological Studies via Potential Outcomes: Concepts and Analytical Approaches.” Annual Review of Public Health 21 (1): 121–145.CrossRefGoogle ScholarPubMed
Masket, Seth E., 2008. “Where you sit is where you stand: The impact of seating proximity on legislative cue-taking.” Quarterly Journal of Political Science 3:301–311.CrossRefGoogle Scholar
Masket, Seth, and Shor, Boris. 2015. “Polarization without Parties: Term Limits and Legislative Partisanship in Nebraska's Unicameral Legislature.” State Politics & Policy Quarterly 15 (1): 67–90.CrossRefGoogle Scholar
Matthews, Donald R. 1959. “The Folkways of the United States Senate: Conformity to Group Norms and Legislative Effectiveness.” American Political Science Review 53 (4): 1064–1089.CrossRefGoogle Scholar
Muchnik, Lev, Aral, Sinan, and Taylor, Sean J.. 2013. “Social Influence Bias: A Randomized Experiment.” Science 341 (6146): 647–651.CrossRefGoogle ScholarPubMed
Nyhan, Brendan, and Montgomery, Jacob M.. 2015. “Connecting the Candidates: Consultant Networks and the Diffusion of Campaign Strategy in American Congressional Elections.” American Journal of Political Science 59 (2): 292–308.CrossRefGoogle Scholar
Paluck, Elizabeth Levy. 2011. “Peer Pressure against Prejudice: A High School Field Experiment Examining Social Network Change.” Journal of Experimental Social Psychology 47 (2): 350–358.CrossRefGoogle Scholar
Peng, Tai-Quan, Liu, Mengchen, Wu, Yingcai, and Liu, Shixia. 2016. “Follower-Followee Network, Communication Networks, and Vote Agreement of the US Members of Congress.” Communication Research 43 (7): 996–1024.CrossRefGoogle Scholar
Porter, Mason A., Mucha, Peter J., Newman, Mark E. J., and Warmbrand, Casey M.. 2005. “A Network Analysis of Committees in the U.S. House of Representatives.” Proceedings of the National Academy of Sciences of the United States of America 102 (20): 7057–7062.CrossRefGoogle ScholarPubMed
Ringe, Nils, Victor, Jennifer Nicoll, and Gross, Justin H.. 2013. “Keeping Your Friends Close and Your Enemies Closer? Information Networks in Legislative Politics.” British Journal of Political Science 43 (3): 601–628.CrossRefGoogle Scholar
Rogowski, Jon C., and Sinclair, Betsy. 2012. “Estimating the Causal Effects of Social Interaction with Endogenous Networks.” Political Analysis 20 (3): 316–328.CrossRefGoogle Scholar
Sekhon, Jasjeet S. 2008. “The Neyman-Rubin Model of Causal Inference and Estimation Via Matching Methods.” In The Oxford Handbook of Political Methodology, eds. Box-Steffensmeier, Janet, Brady, Henry, and Collier, David. Oxford: Oxford University Press, 271–299.Google Scholar
Shalizi, Cosma Rohilla, and Thomas, Andrew C.. 2011. “Homophily and contagion are generi-cally confounded in observational social network studies.” Sociological methods & research 40 (2): 211–239.CrossRefGoogle ScholarPubMed
Simes, R. John. 1986. “An Improved Bonferroni Procedure for Multiple Tests of Significance.” Biometrika 73 (3): 751–754.CrossRefGoogle Scholar
Stevens, John R., Masud, Abdullah Al, and Suyundikov, Anvar. 2017. “A Comparison of Multiple Testing Adjustment Methods with Block-Correlation Positively-Dependent Tests.” PLoS ONE 12(4): e0176124.CrossRefGoogle ScholarPubMed
Valentini, Gabriele, Hamann, Heiko, and Dorigo, Marco. 2014. “Self-Organized Collective Decision Making: The Weighted Voter Model.” In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems. Paris, France: International Foundation for Autonomous Agents and Multiagent Systems, 45–52.Google Scholar
Victor, Jennifer Nicoll, and Ringe, Nils. 2009. “The Social Utility of Informal Institutions: Caucuses as Networks in the 110th U.S. House of Representatives.” American Politics Research 37 (5): 742–766.CrossRefGoogle Scholar
Wojcik, Stefan. 2017. “Why Legislative Networks? Analyzing Legislative Network Formation.” Political Science Research and Methods 7(3): 505–522.CrossRefGoogle Scholar