Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-27T11:37:07.416Z Has data issue: false hasContentIssue false
  • English
  • Français

Dealing with Separation in Logistic Regression Models

Published online by Cambridge University Press:  04 January 2017

Carlisle Rainey
Carlisle Rainey*
Affiliation:
Department of Political Science, Texas A&M University, 2010 Allen Building, College Station, TX 77843, USA
*
e-mail: crainey@tamu.edu (corresponding author)
Get access Rights & Permissions [Opens in a new window]

Abstract

When facing small numbers of observations or rare events, political scientists often encounter separation, in which explanatory variables perfectly predict binary events or nonevents. In this situation, maximum likelihood provides implausible estimates and the researcher might want incorporate some form of prior information into the model. The most sophisticated research uses Jeffreys’ invariant prior to stabilize the estimates. While Jeffreys’ prior has the advantage of being automatic, I show that it often provides too much prior information, producing smaller point estimates and narrower confidence intervals than even highly skeptical priors. To help researchers assess the amount of information injected by the prior distribution, I introduce the concept of a partial prior distribution and develop the tools required to compute the partial prior distribution of quantities of interest, estimate the subsequent model, and summarize the results.

Type
Articles
Information
Political Analysis , Volume 24 , Issue 3 , Summer 2016 , pp. 339 - 355
Copyright
Copyright © The Author 2016. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Author's note: I thank Mark Bell and Nicholas Miller for making their data available (Bell and Miller 2011). I thank Mike Alvarez, Mark Bell, Bryce Corrigan, Justin Esarey, David Firth, Paul Johnson, Nicholas Miller, Jamie Monogan, Chris Zorn, and two anonymous reviewers for helpful comments. I presented earlier versions of this article at the University of Kansas, Texas A&M University, the 2015 Annual Meeting of the Society for Political Methodology, the 2016 Annual Meeting of the Southern Political Science Association, and the 2016 State Politics and Policy Conference. The analyses presented here were conducted with R 3.2.2. All data and computer code (Rainey 2016) necessary for replication are available at github.com/carlislerainey/priors-for-separation and on the Political Analysis Dataverse at dx.doi.org/10.7910/DVN/VW7G2Q. Supplementary materials for this article are available on the Political Analysis Web site.

References

Ahlquist, John S. 2010. Building strategic capacity: The political underpinning of coordinated wage bargaining. American Political Science Review 104(1):171–88.Google Scholar
Albert, A., and Anderson, J. A. 1984. On the existence of maximum likelihood estimates in logistic regression models. Biometrika 71(1):1–10.Google Scholar
Barrilleaux, Charles, and Rainey, Carlisle. 2014a. The politics of need: Examining governors’ decisions to oppose the “Obamacare” Medicaid expansion. State Politics and Policy Quarterly 14(4):437–60.Google Scholar
Barrilleaux, Charles, and Rainey, Carlisle. 2014b. Replication data for: The politics of need: Examining governors’ decisions to oppose the “Obamacare” Medicaid expansion. http://dx.doi.org/10.15139/S3/12130.Google Scholar
Bell, Mark S., and Miller, Nicholas L. 2011. Replication data for: Questioning the stability-instability paradox. http://hdl.handle.net/1902.1/15892.Google Scholar
Bell, Mark S., and Miller, Nicholas L. 2015. Questioning the effect of nuclear weapons on conflict. Journal of Conflict Resolution 59(1):74–92.Google Scholar
Berry, Frances Stokes, and Berry, William D. 1990. State lottery adoptions as policy innovations: An event history analysis. American Political Science Review 84(2):395–415.CrossRefGoogle Scholar
Box, George E.P., and Taio, George C. 2011. Bayesian inference in statistical analysis. New York: John Wiley and Sons.Google Scholar
Brown, Robert L., and Kaplow, Jeffrey M. 2014. Talking peace: IAEA technical cooperation and nuclear proliferation. Journal of Conflict Resolution 58(3):402–28.CrossRefGoogle Scholar
Bueno de Mesquita, Bruce, and Riker, William. 1982. An assessment of the merits of selective nuclear proliferation. Journal of Conflict Resolution 26(2):283–306.Google Scholar
Carpenter, Bob, Gelman, Andrew, Hoffman, Matt, Lee, Daniel, Goodrich, Ben, Betancourt, Michael, Brubaker, Michael A., Guo, Jiqiang, Li, Peter, and Riddell, Allen. 2016. Stan: A probabilistic programming language. Journal of Statistical Software, forthcoming.Google Scholar
Casella, George, and Berger, Roger L. 2002. Statistical inference, 2nd ed. Pacific Grove, CA: Duxbury.Google Scholar
Casellas, Jason P. 2009. Coalitions in the House? The election of minorities to state legislatures and Congress. Political Research Quarterly 62(1):120–31.Google Scholar
Cederman, Lars-Erik, Skrede Gleditsch, Kristian, and Hug, Simon. 2013. Elections and ethnic civil war. Comparative Political Studies 46(3):387–417.CrossRefGoogle Scholar
Cox, Gary W., Kousser, Thad, and McCubbins, Matthew D. 2010. Party power or preferences? Quasi-experimental evidence from American state legislatures. Journal of Politics 72(3):799–811.CrossRefGoogle Scholar
DeRouen, Karl R. Jr., and Bercovitch, Jacob. 2008. Enduring internal rivalries: A new framework for the study of civil war. Journal of Peace Research 45(1):55–74.Google Scholar
Desposato, Scott, and Scheiner, Ethan. 2008. Governmental centralization and party affiliation: Legislator strategies in Brazil and Japan. American Political Science Review 102(4):509–24.Google Scholar
Fearon, James D. 1994. Signaling versus the balance of power and interests: An empirical test of a crisis bargaining model. Journal of Conflict Resolution 38(2):236–69.Google Scholar
Firth, David. 1993. Bias reduction of maximum likelihood estimates. Biometrika 80(1):27–38.Google Scholar
Fuhrmann, Matthew. 2012. How “Atoms for Peace” programs cause nuclear insecurity. Ithica, NY: Cornell University Press.Google Scholar
Gelman, Andrew. 2008. Scaling regression inputs by dividing by two standard deviations. Statistics in Medicine 27(15):2865–73.Google Scholar
Gelman, Andrew, and Jakulin, Aleks. 2007. Bayes: Radical, liberal, or conservative? Statistica Sinica 17(2):422–26.Google Scholar
Gelman, Andrew, Jakulin, Aleks, Grazia Pittau, Maria, and Su, Yu-Sung. 2008. A weakly informative prior distribution for logistic and other regression models. Annals of Applied Statistics 2(4):1360–83.Google Scholar
Geyer, Charles J. 2009. Likelihood inference in exponential families and directions of recession. Electronic Journal of Statistics 3:259–89.Google Scholar
Gill, Jeff. 2008. Bayesian methods: A social and behavioral science approach, 2nd ed. Boca Raton, FL: Chapman and Hall.Google Scholar
Gill, Jeff, and Walker, Lee D. 2005. Elicited priors for Bayesian model specifications in political science research. Journal of Politics 67(3):841–72.Google Scholar
Gross, Justin H. 2015. Testing what matters (if you must test at all): A context-driven approach to substantive and statistical significance. American Journal of Political Science 59(3):775–788.Google Scholar
Heinze, Georg, and Schemper, Michael. 2002. A solution to the problem of separation in logistic regression. Statistics in Medicine 21(16):2409–19.Google Scholar
Heller, William B., and Mershon, Carol. 2008. Dealing in discipline: Party switching and legislative voting in the Italian Chamber of Deputies, 1988–2000. American Journal of Political Science 52(4):910–24.Google Scholar
Jackman, Simon. 2000. Estimation and inference via Bayesian simulation: An introduction to Markov chain Monte Carlo. American Journal of Political Science 44(2):369–98.Google Scholar
Jeffreys, H. 1946. An invariant form of the prior probability in estimation problems. Proceedings of the Royal Society of London A 186 (1007):453–61.Google Scholar
King, Gary, Tomz, Michael, and Wittenberg, Jason. 2000. Making the most of statistical analyses: Improving interpretation and presentation. American Journal of Political Science 44(2):341–55.Google Scholar
Leeman, Lucas, and Mares, Isabella. 2014. The adoption of proportional representation. Journal of Politics 76(2):461–78.Google Scholar
Lesaffre, E., and Albert, A. 1989. Partial separation in logistic discrimination. Journal of the Royal Statistical Society B 51(1):109–16.Google Scholar
Mares, Isabela. 2015. From open secrets to secret voting: Democratic electoral reforms and voter autonomy. Cambridge Studies in Comparative Politics. Cambridge, UK: Cambridge University Press.Google Scholar
Martin, Lanny W., and Stevenson, Randolph T. 2001. Government formation in parliamentary democracies. American Journal of Political Science 45(1):33–50.Google Scholar
McCullagh, Peter, and Nelder, John A. 1989. Generalized linear models, 2nd ed. Boca Raton, FL: Chapman and Hall.Google Scholar
Mearsheimer, John J. 1993. The case for a Ukrainian nuclear deterrent. Foreign Affairs 72(3):50–66.CrossRefGoogle Scholar
Minozzi, William, and Volden, Craig. 2013. Who heeds the call of the party in Congress? Journal of Politics 75(3):787–802.Google Scholar
Mueller, John. 1988. The essential irrelevance of nuclear weapons: Stability in the postwar world. International Security 13(2):55–79.Google Scholar
Peterson, Timothy M., and Cooper Drury, A. 2011. Sanctioning violence: The effect of third-party economic coercion on militarized conflict. Journal of Conflict Resolution 55(4):580–605.Google Scholar
Poirier, Dale. 1994. Jeffreys’ prior for logit models. Journal of Econometrics 63(2):327–39.Google Scholar
Rainey, Carlisle. 2014. Arguing for a negligible effect. American Journal of Political Science 58(4):1083–91.Google Scholar
Rainey, Carlisle. 2016. Replication data for: Dealing with separation in logistic regression models. http://dx.doi.org/10.7910/DVN/VW7G2Q.Google Scholar
Rauchhaus, Robert. 2009. Evaluating the nuclear peace hypothesis: A quantitative approach. Journal of Conflict Resolution 53(2):258–78.Google Scholar
Reiter, Dan. 2014. Security commitments and nuclear proliferation. Foreign Policy Analysis 10(1):61–80.Google Scholar
Rocca, Michael S., Sanchez, Gabriel R., and Morin, Jason L. 2011. The institutional mobility of minority members of Congress. Political Research Quarterly 64(4):897–909.Google Scholar
Sagan, Scott D. 1994. The perils of proliferation. International Security 18(4):66–107.Google Scholar
Smith, Daniel A., and Fridkin, Dustin. 2008. Delegating direct democracy: Interparty legislative competition and the adoption of the initiative in the American states. American Political Science Review 102(3):333–350.Google Scholar
Stan Development Team. 2016a. RStan: The R interface to Stan, Version 2.9.0. http://mc-stan.org.Google Scholar
Stan Development Team. 2016b. RStanArm: Bayesian applied regression modeling via Stan. http://mc-stan.org.Google Scholar
Vining, Richard L. Jr., Wilhelm, Teena, and Collens, Jack D. 2015. A market-based model of state Supreme Court news: Lessons from capital cases. State Politics and Policy Quarterly 15(1):3–23.Google Scholar
Weisiger, Alex. 2014. Victory without peace: Conquest, insurgency, and war termination. Conflict Management and Peace Science 31(4):357–82.Google Scholar
Western, Bruce, and Jackman, Simon. 1994. Bayesian inference for comparative research. American Political Science Review 88(2):412–23.Google Scholar
Wolfinger, Raymond E., and Rosenstone, Steven J. 1980. Who votes? New Haven: Yale University Press.Google Scholar
Zorn, Christopher. 2005. A solution to separation in binary response models. Political Analysis 13(2):157–70.Google Scholar
Supplementary material: PDF

Rainey supplementary material

Appendix

Download Rainey supplementary material(PDF)
PDF 309.4 KB