Hostname: page-component-78c5997874-j824f Total loading time: 0 Render date: 2024-11-11T07:00:12.976Z Has data issue: false hasContentIssue false
  • English
  • Français

A Theory of Statistical Inference for Matching Methods in Causal Research

Published online by Cambridge University Press:  04 October 2018

Stefano M. Iacus ,
Gary King  and
Giuseppe Porro
Stefano M. Iacus*
Affiliation:
Department of Economics, Management and Quantitative Methods, University of Milan, Via Conservatorio 7, I-20124 Milan, Italy. Email: stefano.iacus@unimi.it
Gary King
Affiliation:
Institute for Quantitative Social Science, 1737 Cambridge Street, Harvard University, Cambridge MA 02138, USA. Email: king@harvard.edu, URL: https://gking.harvard.edu/
Giuseppe Porro
Affiliation:
Department of Law, Economics and Culture, University of Insubria, Via S.Abbondio 12, I-22100 Como, Italy. Email: giuseppe.porro@uninsubria.it
*
*Email: stefano.iacus@unimi.it
Get access Rights & Permissions [Opens in a new window]

Abstract

Researchers who generate data often optimize efficiency and robustness by choosing stratified over simple random sampling designs. Yet, all theories of inference proposed to justify matching methods are based on simple random sampling. This is all the more troubling because, although these theories require exact matching, most matching applications resort to some form of ex post stratification (on a propensity score, distance metric, or the covariates) to find approximate matches, thus nullifying the statistical properties these theories are designed to ensure. Fortunately, the type of sampling used in a theory of inference is an axiom, rather than an assumption vulnerable to being proven wrong, and so we can replace simple with stratified sampling, so long as we can show, as we do here, that the implications of the theory are coherent and remain true. Properties of estimators based on this theory are much easier to understand and can be satisfied without the unattractive properties of existing theories, such as assumptions hidden in data analyses rather than stated up front, asymptotics, unfamiliar estimators, and complex variance calculations. Our theory of inference makes it possible for researchers to treat matching as a simple form of preprocessing to reduce model dependence, after which all the familiar inferential techniques and uncertainty calculations can be applied. This theory also allows binary, multicategory, and continuous treatment variables from the outset and straightforward extensions for imperfect treatment assignment and different versions of treatments.

Keywords

causal inferencematchingobservational studiesmultiple treatmentsstratification
Type
Articles
Information
Political Analysis , Volume 27 , Issue 1 , January 2019 , pp. 46 - 68
Copyright
Copyright © The Author(s) 2018. Published by Cambridge University Press on behalf of the Society for Political Methodology. 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Authors’ note: Our thanks to Alberto Abadie, Adam Glynn, Kosuke Imai, and Molly Roberts for helpful comments on an earlier draft. The replication code can be found in Iacus (2018).

Contributing Editor: Jonathan N. Katz

References

Abadie, Alberto, and Imbens, Guido W.. 2002. Simple and bias-corrected matching estimators for average treatment effects. NBER Technical Working Paper 283.Google Scholar
Abadie, Alberto, and Imbens, Guido W.. 2006. Large sample properties of matching estimators for average treatment effects. Econometrica 74(1):235–267.Google Scholar
Abadie, Alberto, and Imbens, Guido W.. 2011. Bias-corrected matching estimators for average treatment effects. Journal of Business & Economic Statistics 29(1):1–11.Google Scholar
Abadie, Alberto, and Imbens, Guido W.. 2012. A martingale representation for matching estimators. Journal of the American Statistical Association 107(498):833–843.Google Scholar
Angrist, Joshua D., Imbens, Guido W., and Rubin, Donald B.. 1996. Identification of causal effects using instrumental variables (with discussion). Journal of the American Statistical Association 91(434):444–455.Google Scholar
Berkson, Joseph. 1950. Are there two regressions? Journal of the American Statistical Association 45(250):164–180.Google Scholar
Cochran, William G. 1968. The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics 24:295–313.Google Scholar
Cochran, William G., and Rubin, Donald B.. 1973. Controlling bias in observational studies: A review. Sankhya: The Indian Journal of Statistics, Series A 35, Part 4:417–466.Google Scholar
Cox, David R. 1958. Planning of experiments . New York: John Wiley.Google Scholar
Crump, Richard K., Hotz, V. Joseph, Imbens, Guido W., and Mitnik, Oscar. 2009. Dealing with limited overlap in estimation of average treatment effects. Biometrika 96(1):187.Google Scholar
De Crescenzo, Antonio. 1999. A Probabilistic analogue of the mean value theorem and its applications to reliability theory. Journal of Applied Probability 36:706–719.Google Scholar
Dehejia, Rajeev H., and Wahba, Sadek. 1999. Causal effects in nonexperimental studies: Re-evaluating the evaluation of training programs. Journal of the American Statistical Association 94(448):1053–1062.Google Scholar
Ding, Peng. 2016. A paradox from randomization-based causal inference. Preprint, arXiv:1402.0142.Google Scholar
Heitjan, D. F., and Rubin, Donald B.. 1991. Ignorability and coarse data. The Annals of Statistics 19(4):2244–2253.Google Scholar
Ho, Daniel E., Imai, Kosuke, King, Gary, and Stuart, Elizabeth A.. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15(3):199–236.Google Scholar
Holland, Paul W. 1986. Statistics and causal inference. Journal of the American Statistical Association 81:945–960.Google Scholar
Hyslop, Dean R., and Imbens, Guido W.. 2001. Bias from classical and other forms of measurement error. Journal of Business and Economic Statistics 19(4):475–481.Google Scholar
Iacus, Stefano M., King, Gary, and Porro, Giuseppe. 2011. Multivariate matching methods that are monotonic imbalance bounding. Journal of the American Statistical Association 106(493):345–361.Google Scholar
Iacus, Stefano. 2018. Replication script for Iacus, King, Porro (2018), “A Theory of Statistical Inference for Matching Methods in Causal Research”. https://doi.org/10.7910/DVN/AOY452, Harvard Dataverse, V1, UNF:6:OEbDh6lIbV89a2sMhvelCQ==.Google Scholar
Imai, Kosuke. 2008. Variance identification and efficiency analysis in randomized experiments under the matched-pair design. Statistics in Medicine 27(24):4857.Google Scholar
Imai, Kosuke, and van Dyk, David A.. 2004. Causal inference with general treatment treatment regimes: Generalizing the propensity score. Journal of the American Statistical Association 99(467):854–866.Google Scholar
Imai, Kosuke, King, Gary, and Nall, Clayton. 2009. The essential role of pair matching in cluster-randomized experiments, with application to the Mexican universal health insurance evaluation. Statistical Science 24(1):29–53.Google Scholar
Imbens, Guido W. 2000. The role of the propensity score in estimating the dose-response functions. Biometrika 87(3):706–710.Google Scholar
Imbens, Guido W. 2004. Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and Statistics 86(1):4–29.Google Scholar
Imbens, Guido W., and Wooldridge, J. M.. 2009. Recent developments in the econometrics of program evaluation. Journal of Economic Literature 47(1):5–86.Google Scholar
King, Gary, Lucas, Christopher, and Nielsen, Richard A.. 2017. The balance-sample size frontier in matching methods for causal inference. American Journal of Political Science 61(2):473–489.Google Scholar
King, Gary, and Nielsen, Richard A.. 2017. Why propensity scores should not be used for matching. Working Paper. URL: http://j.mp/PSMnot.Google Scholar
King, Gary, and Zeng, Langche. 2006. The dangers of extreme counterfactuals. Political Analysis 14(2):131–159.Google Scholar
Lalonde, Robert. 1986. Evaluating the econometric evaluations of training programs. American Economic Review 76:604–620.Google Scholar
Lechner, Michael. 2001. Identification and estimation of causal effects of multiple treatments under the conditional independence assumption. In Econometric evaluation of labour market policies , ed. Lechner, M. and Pfeiffer, F.. Heidelberg: Physica, pp. 43–58.Google Scholar
Lin, Winston. 2013. Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique. The Annals of Applied Statistics 7(1):295–318.Google Scholar
Mielke, P. W., and Berry, K. J.. 2007. Permutation methods: A distance function approach . New York: Springer.Google Scholar
Miratrix, Luke W., Sekhon, Jasjeet S., and Yu, Bin. 2013. Adjusting treatment effect estimates by post-stratification in randomized experiments. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 75(2):369–396.Google Scholar
Morgan, Stephen L., and Winship, Christopher. 2014. Counterfactuals and causal inference: Methods and principles for social research , 2nd edn Cambridge: Cambridge University Press.Google Scholar
Neyman, J. 1935. Statistical problems in agricultural experimentation. Journal of the Royal Statistical Society II 2:107–154.Google Scholar
Robins, James M. 1986. A new approach to causal inference in mortality studies with sustained exposure period - application to control of the healthy worker survivor effect. Mathematical Modelling 7:1393–1512.Google Scholar
Rosenbaum, Paul R. 1988. Permutation tests for matched pairs with adjustments for covariates. Applied Statistics 37(3):401–411.Google Scholar
Rosenbaum, Paul R., and Rubin, Donald B.. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70:41–55.Google Scholar
Rubin, Donald B. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 6:688–701.Google Scholar
Rubin, Donald B. 1977. Assignment to treatment group on the basis of a covariate. Journal of Educational Statistics 2(1–26):1.Google Scholar
Rubin, Donald B. 1980. Comments on “Randomization analysis of experimental data: The Fisher randomization test,” by D. Basu. Journal of the American Statistical Association 75:591–593.Google Scholar
Rubin, Donald B. 1990. On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Comment: Neyman (1923) and causal inference in experiments and observational studies. Statistical Science 5(4):472–480.Google Scholar
Rubin, Donald B. 1991. Practical implications of modes of statistical inference for causal effects and the critical role of the assignment mechanism. Biometrics 47:1213–1234.Google Scholar
Rubin, Donald B. 2010. On the limitations of comparative effectiveness research. Statistics in Medicine 29(19):1991–1995.Google Scholar
Smith, Jeffrey A., and Todd, Petra E.. 2005. Does matching overcome LaLonde’s critique of nonexperimental estimators? Journal of Econometrics 125(1–2):305–353.Google Scholar
Stuart, Elizabeth A. 2010. Matching methods for causal inference: A review and a look forward. Statistical Science 25(1):1–21.Google Scholar
VanderWeele, Tyler J., and Hernan, Miguel A.. 2012. Causal inference under multiple versions of treatment. Journal of Causal Inference 1:1–20.Google Scholar