Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-07T18:03:21.302Z Has data issue: false hasContentIssue false
  • English
  • Français

A Two-Step Bayesian Approach for Propensity Score Analysis: Simulations and Case Study

Published online by Cambridge University Press:  01 January 2025

David Kaplan  and
Jianshen Chen
David Kaplan*
Affiliation:
Department of Educational Psychology, University of Wisconsin–Madison
Jianshen Chen
Affiliation:
Department of Educational Psychology, University of Wisconsin–Madison
*
Requests for reprints should be sent to David Kaplan, Department of Educational Psychology, University of Wisconsin–Madison, 1025 W. Johnson St., Madison, WI 53706, USA. E-mail: dkaplan@education.wisc.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

A two-step Bayesian propensity score approach is introduced that incorporates prior information in the propensity score equation and outcome equation without the problems associated with simultaneous Bayesian propensity score approaches. The corresponding variance estimators are also provided. The two-step Bayesian propensity score is provided for three methods of implementation: propensity score stratification, weighting, and optimal full matching. Three simulation studies and one case study are presented to elaborate the proposed two-step Bayesian propensity score approach. Results of the simulation studies reveal that greater precision in the propensity score equation yields better recovery of the frequentist-based treatment effect. A slight advantage is shown for the Bayesian approach in small samples. Results also reveal that greater precision around the wrong treatment effect can lead to seriously distorted results. However, greater precision around the correct treatment effect parameter yields quite good results, with slight improvement seen with greater precision in the propensity score equation. A comparison of coverage rates for the conventional frequentist approach and proposed Bayesian approach is also provided. The case study reveals that credible intervals are wider than frequentist confidence intervals when priors are non-informative.

Keywords

propensity score analysisBayesian inference
Type
Original Paper
Information
Psychometrika , Volume 77 , Issue 3 , July 2012 , pp. 581 - 609
Copyright
Copyright © 2012 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abadie, A., Imbens, G. (2006). Large sample properties of matching estimators for average treatment effects. Econometrica, 74, 235–267CrossRefGoogle Scholar
Abadie, A., Imbens, G.W. (2008). On the failure of the bootstrap for matching estimators. Econometrica, 76, 1537–1558Google Scholar
Abadie, A., & Imbens, G.W. (2009). Matching on the estimated propensity score (NBER Working Paper 15301). CrossRefGoogle Scholar
Abbas, A.E., Budescu, D.V., Gu, Y. (2010). Assessing joint distributions with isoprobability countours. Management Science, 56, 997–1011CrossRefGoogle Scholar
Abbas, A.E., Budescu, D.V., Yu, H.T., Haggerty, R. (2008). A comparison of two probability encoding methods: fixed probability vs. fixed variable values. Decision Analysis, 5, 190–202CrossRefGoogle Scholar
An, W. (2010). Bayesian propensity score estimators: incorporating uncertainties in propensity scores into causal inference. Sociological Methodology, 40, 151–189CrossRefGoogle Scholar
Austin, P.C., Mamdani, M.M. (2006). A comparison of propensity score methods: a case-study estimating the effectiveness of post-AMI statin use. Statistics in Medicine, 25, 2084–2106CrossRefGoogle ScholarPubMed
Benjamin, D.J. (2003). Does 401(k) eligibility increase saving? Evidence from propensity score subclassification. Journal of Public Economics, 87, 1259–1290CrossRefGoogle Scholar
Cochran, W.G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24, 295–313CrossRefGoogle ScholarPubMed
Dawid, A.P. (1982). The well-calibrated Bayesian. Journal of the American Statistical Association, 77, 605–610CrossRefGoogle Scholar
Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B. (2003). Bayesian data analysis, (2rd ed.). London: Chapman and HallCrossRefGoogle Scholar
Geman, S., Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741CrossRefGoogle ScholarPubMed
Guo, S., Fraser, M.W. (2010). Propensity score analysis: statistical methods and applications, Thousand Oaks: SageGoogle Scholar
Hansen, B.B. (2004). Full matching in an observational study of coaching for the SAT. Journal of the American Statistical Association, 99, 609–618CrossRefGoogle Scholar
Hansen, B.B., Klopfer, S.O. (2006). Optimal full matching and related designs via network flow. Journal of Computational and Graphical Statistics, 15, 609–627CrossRefGoogle Scholar
Heckman, J.J. (2005). The scientific model of causality. In Stolzenberg, R.M. Sociological methodology, Boston: Blackwell Publishing 197Google Scholar
Hirano, K., Imbens, G.W. (2001). Estimation of causal effects using propensity score weighting: an application to data on right heart catheterization. Health Services and Outcomes Research Methodology, 2, 259–278CrossRefGoogle Scholar
Hirano, K., Imbens, G.W., Ridder, G. (2003). Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71, 1169–1189CrossRefGoogle Scholar
Holland, P.W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945–960CrossRefGoogle Scholar
Horvitz, D.G., Thompson, D.J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47, 663–685CrossRefGoogle Scholar
Hoshino, T. (2008). A Bayesian propensity score adjustment for latent variable modeling and MCMC algorithm. Computational Statistics & Data Analysis, 52, 1413–1429CrossRefGoogle Scholar
Larsen, M.D. (1999). An analysis of survey data on smoking using propensity scores. Sankya. The Indian Journal of Statistics, 61, 91–105Google Scholar
Lechner, M. (2002). Some practical issues in the evaluation of heterogeneous labour market programmes by matching methods. Journal of the Royal Statistical Society. Series A. Statistics in Society, 165, 59–82CrossRefGoogle Scholar
Lunceford, J.K., Davidian, M. (2004). Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Statistics in Medicine, 23, 2937–2960CrossRefGoogle ScholarPubMed
Martin, A.D., Quinn, K.M., & Park, J.H. (2010, May 10). Markov chain Monte Carlo (MCMC) package. http://mcmcpack.wustl.edu/. Google Scholar
McCandless, L.C., Gustafson, P., Austin, P.C. (2009). Bayesian propensity score analysis for observational data. Statistics in Medicine, 28, 94–112CrossRefGoogle ScholarPubMed
NCES (2001). Early childhood longitudinal study: kindergarten class of 1998–99: base year public-use data files user’s manual (Tech. Rep. No. NCES 2001-029). U.S. Government Printing Office. Google Scholar
Neyman, J.S. (1923). Statistical problems in agriculture experiments. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 2, 107–180CrossRefGoogle Scholar
O’Hagan, A., Buck, C.E., Daneshkhah, A., Eiser, J.R., Garthwaite, P.H., Jenkinson, D.J. et al. (2006). Uncertain judgements: eliciting experts’ probabilities, West Sussex: WileyCrossRefGoogle Scholar
Perkins, S.M., Tu, W., Underhill, M.G., Zhou, X.H., Murray, M.D. (2000). The use of propensity scores in pharmacoepidemiologic research. Pharmacoepidemiology and Drug Safety, 9, 93–1013.0.CO;2-I>CrossRefGoogle ScholarPubMed
R Development Core Team (2011). R: a language and environment for statistical computing (Computer software manual). Vienna, Austria. Available from http://www.R-project.org (ISBN 3-900051-07-0). Google Scholar
Rässler, S. (2002). Statistical matching: a frequentist theory, practical applications, and alternative Bayesian approaches, New York: SpringerCrossRefGoogle Scholar
Rosenbaum, P.R. (1987). Model-based direct adjustment. Journal of the American Statistical Association, 82, 387–394CrossRefGoogle Scholar
Rosenbaum, P.R. (1989). Optimal matching for observational studies. Journal of the American Statistical Association, 84, 1024–1032CrossRefGoogle Scholar
Rosenbaum, P.R. (2002). Observational studies, (2rd ed.). New York: SpringerCrossRefGoogle Scholar
Rosenbaum, P.R., Rubin, D.B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55CrossRefGoogle Scholar
Rosenbaum, P.R., Rubin, D.B. (1984). Reducing bias in observational studies using sub-classification on the propensity score. Journal of the American Statistical Association, 79, 516–524CrossRefGoogle Scholar
Rosenbaum, P.R., Rubin, D.B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate a propensity score. American Statistician, 39, 33–38CrossRefGoogle Scholar
Rubin, D.B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688–701CrossRefGoogle Scholar
Rubin, D.B. (1985). The use of propensity scores in applied Bayesian inference. Bayesian Statistics, 2, 463–472Google Scholar
Rubin, D.B. (2006). Matched sampling for causal effects, Cambridge: Cambridge University PressCrossRefGoogle Scholar
Rubin, D.B., Thomas, N. (1992). Affinely invariant matching methods with ellipsoidal distributions. Annals of Statistics, 20, 1079–1093CrossRefGoogle Scholar
Rubin, D.B., Thomas, N. (1992). Characterizing the effect of matching using linear propensity score methods with normal distributions. Biometrika, 79, 797–809CrossRefGoogle Scholar
Rubin, D.B., Thomas, N. (1996). Matching using estimated propensity scores. Biometrics, 52, 249–264CrossRefGoogle ScholarPubMed
Spiegelhalter, D.J., Best, N.G., Carlin, B.P., van der Linde, A. (2002). Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society. Series B. Statistical Methodology, 64, 583–639CrossRefGoogle Scholar
Steiner, P.M., & Cook, D. (in press). Matching and propensity scores. In T. Little (Ed.), Oxford handbook of quantitative methods. Oxford: Oxford University Press. Google Scholar
Steiner, P.M., Cook, T.D., Shadish, W.R. (2011). On the importance of reliable covariate measurement in selection bias adjustments using propensity scores. Journal of Educational and Behavioral Statistics, 36, 213–236CrossRefGoogle Scholar
Steiner, P.M., Cook, T.D., Shadish, W.R., Clark, M.H. (2010). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods, 15, 250–267CrossRefGoogle ScholarPubMed
Thoemmes, F.J., Kim, E.S. (2011). A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research, 46, 90–118CrossRefGoogle ScholarPubMed
van Buuren, S., Groothuis-Oudshoorn, K. (2011). MICE: multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 167 Available from http://www.jstatsoft.org/v45/i03/Google Scholar
Yuan, Y., MacKinnon, D.P. (2009). Bayesian mediation analysis. Psychological Methods, 14, 301–322CrossRefGoogle ScholarPubMed
Zanutto, E.L., Lu, B., Hornik, R. (2005). Using propensity score subclassification for multiple treatment doses to evaluate a national anti-drug media campaign. Journal of Educational and Behavioral Statistics, 30, 59–73CrossRefGoogle Scholar