Hostname: page-component-5f745c7db-szhh2 Total loading time: 0 Render date: 2025-01-06T07:18:53.321Z Has data issue: true hasContentIssue false
  • English
  • Français

Compensation and Amplification of Attenuation Bias in Causal Effect Estimates

Published online by Cambridge University Press:  01 January 2025

Marie-Ann Sengewald Open the ORCID record for Marie-Ann Sengewald [Opens in a new window]  and
Steffi Pohl
Marie-Ann Sengewald*
Affiliation:
Otto-Friedrich-Universität Bamberg
Steffi Pohl
Affiliation:
Freie Universität Berlin
*
Correspondence should be made to Marie-Ann Sengewald, Department of Psychological Methods of Educational Research, Otto-Friedrich-Universität Bamberg, Wilhelmsplatz 3, 96047 Bamberg, Germany. Email: Marie-Ann.Sengewald@uni-bamberg.de
Get access Rights & Permissions [Opens in a new window]

Abstract

Covariate-adjusted treatment effects are commonly estimated in non-randomized studies. It has been shown that measurement error in covariates can bias treatment effect estimates when not appropriately accounted for. So far, these delineations primarily assumed a true data generating model that included just one single covariate. It is, however, more plausible that the true model consists of more than one covariate. We evaluate when a further covariate may reduce bias due to measurement error in another covariate and in which cases it is not recommended to include a further covariate. We analytically derive the amount of bias related to the fallible covariate’s reliability and systematically disentangle bias compensation and amplification due to an additional covariate. With a fallible covariate, it is not always beneficial to include an additional covariate for adjustment, as the additional covariate can extensively increase the bias. The mechanisms for an increased bias due to an additional covariate can be complex, even in a simple setting of just two covariates. A high reliability of the fallible covariate or a high correlation between the covariates cannot in general prevent from substantial bias. We show distorting effects of a fallible covariate in an empirical example and discuss adjustment for latent covariates as a possible solution.

Keywords

causal inferencemeasurement errorbias amplificationANCOVApropensity scores
Type
Original Paper
Information
Psychometrika , Volume 84 , Issue 2 , June 2019 , pp. 589 - 610
Copyright
Copyright © 2019 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11336-019-09665-6) contains supplementary material, which is available to authorized users.

We thank Peter M. Steiner and Rolf Steyer for valuable discussions as well as Renate Soellner and Jens Eisermann who provided the empirical data. Furthermore, we thank the anonymous reviewers for insightful comments and constructive suggestions.

References

Aiken, L. S., West, S. G. (1991). Multiple regression: Testing and interpreting interactions, Newbury Park: Sage.Google Scholar
Clarke, K. A. (2005). The phantom menace: Omitted variable bias in econometric research. Conflict. Management and Peace Science, 22, 341–352.Google Scholar
Cohen, J., Cohen, P., West, S. G., Aiken, L. S. (2003). Applied multiple regression/correlation analyses for the behavioral sciences, Hillsdale, NJ: Erlbaum.Google Scholar
Cole, S. R., Platt, R. W., Schisterman, E. F., Chu, H., Westreich, D., Richardson, D., Poole, C. (2010). Illustrating bias due to conditioning on a collider. International Journal of Epidemiology, 39(2), 417–420.CrossRefGoogle ScholarPubMed
Cook, T. D., Steiner, P. M., Pohl, S. (2009). How bias reduction is affected by covariate choice, unreliability, and mode of data analysis: Results from two types of within-study comparisons. Multivariate Behavioral Research, 44(6), 828–847.CrossRefGoogle ScholarPubMed
Ding, P., Miratrix, L. W. (2015). To adjust or not to adjust? Sensitivity analysis of M-bias and butterfly-bias. Journal of Causal Inference, 3(1), 41–57.CrossRefGoogle Scholar
Duncan, O. D. (1975). Introduction to structural equation models, New York: Academic.Google Scholar
Fritz, M. S., Kenny, D. A., MacKinnon, D. P. (2016). The combined effects of measurement error and omitted confounding in the single-mediator model. Multivariate Behavioral Research, 51(5), 681–697.CrossRefGoogle ScholarPubMed
Greenland, S. (2003). Quantifying biases in causal models: Classical confounding vs. collider-stratification bias. Epidemiology, 14(3), 300–306.CrossRefGoogle ScholarPubMed
Hong, H., Rudolph, K. E., Stuart, E. A. (2017). Bayesian approach for addressing differential covariate measurement error in propensity score methods. Psychometrika, 82(4), 1078–1096.CrossRefGoogle ScholarPubMed
Kenny, D. A. (1979). Correlation and causation, New York: Wiley.Google Scholar
Kuroki, M., Pearl, J. (2014). Measurement bias and effect restoration in causal inference. Biometrika, 101(2), 423–437.CrossRefGoogle Scholar
Lockwood, J., McCaffrey, D. (2014). Correcting for test score measurement error in ANCOVA models for estimating treatment effects. Journal of Educational and Behavioral Statistics, 39(1), 22–51.CrossRefGoogle Scholar
Lockwood, J., McCaffrey, D. (2015). Simulation-extrapolation for estimating means and causal effects with mismeasured covariates. Observational Studies, 1, 241–290.CrossRefGoogle Scholar
Lockwood, J., McCaffrey, D. (2016). Matching and weighting with functions of error-prone covariates for causal inference. Journal of the American Statistical Association, 111, 1831–1839.CrossRefGoogle Scholar
Lockwood, J., & McCaffrey, D. (2018). Hidden information, wobbly surfaces, and test score measurement error: Unpacking student-teacher selection mechanisms. Manuscript under review.Google Scholar
Lockwood, J., McCaffrey, D. (2017). Simulation-extrapolation with latent heteroskedastic error variance. Psychometrika, 82(3), 717–736.CrossRefGoogle Scholar
Mayer, A., Dietzfelbinger, L., Rosseel, Y. (2016). The EffectLiteR approach for analyzing average and conditional effects. Multivariate Behavioral Research, 51 2–3374–91.CrossRefGoogle ScholarPubMed
Myers, J. A., Rassen, J. A., Gagne, J. J., Huybrechts, K. F., Schneeweiss, S., Rothman, K. J., Joffe, M. M., Glynn, R. J. (2011). Effects of adjusting for instrumental variables on bias and precision of effect estimates. American Journal of Epidemiology, 174(11), 1213–1222.CrossRefGoogle ScholarPubMed
McCaffrey, D., Lockwood, J., Setodji, C. (2013). Inverse probability weighting with error-prone covariates. Biometrika, 100(3), 671–680.CrossRefGoogle ScholarPubMed
Pearl, J. (2009). Causality: Models, reasoning, and inference, 2Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Pearl, J. (2011). Understanding bias amplification (Invited commentary). American Journal of Epidemiology, 174(11), 1223–1227.CrossRefGoogle ScholarPubMed
Pearl, J. (2013). Linear models: A useful "microscope" for causal analysis. Journal of Causal Inference, 1(1), 155–17.CrossRefGoogle Scholar
Pearl, J. (2014). A short note on the virtues of graphical tools. Technical report. Los Angeles, CA: Computer Science Department, University of California Los Angeles.Google Scholar
Pohl, S., Sengewald, M.-A., Steyer, R. (2016). Adjustment when covariates are fallible. In Wiedermann, W., Eye, A. V. (Eds), Statistics and causality: Methods for applied empirical research, Hoboken: Wiley.Google Scholar
Pohl, S., Steiner, P. M., Eisermann, J., Soellner, R., Cook, T. D. (2009). Unbiased causal inference from an observational study: Results of a within-study comparison. Education Evaluation and Policy Analysis, 31(4), 463–479.CrossRefGoogle Scholar
Raykov, T. (2012). Propensity score analysis with fallible covariates: A note on a latent variable modeling approach. Educational and Psychological Measurement, 72, 715–733.CrossRefGoogle Scholar
Rosseel, Y. (2012). Lavaan: an R package for structural equation modeling. Journal of Statistical Software, 48, 1–36. Retrieved from http://www.jstatsoft.org/v48/i02/.CrossRefGoogle Scholar
Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100, 322–331.CrossRefGoogle Scholar
Sengewald, M.-A., Steiner, P. M., & Pohl, S. (2018). When does measurement error in covariates impact causal effect estimates? Analytical derivations of different scenarios and an empirical illustration. British Journal of Mathematical and Statistical Psychology. https://doi.org/10.1111/bmsp.12146.CrossRefGoogle Scholar
Shadish, W. R., Cook, T. D., Campbell, D. T. (2002). Experimental and quasi experimental designs for generalized causal inference, Boston: Houghton Mifflin.Google Scholar
Skrondal, A., Laake, P. (2001). Regression among factor scores. Psychometrika, 66, 563–575.CrossRefGoogle Scholar
Steiner, P. M., Cook, T. D., Li, W., Clark, M. H. (2015). Bias reduction in quasi-experiments with little selection theory but many covariates. Journal of Research on Educational Effectiveness, 8(4), 552–576.CrossRefGoogle Scholar
Steiner, P. M., Cook, T. D., Shadish, W. R. (2011). On the importance of reliable covariate measurement in selection bias adjustments using propensity scores. Journal of Educational and Behavioral Statistics, 36(2), 213–236.CrossRefGoogle Scholar
Steiner, P. M., & Kim, Y. (2016). The mechanics of omitted variable bias: Bias amplification and cancellation of offsetting biases.Journal of Causal Inference. https://doi.org/10.1515/jci-2016-0009.CrossRefGoogle Scholar
Steyer, R., Mayer, A., Fiege, C. (2014). Causal inference on total, direct, and indirect effects. In Michalos, A. C. (Eds), Encyclopedia of quality of life and well-being research, Dordrecht: Springer 606–631.CrossRefGoogle Scholar
Thoemmes, F., Rose, N. (2014). A cautious note on auxiliary variables that can increase bias in missing data problems. Multivariate Behavioral Research, 49, 443–459.CrossRefGoogle ScholarPubMed
Wright, S. (1922). Coefficients of inbreeding and relationship. The American Naturalist, 56, 330–338.CrossRefGoogle Scholar
Yi, G., Ma, Y., Carroll, R. (2012). A functional generalized method of moments approach for longitudinal studies with missing responses and covariate measurement error. Biometrika, 99(1), 151–165.CrossRefGoogle ScholarPubMed
Supplementary material: File

Sengewald and Pohl Supplementary material

Sengewald and Pohl Supplementary material 1
Download Sengewald and Pohl Supplementary material(File)
File 3 MB
Supplementary material: File

Sengewald and Pohl Supplementary material

Sengewald and Pohl Supplementary material 2
Download Sengewald and Pohl Supplementary material(File)
File 374.7 KB