Hostname: page-component-5f745c7db-6bmsf Total loading time: 0 Render date: 2025-01-06T21:23:31.112Z Has data issue: true hasContentIssue false
  • English
  • Français

Simulation-Extrapolation with Latent Heteroskedastic Error Variance

Published online by Cambridge University Press:  01 January 2025

J. R. Lockwood  and
Daniel F. McCaffrey
J. R. Lockwood*
Affiliation:
Educational Testing Service
Daniel F. McCaffrey
Affiliation:
Educational Testing Service
*
ajrlockwood@ets.org
Get access Rights & Permissions [Opens in a new window]

Abstract

This article considers the application of the simulation-extrapolation (SIMEX) method for measurement error correction when the error variance is a function of the latent variable being measured. Heteroskedasticity of this form arises in educational and psychological applications with ability estimates from item response theory models. We conclude that there is no simple solution for applying SIMEX that generally will yield consistent estimators in this setting. However, we demonstrate that several approximate SIMEX methods can provide useful estimators, leading to recommendations for analysts dealing with this form of error in settings where SIMEX may be the most practical option.

Keywords

achievement test scoresmeasurement erroritem response theorycovariate adjustmentnonlinear models
Type
Original paper
Information
Psychometrika , Volume 82 , Issue 3 , September 2017 , pp. 717 - 736
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s11336-017-9556-y) contains supplementary material, which is available to authorized users.

References

Akram, K., Erickson, F., & Meyer, R. (2013). Issues in the estimation of student growth percentiles. Paper presented at the annual meeting of the Association for Education Finance and Policy, New Orleans, LA..Google Scholar
Battauz, M., & Bellio, R.. (2011). Structural modeling of measurement error in generalized linear models with Rasch measures as covariates. Psychometrika, 76(1), 40–56. doi:10.1007/s11336-010-9195-z.CrossRefGoogle Scholar
Battauz, M., Bellio, R., & Gori, E.. (2008). Reducing measurement error in student achievement estimation. Psychometrika, 73(2), 289–302. doi:10.1007/s11336-007-9050-z.CrossRefGoogle Scholar
Betebenner, D.. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4), 42–51. doi:10.1111/j.1745-3992.2009.00161.x.CrossRefGoogle Scholar
Bollen, K.Structural equations with latent variables 1989 New York: Wileydoi:10.1002/9781118619179.CrossRefGoogle Scholar
Buonaccorsi, J.Measurement error: Models, methods, and applications 2010 Boca Raton, FL: Chapman and Hall/CRC Interdisciplinary Statisticsdoi:10.1201/9781420066586.CrossRefGoogle Scholar
Carroll, R., Ruppert, D., Stefanski, L., & Crainiceanu, C.Measurement error in nonlinear models: A modern perspective 2006 2London: Chapman and Halldoi:10.1201/9781420010138.CrossRefGoogle Scholar
Carroll, R., & Wang, Y.. (2008). Nonparametric variance estimation in the analysis of microarray data: A measurement error approach. Biometrika, 95(2), 437–449. doi:10.1093/biomet/asn017 2670068.CrossRefGoogle Scholar
Cook, J., & Stefanski, L.. (1994). Simulation-extrapolation estimation in parametric measurement error models. Journal of the American Statistical Association, 89(428), 1314–1328. doi:10.1080/01621459.1994.10476871.CrossRefGoogle Scholar
Dawid, A. P.. (1979). Conditional independence in statistical theory. Journal of the Royal Statistical Society, Series B, 41(1), 1–31.CrossRefGoogle Scholar
Devanarayan, V., & Stefanski, L.. (2002). Empirical simulation extrapolation for measurement error models with replicate measurements. Statistics and Probability Letters, 59, 219–225. doi:10.1016/S0167-7152(02)00098-6.CrossRefGoogle Scholar
Fox, J-P, & Glas, C.. (2003). Bayesian modeling of measurement error in predictor variables using item response theory. Psychometrika, 68(2), 169–191. doi:10.1007/BF02294796.CrossRefGoogle Scholar
Fuller, W.Measurement error models 2006 2New York: Wiley.Google Scholar
Junker, B., Schofield, L., & Taylor, L.. (2012). The use of cognitive ability measures as explanatory variables in regression analysis. IZA Journal of Labor Economics, 1(4), 1–19.CrossRefGoogle ScholarPubMed
Kolen, M., & Brennan, R.Test equating, scaling, and linking: Methods and practice 2004 2New York: Springerdoi:10.1007/978-1-4757-4310-4.CrossRefGoogle Scholar
Laird, N.. (1978). Nonparametric maximum likelihood estimation of a mixing distribution. Journal of the American Statistical Association, 73(364), 215–232. doi:10.1080/01621459.1978.10480103.CrossRefGoogle Scholar
Lockwood, J. R., & Castellano, K. E.. (2015). Alternative statistical frameworks for student growth percentile estimation. Statistics and Public Policy, 2(1), 1–8. doi:10.1080/2330443X.2014.962718.CrossRefGoogle Scholar
Lockwood, J. R., & Castellano, K. E. (2016). Estimating true student growth percentile distributions using latent regression multidimensional IRT models. Educational and Psychological Measurement. doi:10.1177/0013164416659686.CrossRefGoogle Scholar
Lockwood, J. R., & McCaffrey, D. F.. (2014). Correcting for test score measurement error in ANCOVA models for estimating treatment effects. Journal of Educational and Behavioral Statistics, 39(1), 22–52. doi:10.3102/1076998613509405.CrossRefGoogle Scholar
Lockwood, J. R., McCaffrey, D. F., & Lissitz, R.. (2014). Should nonlinear functions of test scores be used as covariates in a regression model?. Value-added modeling and growth modeling with particular application to teacher and school effectiveness. Charlotte, NC: Information Age Publishing.Google Scholar
Lockwood, J. R., & McCaffrey, D. F.. (2015). Simulation-extrapolation for estimating means and causal effects with mismeasured covariates. Observational Studies, 1, 241–290.CrossRefGoogle Scholar
Lord, F.Applications of item response theory to practical testing problems 1980 Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Lord, F.. (1984). Standard errors of measurement at different ability levels. Journal of Educational Measurement, 21(3), 239–243. doi:10.1111/j.1745-3984.1984.tb01031.x.CrossRefGoogle Scholar
Maryland State Department of Education. (2014). Maryland high school assessment and modified high school assessment 2013 technical report. Princeton, NJ: Educational Testing Service..Google Scholar
McCaffrey, D. F., Castellano, K. E., & Lockwood, J. R.. (2015). The impact of measurement error on the accuracy of individual and aggregate SGP. Educational Measurement: Issues and Practice, 34(1), 15–21. doi:10.1111/emip.12062.CrossRefGoogle Scholar
McCaffrey, D. F., Lockwood, J. R., & Setodji, C.. (2013). Inverse probability weighting with error-prone covariates. Biometrika, 100(3), 671–680. doi:10.1093/biomet/ast022 4006991.CrossRefGoogle ScholarPubMed
Monroe, S., & Cai, L.. (2015). Examining the reliability of student growth percentiles using multidimensional IRT. Educational Measurement: Issues and Practice, 34(4), 21–30. doi:10.1111/emip.12092.CrossRefGoogle Scholar
Novick, S., & Stefanski, L.. (2002). Corrected score estimation via complex variable simulation extrapolation. Journal of the American Statistical Association, 97(458), 472–481. doi:10.1198/016214502760047005.CrossRefGoogle Scholar
Pearson. (2014). New York State Testing Program 2014: English Language Arts and Mathematics Grades 3–8. http://www.p12.nysed.gov/assessment/reports/2014/gr38cc-tr14.Google Scholar
R Core Team. (2016). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing..Google Scholar
Rabe-Hesketh, S., Pickles, A., & Skrondal, A.. (2003). Correcting for covariate measurement error in logistic regression using nonparametric maximum likelihood estimation. Statistical Modelling, 3, 215–232. doi:10.1191/1471082X03st056oa.CrossRefGoogle Scholar
Shang, Y., Van Iwaarden, A., & Betebenner, D.. (2015). Covariate measurement error correction for student growth percentiles using the SIMEX method. Educational Measurement: Issues and Practice, 34(1), 4–14. doi:10.1111/emip.12058.CrossRefGoogle Scholar
Skrondal, A., & Rabe-Hesketh, S.Generalized latent variable modeling: Multilevel, longitudinal and structural equation models 2004 Boca Raton, FL: Chapman and Hall/CRC Pressdoi:10.1201/9780203489437.CrossRefGoogle Scholar
Stefanski, L., & Cook, J.. (1995). Simulation-extrapolation: The measurement error jackknife. Journal of the American Statistical Association, 90(432), 1247–1256. doi:10.1080/01621459.1995.10476629.CrossRefGoogle Scholar
Steiner, P., Cook, T., & Shadish, W.. (2011). On the importance of reliable covariate measurement in selection bias adjustments using propensity scores. Journal of Educational and Behavioral Statistics, 36(2), 213–236. doi:10.3102/1076998610375835.CrossRefGoogle Scholar
Stuart, E.. (2007). Estimating causal effects using school-level data sets. Educational Researcher, 36(4), 187–198. doi:10.3102/0013189X07303396.CrossRefGoogle Scholar
van der Linden, W., & Hambleton, R.Handbook of modern item response theory 1997 New York, NY: Springerdoi:10.1007/978-1-4757-2691-6.CrossRefGoogle Scholar
Wang, Y., Ma, Y., & Carroll, R.. (2009). Variance estimation in the analysis of microarray data. Journal of the Royal Statistical Society: Series B, 71(2), 425–445. doi:10.1111/j.1467-9868.2008.00690.x.CrossRefGoogle ScholarPubMed
Warm, T.. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54(3), 427–450. doi:10.1007/BF02294627.CrossRefGoogle Scholar
Yen, W. M.. (1984). Obtaining maximum likelihood trait estimates from number-correct scores for the three-parameter logistic model. Journal of Educational Measurement, 21(2), 93–111. doi:10.1111/j.1745-3984.1984.tb00223.x.CrossRefGoogle Scholar
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 1
Download Lockwood and McCaffrey supplementary material(File)
File 5.8 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 2
Download Lockwood and McCaffrey supplementary material(File)
File 15.3 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 3
Download Lockwood and McCaffrey supplementary material(File)
File 7.5 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 4
Download Lockwood and McCaffrey supplementary material(File)
File 19.5 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 5
Download Lockwood and McCaffrey supplementary material(File)
File 2.4 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 6
Download Lockwood and McCaffrey supplementary material(File)
File 5.3 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 7
Download Lockwood and McCaffrey supplementary material(File)
File 1.2 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 8
Download Lockwood and McCaffrey supplementary material(File)
File 17 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 9
Download Lockwood and McCaffrey supplementary material(File)
File 25.7 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 10
Download Lockwood and McCaffrey supplementary material(File)
File 17 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 11
Download Lockwood and McCaffrey supplementary material(File)
File 25.4 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 12
Download Lockwood and McCaffrey supplementary material(File)
File 2.4 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 13
Download Lockwood and McCaffrey supplementary material(File)
File 5.3 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 14
Download Lockwood and McCaffrey supplementary material(File)
File 1.2 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 15
Download Lockwood and McCaffrey supplementary material(File)
File 16 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 16
Download Lockwood and McCaffrey supplementary material(File)
File 23 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 17
Download Lockwood and McCaffrey supplementary material(File)
File 16 KB
Supplementary material: File

Lockwood and McCaffrey supplementary material

Lockwood and McCaffrey supplementary material 18
Download Lockwood and McCaffrey supplementary material(File)
File 22.8 KB