Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-01-07T17:33:28.876Z Has data issue: false hasContentIssue false
  • English
  • Français

Predictive Inference Using Latent Variables with Covariates

Published online by Cambridge University Press:  01 January 2025

Lynne Steuerle Schofield ,
Brian Junker ,
Lowell J. Taylor  and
Dan A. Black
Lynne Steuerle Schofield*
Affiliation:
Swarthmore College
Brian Junker
Affiliation:
Carnegie Mellon University
Lowell J. Taylor
Affiliation:
Carnegie Mellon University
Dan A. Black
Affiliation:
University of Chicago
*
Correspondence should be made to Lynne Steuerle Schofield, Department of Mathematics and Statistics, Swarthmore College, 500 College Avenue, Swarthmore, PA 19081, USA. E-mail: lschofi1@swarthmore.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Plausible values (PVs) are a standard multiple imputation tool for analysis of large education survey data, which measures latent proficiency variables. When latent proficiency is the dependent variable, we reconsider the standard institutionally generated PV methodology and find it applies with greater generality than shown previously. When latent proficiency is an independent variable, we show that the standard institutional PV methodology produces biased inference because the institutional conditioning model places restrictions on the form of the secondary analysts’ model. We offer an alternative approach that avoids these biases based on the mixed effects structural equations model of Schofield (Modeling measurement error when using cognitive test scores in social science research. Doctoral dissertation. Department of Statistics and Heinz College of Public Policy. Pittsburgh, PA: Carnegie Mellon University, 2008).

Keywords

latent variable analysisNAEPplausible value methodologymarginal estimation procedures
Type
Original Paper
Information
Psychometrika , Volume 80 , Issue 3 , September 2015 , pp. 727 - 747
Copyright
Copyright © 2014 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Allen, N.L., Carlson, J.E., & Zelenak, C.A. (1999). The NAEP 1996 Technical Report (NCES 99452). Washington, DC: National Center for Education Statistics.Google Scholar
Allen, N. L., Donoghue, J. R., & Schoeps, T. L., (2001). The NAEP 1998 Technical Report (NCES 2001–509). Washington, DC: National Center for Education Statistics. Retrieved from http://nces.ed.gov/nationsreportcard/pubs/main1998/2001509.asp.Google Scholar
Ayers, E., & Junker, B. (2008). IRT modeling of tutor performance to predict end-of-year exam scores. Educational and Psychological Measurement, 68, 972–987.CrossRefGoogle Scholar
Beaton, A. E., & Zwick, R., (1992). Overview of the National Assessment of Educational Progress. Journal of Educational Statistics, 17(2), 95–109. Special Issue: National Assessment of Educational Progress.Google Scholar
Billingsley, P. (1986). Probability and measure. New York, NY: Wiley.Google Scholar
Bollen, K.A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 55, 605–634.CrossRefGoogle Scholar
Carstens, R., & Hastedt, D., (2010). The effect of not using plausible values when they should be: An illustration using TIMSS 2007 grade 8 mathematics data. Paper presented at the 4th IEA International Research Conference (IRC-2010), Gothenburg, Sweden. Retrieved July 2013 from http://www.iea.nl/irc-2010.html.Google Scholar
Chang, H., & Stout, W. (1993). The asymptotic posterior normality of the latent trait in an IRT model. Psychometrika, 58, 37–52.CrossRefGoogle Scholar
Dresher, A., (2006). Results from NAEP marginal estimation research. Presented at the annual meeting of the National Council on Measurement in Education (NCME), San Francisco, CA.Google Scholar
Ellis, J.L., & Junker, B.W. (1997). Tail-measurability in monotone latent variable models. Psychometrika, 62, 495523.CrossRefGoogle Scholar
Fox, J-P (2010). Bayesian item response modeling: Theory and applications. New York, NY: Springer.CrossRefGoogle Scholar
Johnson, M.S., & Jenkins, F., (2005). A Bayesian hierarchical model for large-scale educational surveys: An application to the National Assessment of Educational Progress. Research Report #RR-04-38. Princeton, NJ: Educational Testing Service. Retrieved from http://www.ets.org/research/policy_research_reports/publications/report/2005/hyzl.Google Scholar
Joreskog, K.G., & Goldberger, A.S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70, 631–639.Google Scholar
Junker, B.W., Schofield, L.S., & Taylor, L. (2012). The use of cognitive ability measures as explanatory variables in regression analysis. IZA Journal of Labor Economics, 1(4), 4.CrossRefGoogle ScholarPubMed
Kirsch, I. et al. (2000). Technical report and data file user’s manual for the 1992 National Adult Literacy Survey. Washington, DC: U.S. Department of Education, National Center for Education Statistics.Google Scholar
Lang, K., & Manove, M. (2011). Education and labor market discrimination. American Economic Review, 101, 1467–1496.CrossRefGoogle Scholar
Li, D., Oranje, A., & Jiang, Y., (2009). On the estimation of hierarchical latent regression models for large-scale assessments. Journal of Educational and Behavioral Statistics, 34, 433–463.CrossRefGoogle Scholar
Meng, X-L. (1994). Multiple-imputation inferences with uncongenial sources of input. Statistical Science, 9(4), 538–558.Google Scholar
Mislevy, R.J. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56, 177–196.CrossRefGoogle Scholar
Mislevy, R.J. (1993). Should “multiple imputations” be treated as “multiple indicators”?. Psychometrika, 58, 79–85.CrossRefGoogle Scholar
Mislevy, R. J., Beaton, A. E., Kaplan, B., & Sheehan, K. M., (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29(2) NAEP, 133–161.CrossRefGoogle Scholar
NCES (2008). NAEP secondary analysis grant abstracts. Washington, DC: National Center for Education Statistics. Retrieved from http://nces.ed.gov/nationsreportcard/researchcenter/naepgrants.asp.Google Scholar
NCES (2009). NAEP technical documentation. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics (NCES). Retrieved July 2013 from http://nces.ed.gov/nationsreportcard/tdw/.Google Scholar
Neal, D., & Johnson, W. (1996). The role of pre-market factors in black-white wage differences. Journal of Political Economy, 104, 869–895.CrossRefGoogle Scholar
Olson, J. F., Martin, M. O., & Mullis, I. V. S., (Eds.). (2008). TIMSS 2007 technical report. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.Google Scholar
Robitaille, D.F., & Beaton, A.E. (Eds.) (2002). Secondary analysis of the TIMSS data. New York, NY: Springer.CrossRefGoogle Scholar
Rubin, D.B. (1987). Multiple imputation for nonresponse in surveys. New York, NY: Wiley.CrossRefGoogle Scholar
Rubin, D.B. (1996). Multiple imputation after 18+ years (with discussion). Journal of the American Statistical Association, 9, 473–489.CrossRefGoogle Scholar
Schofield, L. S. (2008). Modeling measurement error when using cognitive test scores in social science research. Doctoral dissertation. Pittsburgh, PA: Department of Statistics and Heinz College of Public Policy, Carnegie Mellon University.Google Scholar
Schofield, L. S., (2013). Measurement error in latent variables that predict STEM retention (under review).Google Scholar
Spiegelhalter, D. J., Thomas, A., & Best, N. G., (2000). WinBUGS Version 1.3 User Manual. Cambridge: Medical Research Council Biostatistics Unit. Retrieved from http://www.mrc-bsu.cam.ac.uk/bugs.Google Scholar
van der Linden, W.J., & Hambleton, R.K. (Eds.) (1997). Handbook of modern item response theory. New York, NY: Springer.CrossRefGoogle Scholar
von Davier, M., & Gonzalez, M. (2009). What are plausible values and why are they useful?. In von Davier, M., & Hastedt, D. (Eds.), IERI monograph series, vol. 2: Issues and methodologies in large-scale assessments (pp. 9–36). Princeton, NJ: International Association for the Evaluation of Educational Achievement (IEA) and Educational Testing Service (ETS).Google Scholar
Walker, A.M. (1969). On the asymptotic behavior of posterior distributions. Journal of the Royal Statistical Society, Series B, 31, 80–88.CrossRefGoogle Scholar