Hostname: page-component-745bb68f8f-lrblm Total loading time: 0 Render date: 2025-01-07T18:41:22.072Z Has data issue: false hasContentIssue false
  • English
  • Français

A Multicomponent Latent Trait Model for Diagnosis

Published online by Cambridge University Press:  01 January 2025

Susan E. Embretson  and
Xiangdong Yang
Susan E. Embretson
Affiliation:
Georgia Institute of Technology
Xiangdong Yang*
Affiliation:
East China Normal University
*
Requests for reprints should be sent to Xiangdong Yang, Department of Educational Psychology, East China Normal University, Shanghai, China. E-mail: xdyang50@hotmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper presents a noncompensatory latent trait model, the multicomponent latent trait model for diagnosis (MLTM-D), for cognitive diagnosis. In MLTM-D, a hierarchical relationship between components and attributes is specified to be applicable to permit diagnosis at two levels. MLTM-D is a generalization of the multicomponent latent trait model (MLTM; Whitely in Psychometrika, 45:479–494, 1980; Embretson in Psychometrika, 49:175–186, 1984) to be applicable to measures of broad traits, such as achievement tests, in which component structure varies between items. Conditions for model identification are described and marginal maximum likelihood estimators are presented, along with simulation data to demonstrate parameter recovery. To illustrate how MLTM-D can be used for diagnosis, an application to a large-scale test of mathematics achievement is presented. An advantage of MLTM-D for diagnosis is that it may be more applicable to large-scale assessments with more heterogeneous items than are latent class models.

Keywords

item response theorycognitive diagnosismultidimensional measurement models
Type
Original Paper
Information
Psychometrika , Volume 78 , Issue 1 , January 2013 , pp. 14 - 36
Copyright
Copyright © The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Preparation of this paper was partially supported by Institute of Educational Sciences Grant R305A100234, Susan Embretson, PI.

References

Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika, 46, 443–445CrossRefGoogle Scholar
Bolt, D.M., & Lall, V.F. (2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo. Applied Psychological Measurement, 27, 395–414CrossRefGoogle Scholar
Cai, L. (2010). Metropolis-Hastings Robbins-Monroe algorithm for confirmatory factor analysis. Journal of Educational and Behavioral Statistics, 35, 307–335CrossRefGoogle Scholar
Cai, L., du Toit, S.H.C., & Thissen, D. (2012). IRTPRO: Flexible, multidimensional, multiple category IRT modeling, Chicago: Scientific Software International Computer softwareGoogle Scholar
Cho, S.-J., & Rabe-Hesketh, S. (2011). Alternating imputation posterior estimation of models with crossed random effects. Computational Statistics & Data Analysis, 55, 12–25CrossRefGoogle Scholar
Daniel, R.C., & Embretson, S.E. (2010). Designing cognitive complexity in mathematical problem solving items. Applied Psychological Measurement, 348–364. CrossRefGoogle Scholar
DeCarlo, L.T. (2011). The analysis of fraction subtraction data: the DINA model, classification, latent class sizes, and the Q-matrix. Applied Psychological Measurement, 35, 8–26CrossRefGoogle Scholar
de la Torre, J., & Chiu, C.-Y. (2009). Q-matrix validation under the generalized DINA model framework. In The international meeting of the psychometric society, Cambridge, St John’s College, July 20th–24th.Google Scholar
de la Torre, J., & Douglas, J.A. (2004). Higher order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353CrossRefGoogle Scholar
DiBello, L.V., Stout, W.F., & Roussos, L.A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. In Nichols, P.D., Chipman, S.F., & Brennan, R.L. (Eds.), Cognitively diagnostic assessment, Erlbaum: Hillsdale 327359Google Scholar
Embretson, S.E. (1984). A general multicomponent latent trait model for response processes. Psychometrika, 49, 175–186CrossRefGoogle Scholar
Embretson, S.E. (1995). Working memory capacity versus general central processes in intelligence. Intelligence, 20, 169–189CrossRefGoogle Scholar
Embretson, S.E. (1999). Generating items during testing: psychometric issues and models. Psychometrika, 64, 407–433CrossRefGoogle Scholar
Embretson, S.E. (2010). An adaptive testing system for diagnosing sources of mathematics difficulties. Project R305A100234. Washington, Institute of Educational Sciences. Google Scholar
Embretson, S.E., & Daniel, R.C. (2008). Understanding and quantifying cognitive complexity level in mathematical problem solving items. Psychology Science, 50, 328–344Google Scholar
Embretson, S.E., & McCollam, K.M. (2000). A multicomponent Rasch model for covert processes. In Wilson, M., & Engelhard, G. (Eds.), Objective measurement V, Norwood: AblexGoogle Scholar
Fischer, G.H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359–374CrossRefGoogle Scholar
Gorin, J. (2007). Test design with cognition in mind. Educational Measurement, Issues and Practice, 4, 21–35Google Scholar
Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352CrossRefGoogle Scholar
Hartz, S. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: blending theory with practice. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign. Google Scholar
Henson, R.A., Templin, J.L., & Willse, J.T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210CrossRefGoogle Scholar
Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272CrossRefGoogle Scholar
Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 197–212CrossRefGoogle Scholar
Mislevy, R.J., & Bock, R.D. (1990). BILOG 3: item analysis and test scoring with binary logistic models, Chicago: Scientific Software InternationalGoogle Scholar
National Mathematics Advisory Panel (2008). Foundations for success: final report. U.S. Department of Education. Google Scholar
Pellegrino, J., Goldman, S., DiBello, L., Gomez, K., & Stout, W. (2011). Evaluating the cognitive, psychometric and instructional affordances of curriculum-embedded assessments: a comprehensive validity-based approach, Chicago: Learning Sciences Institute, University of IllinoisGoogle Scholar
Poggio, J.P. (2011). Indicators, benchmarks and standards reflected in items for mathematical achievement tests in middle school (Technical Report KU-10012011). Institute of Educational Science Mathematics Diagnosis Project, Lawrence, Kansas. Google Scholar
Roussos, L., DiBello, L., Henson, R., Jang, E., & Templin, J. (2010). Skills diagnosis for education and psychology with IRT-based parametric latent class models. In Embretson, S.E. (Eds.), Measuring psychological constructs: advances in model-based approaches, Washington: American Psychological AssociationGoogle Scholar
Rupp, A., & Templin, J. (2008). The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68, 78–96CrossRefGoogle Scholar
Rupp, A.A., Templin, J., & Henson, R.J. (2010). Diagnostic measurement: theory, methods, and applications, New York: Guilford PressGoogle Scholar
Stout, W. (2007). Skills diagnosis using IRT-based continuous latent trait models. Journal of Educational Measurement, 44, 313–324CrossRefGoogle Scholar
Tatsuoka, K.K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach. Journal of Educational Statistics, 12, 55–73CrossRefGoogle Scholar
Templin, J.L., & Henson, R.A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305CrossRefGoogle ScholarPubMed
von Davier, M. (2005). A general diagnostic model applied to language testing data (Research Report RR-05-16). Princeton: ETS. Google Scholar
von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical & Statistical Psychology, 61, 287–307CrossRefGoogle ScholarPubMed
von Davier, M. (2009). Some notes on the reinvention of latent structure models as diagnostic classification models. Measurement: Interdisciplinary Research and Perspectives, 7(1), 6774Google Scholar
von Davier, M., & Sinharay, S. (2010). Stochastic approximation for latent regression item response models. Journal of Educational and Behavioral Statistics, 35, 174–193CrossRefGoogle Scholar
von Davier, M., & Yamamoto, K. (2004). A class of models for cognitive diagnosis. Paper presented at the 4th Spearman conference. Educational testing service: the inn at Penn. Philadelphia, October. Google Scholar
von Davier, M., & Yamamoto, K. (2007). Mixture distribution Rasch models and hybrid Rasch models. In Davier, M., & Carstensen, C.H. (Eds.), Multivariate and mixture distribution Rasch models, New York: SpringerCrossRefGoogle Scholar
Webb, N.L. (1999). Alignment of science and mathematics standards and assessments in four states (Research Monograph No. 18). Madison, National Institute for Science Education University of Wisconsin-Madison. Google Scholar
Whitely, S.E. (1980). Multicomponent latent trait models for ability tests. Psychometrika, 45, 479–494CrossRefGoogle Scholar
Yang, X. (2010). Construct theory-driven cognitive diagnostic testing in the domain of algebra story problems, Shanghai: National Science Foundation of ChinaGoogle Scholar
Yang, X., & Embretson, S.E. (2007). Construct validity and cognitive diagnostic assessment. In Leighton, J.P., & Gierl, M. (Eds.), Cognitive diagnostic assessment for education, New York: Cambridge University PressGoogle Scholar