Hostname: page-component-cd9895bd7-gbm5v Total loading time: 0 Render date: 2025-01-04T03:32:02.923Z Has data issue: false hasContentIssue false
  • English
  • Français

On the Identifiability of Diagnostic Classification Models

Published online by Cambridge University Press:  01 January 2025

Guanhua Fang Open the ORCID record for Guanhua Fang [Opens in a new window] ,
Jingchen Liu  and
Zhiliang Ying
Guanhua Fang
Affiliation:
Columbia University
Jingchen Liu*
Affiliation:
Columbia University
Zhiliang Ying
Affiliation:
Columbia University
*
Correspondence should be made to Jingchen Liu, Columbia University, New York, USA. Email: jcliu@stat.columbia.edu; URL: http://stat.columbia.edu/jcliu/
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper establishes fundamental results for statistical analysis based on diagnostic classification models (DCMs). The results are developed at a high level of generality and are applicable to essentially all diagnostic classification models. In particular, we establish identifiability results for various modeling parameters, notably item response probabilities, attribute distribution, and Q-matrix-induced partial information structure. These results are stated under a general setting of latent class models. Through a nonparametric Bayes approach, we construct an estimator that can be shown to be consistent when the identifiability conditions are satisfied. Simulation results show that these estimators perform well under various model settings. We also apply the proposed method to a dataset from the National Epidemiological Survey on Alcohol and Related Conditions (NESARC).

Keywords

identifiabilitydiagnostic classification modelsDirichlet allocation
Type
Original Paper
Information
Psychometrika , Volume 84 , Issue 1 , 15 March 2019 , pp. 19 - 40
Copyright
Copyright © 2019 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11336-018-09658-x) contains supplementary material, which is available to authorized users.

References

Allman, E. S., Matias, C., & Rhodes, J. A. (2009). Identifiability of parameters in latent structure models with many observed variables. The Annals of Statistics, 37, 3099– 3132. CrossRefGoogle Scholar
Chen, Y., Culpepper, S. A., Chen, Y., Douglas, J. Bayesian estimation of the DINA Q \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q$$\end{document} matrix. (2018). Psychometrika, 83, 89– 108. CrossRefGoogle Scholar
Chen, Y., Liu, J., Xu, G., & Ying, Z. Statistical analysis of Q \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q$$\end{document} -matrix based diagnostic classification models. (2015). Journal of the American Statistical Association, 110, 850– 866. CrossRefGoogle Scholar
Chen, Y., Liu, J., & Ying, Z. Online item calibration for Q-matrix in CD-CAT. (2015). Applied Psychological Measurement, 39, 5– 15. CrossRefGoogle ScholarPubMed
Chiu, C-Y, Douglas, J. A., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633– CrossRefGoogle Scholar
De La Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of educational and behavioral statistics, 34, 115– 130. CrossRefGoogle Scholar
De La Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179– 199. CrossRefGoogle Scholar
De La Torre, J., & Douglas, J. A. (2004). Higher order latent trait models for cognitive diagnosis. Psychometrika, 69, 333– 353. CrossRefGoogle Scholar
DeCarlo, L. T. On the analysis of fraction subtraction data: The DINA model, classification, latent class sizes, and the Q \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q$$\end{document} -matrix. (2011). Applied Psychological Measurement, 35, 8– 26. CrossRefGoogle Scholar
DiBello, L. V., Stout, W. F., & Roussos, L. A. Chipman, S. F., Nichols, P. D., & Brennan, R. L. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. Cognitively diagnostic assessment, Hillsdale, NJ: Erlbaum. 361– 389. Google Scholar
Dunson, D., Xing, C. (2009). Nonparametric Bayes modeling of multivariate categorical data. Journal of the American Statistical Association, 104, 1042– 1051. CrossRefGoogle Scholar
Grant, B. F., Kaplan, K., Shepard, J., & Moore, T. (2003). Source and accuracy statement for wave 1 of the 2001–2002 national epidemiologic survey on alcohol and related conditions, Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism. Google Scholar
Gu, Y., & Xu, G. (2018). https://doi.org/10.1007/s11336-018-9619-8. The sufficient and necessary condition for the identifiability and estimability of the DINA model. Psychometrika.Google Scholar
Gu, Y., Liu, J., Xu, G., & Ying, Z. (2018). Hypothesis testing of the Q \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q$$\end{document} -matrix. Psychometrika, 83, 515– 537. CrossRefGoogle Scholar
Hartz, S.  M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Doctoral Dissertation, University of Illinois, Urbana-Champaign.Google Scholar
Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191– 210. CrossRefGoogle Scholar
Iza, M., Wall, M., Heimberg, R., Rodebaugh, T., Schneier, F., Liu, S. -M., & Blanco, C. (2014). Latent structure of social fears and social anxiety disorders. Psychological medicine, 44, 361– 370. CrossRefGoogle ScholarPubMed
Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258– 272. CrossRefGoogle Scholar
Kruskal, J. (1977). Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and its Application, 18, 95– 138. CrossRefGoogle Scholar
Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The attribute hierarchy model for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41, 205– 237. CrossRefGoogle Scholar
Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of Q \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q$$\end{document} -matrix. Applied Psychological Measurement, 36, 609– 618. Google Scholar
Liu, J., Xu, G., Ying, Z. Theory of the self-learning Q \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q$$\end{document} -matrix. (2013). Bernoulli: Official Journal of the Bernoulli Society for Mathematical Statistics and Probability, 19, 1790 Google Scholar
Roussos, L. A., Templin, J. L., & Henson, R. A. (2007). Skills diagnosis using IRT-based latent class models. Journal of Educational Measurement, 44, 293– 311. CrossRefGoogle Scholar
Rupp, A. A., & Templin, J. (2008a). The effects of Q \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q$$\end{document} -matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68, 78– 96. CrossRefGoogle Scholar
Rupp, A. A., Templin, J. L. (2008b). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement: Interdisciplinary Research and Perspective, 6, 219– 262. Google Scholar
Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications, New York: Guilford Press. Google Scholar
Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statistica Sinica, 4, 639– 650. Google Scholar
Stout, W. (2007). Skills diagnosis using IRT-based continuous latent trait models. Journal of Educational Measurement, 44, 313– 324. CrossRefGoogle Scholar
Tatsuoka, K. K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach. Journal of Educational Statistics, 12, 55– 73. CrossRefGoogle Scholar
Tatsuoka, K. K. (2009). Cognitive assessment: An introduction to the rule space method, Boca Raton: CRC Press. CrossRefGoogle Scholar
Templin, J., He, X., Roussos, L., & Stout, W. (2003). The pseudo-item method: a simple technique for analysis of polytomous data with the fusion model. External diagnostic research group technical report.Google Scholar
Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287– 305. CrossRefGoogle ScholarPubMed
Van der Vaart, A. W. (1998). Asymptotic statistics, Cambridge: Cambridge University Press. CrossRefGoogle Scholar
von Davier, M. (2005). A general diagnosis model applied to language testing data. Research report: Educational testing service.Google Scholar
Vonesh, E. F., & Chinchilli, V. G. (1997). Linear and nonlinear models for the analysis of repeated measurements, London: Chapman and Hall. Google Scholar
Walker, S. G. (2007). Sampling the dirichlet mixture model with slices. Communications in Statistics-Simulation and Computation, 36, 45– 54. CrossRefGoogle Scholar
Xu, G. (2017). Identifiability of restricted latent class models with binary responses. The Annals of Statistics, 45, 675– 707. CrossRefGoogle Scholar
Xu, G., & Shang, Z. (2018). Identifying latent structures in restricted latent class models. Journal of the American Statistical Association, 113, 523 1284– 1295. CrossRefGoogle Scholar
Xu, G., & Zhang, S. (2016). Identifiability of diagnostic classification models. Psychometrika, 81, 625– 649. CrossRefGoogle ScholarPubMed
Supplementary material: File

Fang et al. supplementary material 1

Supplementary to “On the Identifiability of Diagnostic Classification Models”
Download Fang et al. supplementary material 1(File)
File 218.6 KB
Supplementary material: File

Fang et al. supplementary material 2

Fang et al. supplementary material 2
Download Fang et al. supplementary material 2(File)
File 4.6 KB
Supplementary material: File

Fang et al. supplementary material 3

Fang et al. supplementary material 3
Download Fang et al. supplementary material 3(File)
File 4.8 KB