Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2025-01-05T21:09:53.109Z Has data issue: false hasContentIssue false
  • English
  • Français

Learning Latent and Hierarchical Structures in Cognitive Diagnosis Models

Published online by Cambridge University Press:  01 January 2025

Chenchen Ma ,
Jing Ouyang  and
Gongjun Xu Open the ORCID record for Gongjun Xu [Opens in a new window]
Chenchen Ma
Affiliation:
University of Michigan
Jing Ouyang
Affiliation:
University of Michigan
Gongjun Xu*
Affiliation:
University of Michigan
*
Correspondence should be made to Gongjun Xu, Department of Statistics, University of Michigan, 456 West Hall, 1085 South University, Ann Arbor, MI48109, USA. Email: gongjun@umich.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Cognitive Diagnosis Models (CDMs) are a special family of discrete latent variable models that are widely used in educational and psychological measurement. A key component of CDMs is the Q-matrix characterizing the dependence structure between the items and the latent attributes. Additionally, researchers also assume in many applications certain hierarchical structures among the latent attributes to characterize their dependence. In most CDM applications, the attribute–attribute hierarchical structures, the item-attribute Q-matrix, the item-level diagnostic models, as well as the number of latent attributes, need to be fully or partially pre-specified, which however may be subjective and misspecified as noted by many recent studies. This paper considers the problem of jointly learning these latent and hierarchical structures in CDMs from observed data with minimal model assumptions. Specifically, a penalized likelihood approach is proposed to select the number of attributes and estimate the latent and hierarchical structures simultaneously. An expectation-maximization (EM) algorithm is developed for efficient computation, and statistical consistency theory is also established under mild conditions. The good performance of the proposed method is illustrated by simulation studies and real data applications in educational assessment.

Keywords

cognitive diagnosis modelhierarchical structureQ-matrixpenalized EM algorithm
Type
Theory and Methods
Information
Psychometrika , Volume 88 , Issue 1 , March 2023 , pp. 175 - 207
Copyright
Copyright © 2022 The Author(s) under exclusive licence to The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/S0033312300006074.

References

Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers Foundations and Trends in Machine learning 3(1) 1–122 10.1561/2200000016CrossRefGoogle Scholar
Chen, J., Chen, Z., (2008). Extended Bayesian information criteria for model selection with large model spaces Biometrika 95(3) 759–771 10.1093/biomet/asn034CrossRefGoogle Scholar
Chen, J., de la Torre, J., (2014). A procedure for diagnostically modeling extant large-scale assessment data: The case of the programme for international student assessment in reading Psychology 5(18) 1967–1978 10.4236/psych.2014.518200CrossRefGoogle Scholar
Chen, Y., Culpepper, S. A., Chen, Y., Douglas, J., (2018). Bayesian estimation of the DINA Q-matrix Psychometrika 83(1) 89–108 10.1007/s11336-017-9579-4 28861685CrossRefGoogle ScholarPubMed
Chen, Y., Li, X., Liu, J., Ying, Z., (2017). Regularized latent class analysis with application in cognitive diagnosis Psychometrika 82(3) 660–692 10.1007/s11336-016-9545-6CrossRefGoogle Scholar
Chen, Y., Liu, J., Xu, G., Ying, Z., (2015). Statistical analysis of Q-matrix based diagnostic classification models Journal of the American Statistical Association 110(510) 850–866 10.1080/01621459.2014.934827 26294801CrossRefGoogle Scholar
Chiu, C-Y (2013). Statistical refinement of the Q-matrix in cognitive diagnosis Applied Psychological Measurement 37(8) 598–618 10.1177/0146621613488436CrossRefGoogle Scholar
Chung, M. & Johnson, M. S. (2018). An MCMC algorithm for estimating the Q-matrix in a Bayesian framework. arXiv preprint arXiv:1802.02286.Google Scholar
Culpepper, S. A., (2019). Estimating the cognitive diagnosis Q matrix with expert knowledge: Application to the fraction-subtraction dataset Psychometrika 84(2) 333–357 10.1007/s11336-018-9643-8 30456748CrossRefGoogle Scholar
Dahlgren, M. A., Hult, H., Dahlgren, L. O., af Segerstad, H. H., & Johansson, K. (2006). From senior student to novice worker: Learning trajectories in political science, psychology and mechanical engineering. Studies in Higher Education, 31(5), 569–586.CrossRefGoogle Scholar
de la Torre, J., (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications Journal of Educational Measurement 45(4) 343–362 10.1111/j.1745-3984.2008.00069.xCrossRefGoogle Scholar
de la Torre, J., (2011). The generalized DINA model framework Psychometrika 76(2) 179–199 10.1007/s11336-011-9207-7CrossRefGoogle Scholar
de la Torre, J., Chiu, C-Y (2016). A general method of empirical Q-matrix validation Psychometrika 81(2) 253–273 10.1007/s11336-015-9467-8 25943366CrossRefGoogle ScholarPubMed
de la Torre, J., van der Ark, L. A., Rossi, G., (2018). Analysis of clinical data from a cognitive diagnosis modeling framework Measurement and Evaluation in Counseling and Development 51(4) 281–296 10.1080/07481756.2017.1327286CrossRefGoogle Scholar
DeCarlo, L. T., (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model Applied Psychological Measurement 36(6) 447–468 10.1177/0146621612449069CrossRefGoogle Scholar
Delyon, B., Lavielle, M., Moulines, E., (1999). Convergence of a stochastic approximation version of the EM algorithm The Annals of Statistics 27(1) 94–128 10.1214/aos/1018031103CrossRefGoogle Scholar
DiBello, L. V., Stout, W. F., & Roussos, L. A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. Cognitively Diagnostic Assessment, 361389.Google Scholar
Fan, J., Li, R., (2001). Variable selection via nonconcave penalized likelihood and its oracle properties Journal of the American statistical Association 96(456) 1348–1360 10.1198/016214501753382273CrossRefGoogle Scholar
George, A. C., Robitzsch, A., (2015). Cognitive diagnosis models in R: A didactic The Quantitative Methods for Psychology 11(3) 189–205 10.20982/tqmp.11.3.p189CrossRefGoogle Scholar
Goodman, L. A., (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models Biometrika 61(2) 215–231 10.1093/biomet/61.2.215CrossRefGoogle Scholar
Gu, Y., Liu, J., Xu, G., Ying, Z., (2018). Hypothesis testing of the Q-matrix Psychometrika 83(3) 515–537 10.1007/s11336-018-9629-6CrossRefGoogle Scholar
Gu, Y. & Xu, G. (2019a). Identification and estimation of hierarchical latent attribute models. arXiv preprint arXiv:1906.07869.Google Scholar
Gu, Y. & Xu, G. (2019b). Learning attribute patterns in high-dimensional structured latent attribute models. Journal of Machine Learning Research, 20(2019).Google Scholar
Gu, Y., Xu, G., (2019). The sufficient and necessary condition for the identifiability and estimability of the DINA model Psychometrika 84(2) 468–483 10.1007/s11336-018-9619-8 29728918CrossRefGoogle ScholarPubMed
Haertel, E. H., (1989). Using restricted latent class models to map the skill structure of achievement items Journal of Educational Measurement 26(4) 301–321 10.1111/j.1745-3984.1989.tb00336.xCrossRefGoogle Scholar
Henson, R. A., Templin, J. L., Willse, J. T., (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables Psychometrika 74(2) 191 10.1007/s11336-008-9089-5CrossRefGoogle Scholar
Jimoyiannis, A., & Komis, V. (2001). Computer simulations in physics teaching and learning: A case study on students understanding of trajectory motion. Computers & Education, 36(2), 183–204.CrossRefGoogle Scholar
Junker, B. W., Sijtsma, K., (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory Applied Psychological Measurement 25(3) 258–272 10.1177/01466210122032064CrossRefGoogle Scholar
Kuhn, H. W., (1955). The Hungarian method for the assignment problem Naval Research Logistics Quarterly 2 1–2 83–97 10.1002/nav.3800020109CrossRefGoogle Scholar
Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41(3), 205–237.CrossRefGoogle Scholar
Li, C., Ma, C., Xu, G., (2022). Learning large Q-matrix by restricted Boltzmann machines Psychometrika 10.1007/s11336-021-09828-4 35089496CrossRefGoogle ScholarPubMed
Liu, J., Xu, G., Ying, Z., (2012). Data-driven learning of Q-matrix Applied Psychological Measurement 36(7) 548–564 10.1177/0146621612456591 23926363CrossRefGoogle ScholarPubMed
Ma, C., Xu, G., (2021). Hypothesis testing for hierarchical structures in cognitive diagnosis models Journal of Data Science 10.6339/21-JDS1024Google Scholar
O’Brien, K. L., Baggett, H. C., Brooks, W. A., Feikin, D. R., Hammitt, L. L., Higdon, M. M., Howie, S. R., Knoll, M. D., Kotloff, K. L., Levine, O. S., et al. (2019). Causes of severe pneumonia requiring hospital admission in children without HIV infection from Africa and Asia: The PERCH multi-country case-control study The Lancet 394(10200) 757–779 10.1016/S0140-6736(19)30721-4CrossRefGoogle Scholar
Schwarz, G., (1978). Estimating the dimension of a model The Annals of Statistics 6(2) 461–464 10.1214/aos/1176344136CrossRefGoogle Scholar
Shen, X., Pan, W., Zhu, Y., (2012). Likelihood-based selection and sharp parameter estimation Journal of the American Statistical Association 107(497) 223–232 10.1080/01621459.2011.645783 22736876 3378256CrossRefGoogle ScholarPubMed
Simon, M. A., Tzur, R., (2004). Explicating the role of mathematical tasks in conceptual learning: An elaboration of the hypothetical learning trajectory Mathematical Thinking and Learning 6(2) 91–104 10.1207/s15327833mtl0602_2CrossRefGoogle Scholar
Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 345–354.CrossRefGoogle Scholar
Tatsuoka, K. K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. Diagnostic Monitoring of Skill and Knowledge Acquisition, 453–488.Google Scholar
Templin, J., Bradshaw, L., (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies Psychometrika 79(2) 317–339 10.1007/s11336-013-9362-0 24478021CrossRefGoogle ScholarPubMed
Templin, J. L., Henson, R. A., (2006). Measurement of psychological disorders using cognitive diagnosis models Psychological Methods 11(3) 287 10.1037/1082-989X.11.3.287 16953706CrossRefGoogle ScholarPubMed
Tuy, H. (1995). DC optimization: Theory, methods and algorithms. In Handbook of Global Optimization (pp. 149–216). Springer.CrossRefGoogle Scholar
von Davier, M., (2005). A general diagnostic model applied to language testing data ETS Research Report Series 2005(2) i–35 10.1002/j.2333-8504.2005.tb01993.xGoogle Scholar
von Davier, M. (2019). The general diagnostic model. In Handbook of Diagnostic Classification Models (pp. 133–153). Springer.CrossRefGoogle Scholar
von Davier, M., & Haberman, S. J. (2014). Hierarchical diagnostic classification models morphing into unidimensional ‘diagnostic’classification models—A commentary. Psychometrika, 79(2), 340–346.CrossRefGoogle Scholar
Wang, C., & Gierl, M. J. (2011). Using the attribute hierarchy method to make diagnostic inferences about examinees’ cognitive skills in critical reading. Journal of Educational Measurement, 48(2), 165–187.CrossRefGoogle Scholar
Wang, C., Lu, J., (2021). Learning attribute hierarchies from data: Two exploratory approaches Journal of Educational and Behavioral Statistics 46(1) 58–84 10.3102/1076998620931094CrossRefGoogle Scholar
Wang, S., Liao, L., (2001). Decomposition method with a variable parameter for a class of monotone variational inequality problems Journal of Optimization Theory and Applications 109(2) 415–429 10.1023/A:1017522623963CrossRefGoogle Scholar
Wu, C., Kwon, S., Shen, X., Pan, W., (2016). A new algorithm and theory for penalized regression-based clustering Journal of Machine Learning Research 17(1) 6479–6503Google ScholarPubMed
Wu, Z., Deloria-Knoll, M., Hammitt, L. L., Zeger, S. L., and for Child Health Core Team, P. E. R. (2016b). Partially latent class models for case-control studies of childhood pneumonia aetiology. Journal of the Royal Statistical Society: Series C (Applied Statistics), 65(1), 97–114.CrossRefGoogle Scholar
Wu, Z., Deloria-Knoll, M., Zeger, S. L., (2016). Nested partially latent class models for dependent binary data, estimating disease etiology Biostatistics 18(2) 200–213Google Scholar
Xu, G., (2017). Identifiability of restricted latent class models with binary responses The Annals of Statistics 45(2) 675–707 10.1214/16-AOS1464CrossRefGoogle Scholar
Xu, G., Shang, Z., (2018). Identifying latent structures in restricted latent class models Journal of the American Statistical Association 113(523) 1284–1295 10.1080/01621459.2017.1340889CrossRefGoogle Scholar
Xu, G., Zhang, S., (2016). Identifiability of diagnostic classification models Psychometrika 81(3) 625–649 10.1007/s11336-015-9471-z 26155755CrossRefGoogle ScholarPubMed
Supplementary material: File

Ma et al. supplementary material

Ma et al. supplementary material
Download Ma et al. supplementary material(File)
File 283.5 KB