Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-07T18:42:02.489Z Has data issue: false hasContentIssue false
  • English
  • Français

Variational Bayes Inference Algorithm for the Saturated Diagnostic Classification Model

Published online by Cambridge University Press:  01 January 2025

Kazuhiro Yamaguchi Open the ORCID record for Kazuhiro Yamaguchi [Opens in a new window]  and
Kensuke Okada Open the ORCID record for Kensuke Okada [Opens in a new window]
Kazuhiro Yamaguchi*
Affiliation:
University of Iowa Japan Society for the Promotion of Science
Kensuke Okada
Affiliation:
University of Tokyo
*
Correspondence should be made to Kazuhiro Yamaguchi, Department of the Psychological and Quantitative Foundations, University of Iowa, 216 Lindquist Center, 240 S Madison St., Iowa City, IA, 52242, USA. Email: kazz530@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Saturated diagnostic classification models (DCM) can flexibly accommodate various relationships among attributes to diagnose individual attribute mastery, and include various important DCMs as sub-models. However, the existing formulations of the saturated DCM are not better suited for deriving conditionally conjugate priors of model parameters. Because their derivation is the key in developing a variational Bayes (VB) inference algorithm, in the present study, we proposed a novel mixture formulation of saturated DCM. Based on it, we developed a VB inference algorithm of the saturated DCM that enables us to perform scalable and computationally efficient Bayesian estimation. The simulation study indicated that the proposed algorithm could recover the parameters in various conditions. It has also been demonstrated that the proposed approach is particularly suited to the case when new data become sequentially available over time, such as in computerized diagnostic testing. In addition, a real educational dataset was comparatively analyzed with the proposed VB and Markov chain Monte Carlo (MCMC) algorithms. The result demonstrated that very similar estimates were obtained between the two methods and that the proposed VB inference was much faster than MCMC. The proposed method can be a practical solution to the problem of computational load.

Keywords

variational Bayes inferencediagnostic classification modelscognitive diagnostic modelssaturated model
Type
Theory and Methods
Information
Psychometrika , Volume 85 , Issue 4 , December 2020 , pp. 973 - 995
Copyright
Copyright © 2021 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic Supplementary material The online version of this article (https://doi.org/10.1007/s11336-020-09739-w) contains supplementary material, which is available to authorized users.

References

Beal, M. J. (2003). Variational algorithms for approximate Bayesian inference. [Doctoral dissertation, University College London]. https://www.cse.buffalo.edu/faculty/mbeal/thesis/.Google Scholar
Bishop, M. (2006). Pattern recognition and machine learning. Pattern Recognition. https://doi.org/10.1641/B580519.CrossRefGoogle Scholar
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D.(2017). Variational inference: A review for statisticians. Journal of the American Statistical Association,112,859877.CrossRefGoogle Scholar
Brooks, S., Gelman, A., Jones, G. L., & Meng, X. -L.(2011). Handbook of Markov chain Monte Carlo,Roken Sound Parkway:CRC Press.CrossRefGoogle Scholar
Chen, J., & de la Torre, J.(2014). A procedure for diagnostically modeling extant large-scale assessment data? The case of the programme for international student assessment in reading. Psychology,5,19671978.CrossRefGoogle Scholar
Chen, J., de la Torre, J., & Zhang, Z.(2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement,50(2), 123–140.CrossRefGoogle Scholar
Chen, Y., Culpepper, S. A., Chen, Y., & Douglas, J.(2018). Bayesian estimation of the DINA Q matrix. Psychometrika,83(1), 89–108.CrossRefGoogle ScholarPubMed
Chiu, C. Y., & Douglas, J.(2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification,30,225–250.CrossRefGoogle Scholar
Culpepper, S. A.(2015). Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics,40,454–476.CrossRefGoogle Scholar
Culpepper, S. A.(2019). Estimating the cognitive diagnosis Q matrix with expert knowledge: Application to the fraction-subtraction dataset. Psychometrika,84,333–357.CrossRefGoogle Scholar
Culpepper, S. A., & Hudson, A.(2018). An improved strategy for Bayesian estimation of the reduced reparameterized unified model. Applied Psychological Measurement,42(2), 99–115.CrossRefGoogle ScholarPubMed
de la Torre, J.(2011). The generalized DINA model framework. Psychometrika,76,179–199.CrossRefGoogle Scholar
Dempster, A. P., Laird, N. M., & Rubin, D.(1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological),39,1–38.CrossRefGoogle Scholar
Gelman, A., & Rubin, D. B.(1992). Inference from iterative simulation using multiple sequences. Statistical Science,7,457–472.CrossRefGoogle Scholar
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis (3rd ed.). Chapman and Hall/CRC .Google Scholar
George, A. C., Robitzsch, A., Kiefer, T., Groß, J., & Ünlü, A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software. https://doi.org/10.18637/jss.v074.i02.CrossRefGoogle Scholar
Grimmer, J.(2011). An introduction to Bayesian inference via variational approximations. Political Analysis,19,32–47.CrossRefGoogle Scholar
Haertel, E. H.(1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement,26,301–321.CrossRefGoogle Scholar
Hartz, S., & Roussos, L. (2008). The fusion model for skills diagnosis: Blending theory with practice. ETS Research Report Series, 08–71, 1–57. Retrieved from https://www.ets.org/Media/Research/pdf/RR-08-71.pdf.Google Scholar
Henson, R. A., Templin, J. L., & Willse, J. T.(2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika,74,191–210.CrossRefGoogle Scholar
Jeon, M., Rijmen, F., & Rabe-hesketh, S.(2017). A variational maximization–maximization algorithm for generalized linear mixed models with crossed random effects. Psychometrika,82,693–716.CrossRefGoogle Scholar
Junker, B. W., & Sijtsma, K.(2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement,25,258–272.CrossRefGoogle Scholar
Lee, M. D., & Wagenmakers, E. -J.(2013). Bayesian cognitive modeling: A practical course,Cambridge:Cambridge University Press.Google Scholar
Lee, Y. -S., Park, Y. S., & Taylan, D.(2011). A cognitive diagnostic modeling of attribute mastery in massachusetts, minnesota, and the U.S. national sample using the TIMSS 2007. International Journal of Testing,11,144–177.CrossRefGoogle Scholar
Leighton, J. P., & Gierl, M. J.(2007). Cognitive diagnostic assessment for education: Theory and applications,New York:Cambridge University Press.CrossRefGoogle Scholar
Li, H., Hunter, C. V., & Lei, P.-W. (2016). The selection of cognitive diagnostic models for a reading comprehension test. Language Testing, 33(2012), 1–35. https://doi.org/10.1177/0265532215590848.CrossRefGoogle Scholar
Liu, J., Xu, G., & Ying, Z.(2012). Data-driven learning of Q-matrix. Applied Psychological Measurement,36,548–564.CrossRefGoogle ScholarPubMed
Ma, W., & de la Torre, J.(2020). GDINA: An R package for cognitive diagnosis modeling. Journal of Statistical Software,93(10), 1–26.CrossRefGoogle Scholar
Macready, G. B., & Dayton, C. M.(1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics,2,99–120.CrossRefGoogle Scholar
Nakajima, S., Watanabe, K., & Sugiyama, M.(2019). Variational Bayesian learning theory,New York:Cambridge University Press.CrossRefGoogle Scholar
Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. In The 3rd international workshop on distributed statistical computing, 124, 1–8. Retrieved from http://www.ci.tuwien.ac.at/Conferences/DSC-2003/.Google Scholar
R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/.Google Scholar
Rijmen, F., Jeon, M., & Rabe-Hesketh, S.(2016). Variational approximation methods. van der Linden, W. J. Handbook of item response theory, volume two: Statistical tools,Boca Raton:CRC Press 259–270.Google Scholar
Rupp, A. A., Templin, J. L., & Henson, R. A.(2010). Diagnostic measurement: Theory, methods and applications,New York:Guilford Press.Google Scholar
Sessoms, J., & Henson, R. A.(2018). Applications of diagnostic classification models: A literature review and critical commentary. Measurement: Interdisciplinary Research and Perspectives,16,1–17.Google Scholar
Tatsuoka, K. K.(1983). Rule space: An approach for dealing with misconceptions based on item response theory. Jounal of Educational Measurement,20,345–354.CrossRefGoogle Scholar
Tatsuoka, K. K., Corter, J. E., & Tatsuoka, C.(2004). Patterns of diagnosed mathematical content and process skills in TIMSS-R across a sample of 20 countries. American Educational Research Journal,41,901–926.CrossRefGoogle Scholar
Templin, J., & Bradshaw, L.(2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika,79,317–339.CrossRefGoogle ScholarPubMed
Templin, J., & Hoffman, L.(2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice,32,37–50.CrossRefGoogle Scholar
Templin, J. L., & Henson, R. A.(2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods,11,287–305.CrossRefGoogle ScholarPubMed
Tzikas, D. G., Likas, A. C., & Galatsanos, N. P. (2008). The variational approximation for Bayesian inference. IEEE Signal Processing Magazine. https://doi.org/10.1109/MSP.2008.929620.CrossRefGoogle Scholar
van Ravenzwaaij, D., Cassey, P., & Brown, S. D.(2018). A simple introduction to Markov Chain Monte-Carlo sampling. Psychonomic Bulletin and Review,25(eq1), 143–154.CrossRefGoogle ScholarPubMed
von Davier, M.(2008). A general diagnostic model applied to language testing data. The British Journal of Mathematical and Statistical Psychology,61(Pt 2), 287–307.CrossRefGoogle ScholarPubMed
von Davier, M.(2010). Hierarchical mixtures of diagnostic models. Psychological Test and Assessment Modeling,52,8–28.Google Scholar
von Davier, M. (2014). The log-linear cognitive diagnostic model (LCDM) as a special case of the general diagnostic model (GDM). ETS Research Report Series (Vol. RR–14-40). Princeton, NJ. https://doi.org/10.1002/ets2.12043.CrossRefGoogle Scholar
Wang, W. -C., & Qiu, X. -L.(2019). Multilevel modeling of cognitive diagnostic assessment: The multilevel DINA model example. Applied Psychological Measurement,43,34–50.CrossRefGoogle Scholar
Wagenmakers, E-J, Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., & Morey, R. D.(2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin & Review,25,35–57.CrossRefGoogle ScholarPubMed
White, A., & Murphy, T. B.(2014). BayesLCA? An R package for Bayesian latent class. Journal of Statistical Software,61,1–28.CrossRefGoogle Scholar
Yamaguchi, K.(2020). Variational Bayesian inference for the multiple-choice DINA model. Behaviormetrika,47,159–187.CrossRefGoogle Scholar
Yamaguchi, K., & Okada, K.(2018). Comparison among cognitive diagnostic models for the TIMSS 2007 fourth grade mathematics assessment. PLoS ONE,13,e0188691.CrossRefGoogle ScholarPubMed
Yamaguchi, K., & Okada, K.(2020). Variational Bayes inference for the DINA Model. Journal of Educational and Behavioral Statistics,45(5), 569–597.CrossRefGoogle Scholar
Zhan, P., Jiao, H., Man, K., & Wang, L.(2019). Using JAGS for Bayesian cognitive diagnosis modeling: A tutorial. Journal of Educational and Behavioral Statistics,44,473–503.CrossRefGoogle Scholar
Zhan, P., Liao, M., & Bian, Y.(2018). Joint testlet cognitive diagnosis modeling for paired local item dependence in response times and response accuracy. Frontiers in Psychology,9(APR), 1–14.CrossRefGoogle ScholarPubMed
Supplementary material: File

Yamaguchi and Okada supplementary material

Yamaguchi and Okada supplementary material
Download Yamaguchi and Okada supplementary material(File)
File 84.4 KB