Hostname: page-component-5f745c7db-hj587 Total loading time: 0 Render date: 2025-01-06T21:40:36.009Z Has data issue: true hasContentIssue false
  • English
  • Français

Estimating Finite Mixtures of Ordinal Graphical Models

Published online by Cambridge University Press:  01 January 2025

Kevin H. Lee ,
Qian Chen ,
Wayne S. DeSarbo  and
Lingzhou Xue Open the ORCID record for Lingzhou Xue [Opens in a new window]
Kevin H. Lee
Affiliation:
Western Michigan University
Qian Chen
Affiliation:
University of Nebraska–Lincoln
Wayne S. DeSarbo
Affiliation:
Pennsylvania State University
Lingzhou Xue*
Affiliation:
Pennsylvania State University
*
Correspondence should be made to Lingzhou Xue, Department of Statistics, Pennsylvania State University, 318 Thomas Building, University Park, PA 16802, USA. Email: lzxue@psu.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Graphical models have received an increasing amount of attention in network psychometrics as a promising probabilistic approach to study the conditional relations among variables using graph theory. Despite recent advances, existing methods on graphical models usually assume a homogeneous population and focus on binary or continuous variables. However, ordinal variables are very popular in many areas of psychological science, and the population often consists of several different groups based on the heterogeneity in ordinal data. Driven by these needs, we introduce the finite mixture of ordinal graphical models to effectively study the heterogeneous conditional dependence relationships of ordinal data. We develop a penalized likelihood approach for model estimation, and design a generalized expectation-maximization (EM) algorithm to solve the significant computational challenges. We examine the performance of the proposed method and algorithm in simulation studies. Moreover, we demonstrate the potential usefulness of the proposed method in psychological science through a real application concerning the interests and attitudes related to fan avidity for students in a large public university in the United States.

Keywords

Gaussian mixture modelGaussian graphical modelordinal datalatent variablesnetwork psychometricsEM algorithm
Type
Application Reviews and Case Studies
Information
Psychometrika , Volume 87 , Issue 1: Special Issue on Network Psychometrics in Action , March 2022 , pp. 83 - 106
Copyright
Copyright © 2021 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s11336-021-09781-2.

References

Albert, J. H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88 (422), 669– 679. CrossRefGoogle Scholar
Allman, E. S., Matias, C., & Rhodes, J. A. (2009). Identifiability of parameters in latent structure models with many observed variables. The Annals of Statistics, 37 (6A), 3099– 3132. CrossRefGoogle Scholar
Amemiya, T. (1974). Bivariate probit analysis: Minimum chi-square methods. Journal of the American Statistical Association, 69 (348), 940– 944. CrossRefGoogle Scholar
Balakrishnan, S., Wainwright, M. J., & Yu, B. (2017). Statistical guarantees for the EM algorithm: From population to sample-based analysis. The Annals of Statistics, 45 (1), 77– 120. CrossRefGoogle Scholar
Bock, R. D., & Gibbons, R. D. (1996). High-dimensional multivariate probit analysis. Biometrics, 52 (4), 1183– 1194. CrossRefGoogle ScholarPubMed
Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64 (9), 1089– 1108. CrossRefGoogle ScholarPubMed
Borsboom, D., & Cramer, A. O. (2013). Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9 91– 121. CrossRefGoogle Scholar
Borsboom, D., & Molenaar, D. (2015). Psychometrics. International Encyclopedia of the Social & Behavioral Sciences, 19 (2), 418– 422. CrossRefGoogle Scholar
Breen, R., & Luijkx, R. (2010). Mixture models for ordinal data. Sociological Methods & Research, 39 (1), 3– 24. CrossRefGoogle Scholar
Brusco, M. J., Steinley, D., Hoffman, M., Davis-Stober, C., & Wasserman, S. (2019). On Ising models and algorithms for the construction of symptom networks in psychopathological research. Psychological Methods, 24 (6), 735– 753. CrossRefGoogle ScholarPubMed
Cai, T., Liu, W., & Luo, X. A constrained ℓ 1 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _1$$\end{document} minimization approach to sparse precision matrix estimation. (2011). Journal of the American Statistical Association, 106 (494), 594– 607. CrossRefGoogle Scholar
Chen, S., Witten, D. M., & Shojaie, A. (2015). Selection and estimation for mixed graphical models. Biometrika, 102 (1), 47– 64. CrossRefGoogle ScholarPubMed
Cheng, J., Li, T., Levina, E., & Zhu, J. (2017). High-dimensional mixed graphical models. Journal of Computational and Graphical Statistics, 26 (2), 367– 378. CrossRefGoogle Scholar
Chib, S., & Greenberg, E. (1998). Analysis of multivariate probit models. Biometrika, 85 (2), 347– 361. CrossRefGoogle Scholar
Cox, D. R., & Wermuth, N. (1994). A note on the quadratic exponential binary distribution. Biometrika, 81 (2), 403– 408. CrossRefGoogle Scholar
Cramer, A. O., Waldorp, L. J., Van Der Maas, H. L., & Borsboom, D. (2010). Comorbidity: A network perspective. Behavioral and Brain Sciences, 33 (2–3), 137– 150. CrossRefGoogle ScholarPubMed
Dalege, J., Borsboom, D., van Harreveld, F., van den Berg, H., Conner, M., & van der Maas, H. L. (2016). Toward a formalized account of attitudes: The causal attitude network (can) model. Psychological Review, 123 (1), 2– CrossRefGoogle Scholar
Danaher, P., Wang, P., & Witten, D. M. (2014). The joint graphical lasso for inverse covariance estimation across multiple classes. Journal of the Royal Statistical Society: Series B, 76 (2), 373– 397. CrossRefGoogle ScholarPubMed
Dayton, C. M., & Macready, G. B. (1988). Concomitant-variable latent-class models. Journal of the American Statistical Association, 83 (401), 173– 178. CrossRefGoogle Scholar
Dempster, A. P. (1972). Covariance selection. Biometrics, 28 157– 175. CrossRefGoogle Scholar
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B, 39 (1), 1– 22. CrossRefGoogle Scholar
DeSarbo, W. S. (2010). A spatial multidimensional unfolding choice model for examining the heterogeneous expressions of sports fan avidity. Journal of Quantitative Analysis in Sports, 6 (2), 1– 24. CrossRefGoogle Scholar
DeSarbo, W. S., Chen, Q., & Blank, A. S. (2017). A parametric constrained segmentation methodology for application in sport marketing. Customer Needs and Solutions, 4 (4), 37– 55. CrossRefGoogle Scholar
Dwivedi, R., Ho, N., Khamaru, K., Wainwright, M. J. & Jordan, M. I. (2018). Theoretical guarantees for the EM algorithm when applied to mis-specified gaussian mixture models. In Proceedings of the 32nd international conference on neural information processing systems (pp. 9704–9712).Google Scholar
Epskamp, S., Maris, G., Waldorp, L. J. & Borsboom, D. (2018). Network psychometrics. The Wiley handbook of psychometric testing: A multi-disciplinary reference on survey, scale and test development (pp. 953–986).Google Scholar
Epskamp, S., Rhemtulla, M., & Borsboom, D. (2017). Generalized network psychometrics: Combining network and latent variable models. Psychometrika, 82 (4), 904– 927. CrossRefGoogle ScholarPubMed
Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American statistical Association, 96 (456), 1348– 1360. CrossRefGoogle Scholar
Fan, J., Liu, H., Ning, Y., & Zou, H. (2017). High dimensional semiparametric latent graphical model for mixed data. Journal of the Royal Statistical Society: Series B, 79 (2), 405– 421. CrossRefGoogle Scholar
Fan, J., Xue, L., & Zou, H. (2014). Strong oracle optimality of folded concave penalized estimation. The Annals of Statistics, 42 (3), 819– CrossRefGoogle ScholarPubMed
Feng, H., & Ning, Y. (2019). High-dimensional mixed graphical model with ordinal data: Parameter estimation and statistical inference. In The 22nd international conference on artificial intelligence and statistics (pp. 654–663).Google Scholar
Fried, E. I., Bockting, C., Arjadi, R., Borsboom, D., Amshoff, M., & Cramer, A. O. et al (2015). From loss to loneliness: The relationship between bereavement and depressive symptoms. Journal of Abnormal Psychology, 124 (2), 256– CrossRefGoogle ScholarPubMed
Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9 (3), 432– 441. CrossRefGoogle ScholarPubMed
Greene, W. H., & Hensher, D. A. (2003). A latent class model for discrete choice analysis: Contrasts with mixed logit. Transportation Research Part B: Statistical Methodology, 37 (8), 681– 698. CrossRefGoogle Scholar
Grün, B., & Leisch, F. (2008). Finite mixtures of generalized linear regression models. In Recent advances in linear models and related areas (pp. 205–230). Springer.CrossRefGoogle Scholar
Guo, J., Levina, E., Michailidis, G., & Zhu, J. (2011). Joint estimation of multiple graphical models. Biometrika, 98 (1), 1– 15. CrossRefGoogle ScholarPubMed
Guo, J., Levina, E., Michailidis, G., & Zhu, J. (2015). Graphical models for ordinal data. Journal of Computational and Graphical Statistics, 24 (1), 183– 204. CrossRefGoogle ScholarPubMed
Haslbeck, J. M., & Waldorp, L. J. (2016). mgm: Structure estimation for time-varying mixed graphical models in high-dimensional data 30, 39–81. arXiv:1510.06871 Google Scholar
Höfling, H., & Tibshirani, R. (2009). Estimation of sparse binary pairwise Markov networks using pseudo-likelihoods. Journal of Machine Learning Research, 10 883– 906. Google ScholarPubMed
Huang, T., Peng, H., & Kun, Z. (2017). Model selection for Gaussian mixture models. Statistica Sinica, 27 (1), 147– 169. Google Scholar
Ising, E. (1925). Beitrag zur theorie des ferromagnetismus. Zeitschrift für Physik, 31 (1), 253– 258. CrossRefGoogle Scholar
Isvoranu, A.-M., van Borkulo, C. D., Boyette, L.-L., Wigman, J. T., Vinkers, C. H., Borsboom, D., & Investigators, G. (2016). A network approach to psychosis: Pathways between childhood trauma and psychotic symptoms. Schizophrenia Bulletin, 43 (1), 187– 196. CrossRefGoogle ScholarPubMed
Lafit, G., Tuerlinckx, F., Myin-Germeys, I., & Ceulemans, E. (2019). A partial correlation screening approach for controlling the false positive rate in sparse Gaussian graphical models. Scientific Reports, 9 (1), 1– 24. CrossRefGoogle ScholarPubMed
Lauritzen, S. L. (1996). Graphical models, Oxford Clarendon Press CrossRefGoogle Scholar
Lee, J. D., & Hastie, T. J. (2015). Learning the structure of mixed graphical models. Journal of Computational and Graphical Statistics, 24 (1), 230– 253. CrossRefGoogle ScholarPubMed
Lee, K. H., & Xue, L. (2018). Nonparametric finite mixture of Gaussian graphical models. Technometrics, 60 (4), 511– 521. CrossRefGoogle Scholar
Liu, H., Lafferty, J., & Wasserman, L. (2009). The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research, 10 2295– 2328. Google Scholar
Lwin, T., & Martin, P. (1989). Probits of mixtures. Biometrics, 45 (3), 721– 732. CrossRefGoogle ScholarPubMed
Ma, S., Xue, L., & Zou, H. (2021). Alternating direction methods for latent variable Gaussian graphical model selection. Neural Computation, 25 (8), 2172– 2198. CrossRefGoogle Scholar
Marsman, M. (2019). The idiographic ising model. PsyArXiv Preprints https://psyarxiv.com/h3ka5.Google Scholar
Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., & Waldorp, L. et al (2018). An introduction to network psychometrics: Relating Ising network models to item response theory models. Multivariate Behavioral Research, 53 (1), 15– 35. CrossRefGoogle ScholarPubMed
Marsman, M., Waldorp, L., & Borsboom, D. (2019). Towards an encompassing theory of network models. PsyArXiv Preprints https://psyarxiv.com/n98qt.Google Scholar
Meinshausen, N., & Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The Annals of Statistics, 34 (3), 1436– 1462. CrossRefGoogle Scholar
Meinshausen, N., & Bühlmann, P. (2010). Stability selection. Journal of the Royal Statistical Society: Series B, 72 (4), 417– 473. CrossRefGoogle Scholar
Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66 (336), 846– 850. CrossRefGoogle Scholar
Ravikumar, P., Wainwright, M. J., & Lafferty, J. D. High-dimensional Ising model selection using ℓ 1 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _1$$\end{document} -regularized logistic regression. (2010). The Annals of Statistics, 38 (3), 1287– 1319. Google Scholar
Ruan, L., Yuan, M., & Zou, H. (2011). Regularized parameter estimation in high-dimensional Gaussian mixture models. Neural Computation, 23 (6), 1605– 1622. CrossRefGoogle ScholarPubMed
Schmittmann, V. D., Cramer, A. O., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: A network perspective on psychological phenomena. New Ideas in Psychology, 31 (1), 43– 53. CrossRefGoogle Scholar
Städler, N., Bühlmann, P., & Van De Geer, S. ℓ 1 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _1$$\end{document} -penalization for mixture regression models. (2010). Test, 19 (2), 209– 256. CrossRefGoogle Scholar
Suggala, A. S., Yang, E., & Ravikumar, P. (2017). Ordinal graphical models: A tale of two approaches. In International conference on machine learning (pp. 3260–3269).Google Scholar
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B, 58 (1), 267– 288. CrossRefGoogle Scholar
von Davier, M., & Carstensen, C. H. (2007). Multivariate and mixture distribution Rasch models: Extensions and applications, Berlin Springer CrossRefGoogle Scholar
Wedel, M. (2002). Concomitant variables in finite mixture models. Statistica Neerlandica, 56 (3), 362– 375. CrossRefGoogle Scholar
Wedel, M., & DeSarbo, W. S. (1995). A mixture likelihood approach for generalized linear models. Journal of Classification, 12 (1), 21– 55. CrossRefGoogle Scholar
Xue, L., & Zou, H. (2012). Regularized rank-based estimation of high-dimensional nonparanormal graphical models. The Annals of Statistics, 40 (5), 2541– 2571. CrossRefGoogle Scholar
Xue, L., & Zou, H., Cai, T. (2012). Nonconcave penalized composite conditional likelihood estimation of sparse Ising models. The Annals of Statistics, 40 (3), 1403– 1429. CrossRefGoogle Scholar
Yao, W. (2015). Label switching and its solutions for frequentist mixture models. Journal of Statistical Computation and Simulation, 85 (5), 1000– 1012. CrossRefGoogle Scholar
Yuan, M., & Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika, 94 (1), 19– 35. CrossRefGoogle Scholar
Supplementary material: File

Lee et. al supplementary material

Lee et. al supplementary material
Download Lee et. al supplementary material(File)
File 304.2 KB