Hostname: page-component-745bb68f8f-b6zl4 Total loading time: 0 Render date: 2025-01-07T10:17:38.957Z Has data issue: false hasContentIssue false
  • English
  • Français

Asymptotics of AIC, BIC, and RMSEA for Model Selection in Structural Equation Modeling

Published online by Cambridge University Press:  01 January 2025

Po-Hsien Huang
Po-Hsien Huang*
Affiliation:
National Cheng Kung University
*
Correspondence should be made to Po-Hsien Huang, Department of Psychology, National Cheng Kung University, No.1, University Road, Tainan City, 701, Taiwan. Email: psyphh@mail.ncku.edu.tw
Get access Rights & Permissions [Opens in a new window]

Abstract

Model selection is a popular strategy in structural equation modeling (SEM). To select an “optimal” model, many selection criteria have been proposed. In this study, we derive the asymptotics of several popular selection procedures in SEM, including AIC, BIC, the RMSEA, and a two-stage rule for the RMSEA (RMSEA-2S). All of the results are derived under weak distributional assumptions and can be applied to a wide class of discrepancy functions. The results show that both AIC and BIC asymptotically select a model with the smallest population minimum discrepancy function (MDF) value regardless of nested or non-nested selection, but only BIC could consistently choose the most parsimonious one under nested model selection. When there are many non-nested models attaining the smallest MDF value, the consistency of BIC for the most parsimonious one fails. On the other hand, the RMSEA asymptotically selects a model that attains the smallest population RMSEA value, and the RESEA-2S chooses the most parsimonious model from all models with the population RMSEA smaller than the pre-specified cutoff. The empirical behavior of the considered criteria is also illustrated via four numerical examples.

Keywords

structural equation modelingAkaike information criterionBayesian information criterionroot-mean-square error of approximationmodel selection
Type
Original Paper
Information
Psychometrika , Volume 82 , Issue 2 , June 2017 , pp. 407 - 426
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s11336-017-9572-y) contains supplementary material, which is available to authorized users.

References

Akaike, H. (1974). A new look at statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723CrossRefGoogle Scholar
Bentler, P. M., & Weeks, D. G. (1980). Linear structural equations with latent variables. Psychometrika, 45, 289–308CrossRefGoogle Scholar
Bollen, K. A., & Harden, J. J., & Ray, S., & Zavisca, J. (2014). BIC and alternative Bayesian information criteria in the selection of structural equation models. Structural Equation Modeling, 21, 1–19CrossRefGoogle ScholarPubMed
Bozdogan, H. (1987). Model selection and Akaike’s information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52, 345–370CrossRefGoogle Scholar
Browne, M. W. (1974). Generalized least squares estimators in the analysis of covariance structures. South African Statistical Journal, 8, 1–24.Google Scholar
Browne, M. W. (1984). Asymptotic distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83CrossRefGoogle ScholarPubMed
Browne, M. W., & Cudeck, R. Bollen, K. A., & Long, J. S. (1993). Alternative ways of assessing model fit. Testing structural equation models, Newbury Park, CA: Sage 136–162.Google Scholar
Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach, 2New York, NY: Springer.Google Scholar
Cudeck, R., & Henly, S. J. (1991). Model selection in covariance structures analysis and the problem of sample size: A clarification. Psychological Bulletin, 109, 512–519CrossRefGoogle ScholarPubMed
Dziak, J. J., Coffman, D. L., Lanza, S. T., & Li, R. (2012). Sensitivity and specificity of information criteria (Tech. Rep. No. 12–119). University Park, PA: The Pennsylvania State University, The Methodology Center.Google Scholar
Feist, G. J., & Bodner, T. E., & Jacobs, J. F., & Miles, M., & Tan, V. (1995). Integrating top-down and bottom-up structural models of subjective well-being: A longitudinal investigation. Journal of Personality and Social Psychology, 68, 138–150CrossRefGoogle Scholar
Fleishman, A. I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521–532CrossRefGoogle Scholar
Hannan, E. J., & Quinn, B. G. (1979). The determination of the order of an autoregression. Journal of the Royal Statistical Society, Series B, 41, 190–195.CrossRefGoogle Scholar
Haughton, D. M. A. (1988). On the choice of a model to fit data from an exponential family. Annals of Statistics, 16, 342–355CrossRefGoogle Scholar
Haughton, D. M. A., & Oud, J. H. L., & Jansen, RARG (1997). Information and other criteria in structural equation model selection. Communication in Statistics. Part B: Simulation and Computation, 26, 1477–1516.Google Scholar
Homburg, C. (1991). Cross-validation and information criteria in causal modeling. Journal of Marketing Research, 28, 137–144CrossRefGoogle Scholar
Ibrahim, J. G., & Zhu, H-T, & Tang, N-S (2008). Model selection criteria for missing-data problems using the EM algorithm. Journal of the American Statistical Association, 103, 1648–1658CrossRefGoogle ScholarPubMed
Jackson, D. L., & Gillaspy, J. A. JrPurc-Stephenson, R. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14, 6–23CrossRefGoogle ScholarPubMed
Jöreskog, K. G. Bollen, K. A., & Lang, J. S. (1993). Testing structural equation models. Testing structural equation models, Newbury Park, CA: Sage 294–316.Google Scholar
Kaplan, D. (2009). Structural Equation Modeling: Foundations and Extensions, 2Newbury Park, CA: SAGE PublicationsCrossRefGoogle Scholar
Keyes, C. L. M., & Shmotkin, D., & Ryff, C. D. (2002). Optimizing well-being: The empirical encounter of two traditions. Journal of Personality and Social Psychology, 82, 1007–1022CrossRefGoogle ScholarPubMed
Kullback, S., & Leibler, R. A. (1951). On Information and Sufficiency. Annals of Mathematical Statistics, 22, 79–86CrossRefGoogle Scholar
Li, L. & Bentler, P. M. (2006). Robust statistical tests for evaluating the hypothesis of close fit of misspecified mean and covariance structural models. UCLA Statistics Preprint #494.Google Scholar
MacCallum, R. C. (2003). Working with imperfect models. Multivariate Behavioral Research, 38, 113–139CrossRefGoogle ScholarPubMed
MacCallum, R. C., & Austin, J. T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201–224CrossRefGoogle ScholarPubMed
Mallows, C. L. (1973). Some comments on Cp\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_p $$\end{document}. Technometrics, 15, 661–675.Google Scholar
McDonald, R. P. (2010). Structural models and the art of approximation. Perspectives on Psychological Science, 5, 675–686CrossRefGoogle ScholarPubMed
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166CrossRefGoogle Scholar
Pitt, M. A., & Myung, I., & Zhang, S. (2002). Toward a method of selecting among computational models of cognition. Psychological Review, 109, 472–491CrossRefGoogle Scholar
Preacher, K. J. (2006). Quantifying parsimony in structural equation modeling. Multivariate Behavioral Research, 41, 227–259CrossRefGoogle ScholarPubMed
Preacher, K. J., & Merkle, E. C. (2012). The problem of model selection uncertainty in structural equation modeling. Psychologcial Methods, 17, 1–14CrossRefGoogle ScholarPubMed
Preacher, K. J., & Zhang, G., & Kim, C., & Mels, G. (2013). Choosing the optimal number of factors in exploratory factor analysis: A model selection perspective. Multivariate Behavioral Research, 48, 28–56CrossRefGoogle ScholarPubMed
Satorra, A. (1989). Alternative test criteria in covariance structure analysis—A unified approach. Psychometrika, 54, 131–151CrossRefGoogle Scholar
Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507–514CrossRefGoogle Scholar
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464CrossRefGoogle Scholar
Sclove, S. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52, 333–343CrossRefGoogle Scholar
Shah, R., & Goldstein, S. M. (2006). Use of structural equation modeling in operations management research: Looking back and forward. Journal of Operations Management, 24, 148–169CrossRefGoogle Scholar
Shao, J. (1997). An asymptotic theory for model selection. Statistics Sinica, 7, 221–264.Google Scholar
Shapiro, A. (1983). Asymptotic distribution theory in the analysis of covariance structures (a unified approach). South African Statistical Journal, 17, 33–81.Google Scholar
Shapiro, A. (1984). A note on the consistency of estimators in the analysis of moment structures. British Journal of Mathematical and Statistical Psychology, 1984, 84–88CrossRefGoogle Scholar
Shapiro, A. Lee, S-Y (2007). Statistical inference of moment structures. Handbook of latent variable and related models, Amsterdam: Elsevier 229–260.Google Scholar
Shapiro, A. (2009). Asymptotic normality of test statistics under alternative hypotheses. Journal of Multivariate Analysis, 100, 936–945CrossRefGoogle Scholar
Steiger, J. H., & Lind, J. C. (1980). Statistically-based tests for the number of common factors, Iowa City, IA: Paper presented at the annual Spring Meeting of the Psychometric Society.Google Scholar
Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of Royal Statistical Society, Series B, 36, 111–147.CrossRefGoogle Scholar
Vrieze, S. I. (2012). Model selection and psychological theory: A discussion of the differences between Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychological Methods, 17, 228–243CrossRefGoogle ScholarPubMed
Vuong, Q. H. (1989). Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica, 57, 307–333CrossRefGoogle Scholar
Wahba, G. (1990). Spline Models for Observational Data, Philadelphia: SIAMCrossRefGoogle Scholar
West, S. G., & Taylor, A. B., & Wu, W. Hoyle, R. H. (2012). Model fit and model selection in structural equation modeling. Handbook of Structural Equation Modeling, New York: Guilford Press.Google Scholar
White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica, 50, 1–25CrossRefGoogle Scholar
Supplementary material: File

Huang supplementary material

Huang supplementary material
Download Huang supplementary material(File)
File 26.9 KB