Hostname: page-component-745bb68f8f-cphqk Total loading time: 0 Render date: 2025-02-12T00:31:26.310Z Has data issue: false hasContentIssue false
  • English
  • Français

Maximum Likelihood Estimation of Nonlinear Structural Equation Models

Published online by Cambridge University Press:  01 January 2025

Sik-Yum Lee  and
Hong-Tu Zhu
Sik-Yum Lee*
Affiliation:
Department of Statistics, The Chinese University of Hong Kong
Hong-Tu Zhu
Affiliation:
Department of Mathematics and Statistics, University of Victoria
*
Requests for reprints should be sent to S. Y. Lee, Department of Statistics, The Chinese University of Hong Kong, Shatin, N.T. HONG KONG. E-Mail: sylee@sparc2.sta.cuhk.edu.hk
Get access Rights & Permissions [Opens in a new window]

Abstract

The existing maximum likelihood theory and its computer software in structural equation modeling are established based on linear relationships among manifest variables and latent variables. However, models with nonlinear relationships are often encountered in social and behavioral sciences. In this article, an EM type algorithm is developed for maximum likelihood estimation of a general nonlinear structural equation model. To avoid computation of the complicated multiple integrals involved, the E-step is completed by a Metropolis-Hastings algorithm. It is shown that the M-step can be completed efficiently by simple conditional maximization. Standard errors of the maximum likelihood estimates are obtained via Louis's formula. The methodology is illustrated with results from a simulation study and two real examples.

Keywords

nonlinear structural equation modelsmissing dataMCECM algorithmMetropolis-Hastings algorithmstandard errors estimates
Type
Articles
Information
Psychometrika , Volume 67 , Issue 2 , June 2002 , pp. 189 - 210
Copyright
Copyright © 2002 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The order of the authorship is alphabetical. This research is fully supported by a Hong Kong UGC Earmarked grant CUHK 4088/99H. The authors are indebted to the Editor, Associate Editor and anonymous reviewers for valuable comments for improving the paper; and also to ICPSR and the relevant funding agency for allowing use of the data. We thank Xin-Yuan Song, N.H. Tang and Liang Xu for helpful discussions. Assistance of Xin-Yuan Song and Michael K. H. Leung in analyzing the real examples and Esther L.S. Tam in preparing the manuscript is also acknowledged.

References

Anderson, T.W. (1989). Linear latent variable models and covariance structures. Journal of Econometrics, 41, 91–119.CrossRefGoogle Scholar
Arminger, G., & Muthén, B.O. (1998). A Bayesian approach to nonlinear latent variable models using the Gibbs sampler and the Metropolis-Hastings algorithm. Psychometrika, 63, 271–300.CrossRefGoogle Scholar
Bagozzi, R.P., Baumgartner, H., & Yi, Y. (1992). State versus action orientation and the theory of reasoned action: An application to coupon usage. Journal of Consumer Research, 18, 505–517.CrossRefGoogle Scholar
Bentler, P.M. (1983). Some contributions to efficient statistics for structural models: Specification and estimation of moment structures. Psychometrika, 48, 493–517.CrossRefGoogle Scholar
Bentler, P.M. (1992). EQS: Structural equation program manual. Los Angeles, CA: BMDP Statistical Software.Google Scholar
Bentler, P.M., & Dudgeon, P. (1996). Covariance structure analysis: Statistical practice, theory, and directions. Annual Review of Psychology, 47, 541–570.CrossRefGoogle Scholar
Berger, J.O., & Perrichi, L.R. (1996). The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association, 91, 109–122.CrossRefGoogle Scholar
Bollen, K.A., Paxton, P. (1998). Two-stage least squares estimation of interaction effects. In Schumacker, R.E., & Marcoulides, G.A. (Eds.), Interaction and nonlinear effects in structural equation models (pp. 125–151). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Booth, J.G., & Hobert, J.P. (1999). Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. Journal of the Royal Statistical Society, Series B, 61, 265–285.CrossRefGoogle Scholar
Browne, M.W. (1984). Asymptotically distribution-free methods in the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.CrossRefGoogle ScholarPubMed
Browne, M.W. (1987). Robustness of statistical inference in factor analysis and related models. Biometrika, 74, 375–384.CrossRefGoogle Scholar
Busemeyer, J.R., & Jones, L.E. (1983). Analysis of multiplicative combination rules when the causal variables are measured with error. Psychological Bulletin, 93, 549–562.CrossRefGoogle Scholar
Celeux, G., & Diebolt, J. (1989). The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Computational Statistics Quarterly, 2, 73–82.Google Scholar
Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1–38.CrossRefGoogle Scholar
Etezadi-Amoli, J., & McDonald, R.P. (1983). A second generation nonlinear factor analysis. Psychometrika, 48, 315–342.CrossRefGoogle Scholar
Fraser, C. (1980). COSAN user's guide. Toronto, Canada: The Ontario Institute for Studies in Education.Google Scholar
Gelman, A., & Meng, X.L. (1998). Simulating normalizing constants: From importance sampling to bridge sampling to path sampling. Statistical Science, 13, 163–185.CrossRefGoogle Scholar
Gelman, A., Roberts, G.O., & Gilks, W.R. (1995). Efficient Metropolis humping rules. In Bernardo, J.M., Berger, J.O., Dawid, A.P., & Smith, A.F.M. (Eds.), Bayesian statistics 5 (pp. 599–607). Oxford, England: Oxford University Press.Google Scholar
Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.CrossRefGoogle ScholarPubMed
Hastings, W.K. (1970). Monte Carlo sampling methods using Markov chains and their application. Biometrika, 57, 97–109.CrossRefGoogle Scholar
Hu, L., Bentler, P.M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted. Psychological Bulletin, 112, 351–362.CrossRefGoogle ScholarPubMed
Jaccard, J., & Wan, C.K. (1995). Measurement error in the analysis of interaction effects between continuous predictors using multiple regression: Multiple indicator and structural equation approaches. Psychological Bulletin, 117, 348–357.CrossRefGoogle Scholar
Jamshidian, M., & Jennrich, R.I. (1993). Conjugate gradient acceleration of the EM algorithm. Journal of the American Statistical Association, 88, 221–228.CrossRefGoogle Scholar
Jonsson, F.Y. (1998). Modeling interaction and non-linear effects: A step by step LISREL example. In Schumacker, R.E., & Marcoulides, G.A. (Eds.), Interaction and nonlinear effects in structural equation models (pp. 17–42). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Jöreskog, K.G., & Sörbom, D. (1996). LISREL 8: Structural equation modeling with the SIMPLIS command language. Hove and London, England: Scientific Software International.Google Scholar
Jöreskog, K.G., & Yang, F. (1996). Nonlinear structural equation models: The Kenny-Judd model with interaction effects. In Marcoulides, G.A., & Schumacker, R.E. (Eds.), Advanced structural equation modeling techniques (pp. 57–88). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Kass, R.E., & Raftery, A.E. (1995). Bayes Factors. Journal of the American Statistical Association, 90, 773–795.CrossRefGoogle Scholar
Kenny, D.A., & Judd, C.M. (1984). Estimating the nonlinear and interactive effects of latent variables. Psychological Bulletin, 96, 201–210.CrossRefGoogle Scholar
Klein, A., Moosbrugger, H., Schermelleh-Engel, K., & Frandk, D. (1997). A new approach to the estimation of latent interaction effects in structural equation models. In Bandilla, W., & Fanlbaum, F. (Eds.), SOFTSTAT '97—Advances in statistical software (pp. 479–488). Stuttgart, Germany: Lucius & Lucius.Google Scholar
Lange, K. (1995). A gradient algorithm locally equivalent to the EM algorithm. Journal of the Royal Statistical Association, Series B, 57, 425–437.CrossRefGoogle Scholar
Lee, S.Y., Poon, W.Y., & Bentler, P.M. (1995). A two-stage estimation of structural equation models with continuous and polytomous variables. British Journal of Mathematical and Statistical Psychology, 48, 339–358.CrossRefGoogle ScholarPubMed
Lee, S.Y., & Song, X.Y. (2001). Hypothesis testing and model comparison in two-level structural equation models. Multivariate Behavioral Research, 36, 639–655.CrossRefGoogle ScholarPubMed
Lee, S.Y., & Tsang, S.Y. (1999). Constrained maximum likelihood estimation of two-level covariance structure model via EM type algorithms. Psychometrika, 64, 435–450.CrossRefGoogle Scholar
Lee, S.Y., & Zhu, H.T. (2000). Statistical analysis of nonlinear structural equation model with continuous and polytomous data. British Journal of Mathematical and Statistical Psychology, 53, 209–232.CrossRefGoogle Scholar
Li, F., Harmer, P., Duncan, T.E., Duncan, S.C., Acock, A., & Boles, S. (1998). Approaches to testing interaction effects using structural equation modeling methodology. Multivariate Behavioral Research, 33, 1–39.CrossRefGoogle ScholarPubMed
Liu, C., & Rubin, D.B. (1998). Maximum likelihood estimation of factor analysis using the ECME algorithm with complete and incomplete data. Statistica Sinica, 8, 729–747.Google Scholar
Liu, J.S., Liang, F.M., & Wong, W.H. (2000). The use of multiple-try method and local optimization in metropolis sampling. Journal of the American Statistical Association, 95, 121–134.CrossRefGoogle Scholar
Louis, T.A. (1982). Finding the observed information matrix when using EM algorithm. Journal of the Royal Statistical Society, Series B, 44, 226–233.CrossRefGoogle Scholar
Marschner, I.C. (2001). On stochastic version of the EM algorithm. Biometrika, 88, 281–286.CrossRefGoogle Scholar
McDonald, R.P. (1962). A general approach to nonlinear factor analysis. Psychometrika, 27, 123–157.CrossRefGoogle Scholar
McDonald, R.P. (1967). Numerical methods for polynomial models in nonlinear factor analysis. Psychometrika, 32, 77–112.CrossRefGoogle Scholar
McDonald, R.P. (1967). Factor interaction in nonlinear factor analysis. British Journal of Mathematical and Statistical Psychology, 20, 205–215.CrossRefGoogle ScholarPubMed
Meng, X.L., Rubin, D.B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika, 80, 267–278.CrossRefGoogle Scholar
Meng, X.L., & Schilling, S. (1996). Fitting full-information item factor models and an empirical investigation of bridge sampling. Journal of the American Statistical Association, 91, 1254–1267.CrossRefGoogle Scholar
Meng, X.L., & van Dyk, D. (1997). The EM algorithm—An old folk song sung to a fast new tune (with discussion). Journal of the Royal Statistical Society, Series B, 59, 511–567.CrossRefGoogle Scholar
Meng, X.L., & Wong, W.H. (1996). Simulating ratios of normalizing constants via a simple identity: A theoretical exploration. Statistic Sinica, 6, 831–860.Google Scholar
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., & Teller, E. (1953). Equations of state calculations by fast computing machine. Journal of Chemical Physics, 21, 1087–1091.CrossRefGoogle Scholar
Mooijaart, A., & Bentler, P. (1986). Random polynomial factor analysis. In Diday, E., Jambu, M., Lebart, L., Pages, J., & Tomassone, R. (Eds.), Data analysis and informatics, IV (pp. 241–250). North-Holland: Elsevier Science Publishers.Google Scholar
Ping, R.A. (1996). Interaction and quadratic effect estimation: A two step technique using structural equation analysis. Psychological Bulletin, 119, 166–175.CrossRefGoogle Scholar
Ping, R.A. (1996). Latent variable regression: A technique for estimating interaction and quadratic coefficients. Multivariate Behavioral Research, 31, 95–120.CrossRefGoogle ScholarPubMed
Ping, R.A. (1996). Estimating latent variable interactions and quadratics: The state of this art. Journal of Management, 22, 163–183.Google Scholar
Raftery, A.E. (1993). Bayesian model selection in structural equation models. In Bollen, K.A., & Long, J.S. (Eds.), Testing structural equation models (pp. 163–180). Beverly Hills, CA: Sage.Google Scholar
Roberts, G.O. (1996). Markov Chain concepts related to sampling algorithms. In Gilks, W.R., Richardson, S., & Spiegelhalter, D.J. (Eds.), Markov chain Monte Carlo in practice (pp. 45–57). London, England: Chapman and Hall.Google Scholar
Rubin, D.B. (1991). EM and beyond. Psychometrika, 56, 241–254.CrossRefGoogle Scholar
Rubin, D.B., & Thayer, D.T. (1982). EM algorithm for ML factor analysis. Psychometrika, 47, 69–76.CrossRefGoogle Scholar
Schumacker, R.E., & Marcoulides, G.A. (1998). Interaction and nonlinear effects in structural equation models. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Shi, J.Q., & Lee, S.Y. (1998). Bayesian sampling-based approach for factor analysis model with continuous and polytomous data. British Journal of Mathematical and Statistical Psychology, 51, 233–252.CrossRefGoogle Scholar
Shi, J.Q., & Lee, S.Y. (2000). Latent variable models with mixed continuous and polytomous data. Journal of the Royal Statistical Society, Series B, 62, 77–87.CrossRefGoogle Scholar
Wei, G.C.G., & Tanner, M.A. (1990). A Monte Carlo implementation of the EM algorithm and the Poor man's data augmentation algorithm. Journal of the American Statistical Association, 85, 699–704.CrossRefGoogle Scholar
World Values Survey: 1981–1984 & 1990–1993. (1994). Ann Arbor, MI: Inter-University Consortium of Political and Social Research. (For the I.C.P.S.R. version the Institute for Social Research is the producer, and the Inter-University Consortium of Political and Social Research is the distributor.)Google Scholar