Hostname: page-component-745bb68f8f-mzp66 Total loading time: 0 Render date: 2025-01-08T15:28:36.913Z Has data issue: false hasContentIssue false
  • English
  • Français

Application of Local Linear Embedding to Nonlinear Exploratory Latent Structure Analysis

Published online by Cambridge University Press:  01 January 2025

Haonan Wang  and
Hari Iyer
Haonan Wang*
Affiliation:
Colorado State University
Hari Iyer
Affiliation:
Colorado State University
*
Requests for reprints should be sent to E-Mail: wanghn@stat.colostate.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper we discuss the use of a recent dimension reduction technique called Locally Linear Embedding, introduced by Roweis and Saul, for performing an exploratory latent structure analysis. The coordinate variables from the locally linear embedding describing the manifold on which the data reside serve as the latent variable scores. We propose the use of semiparametric penalized spline methods for reconstruction of the manifold equations that approximate the data space. We also discuss a crossvalidation strategy that can guide in selecting an appropriate number of latent variables. Synthetic as well as real data sets are used to illustrate the proposed approach. A nonlinear latent structure representation of a data set also serves as a data visualization tool.

Keywords

Locally Linear Embeddingfactor analysispenalized spline smoothingdimension reductionmanifold reconstructionprincipal components analysiscrossvalidation
Type
Original Paper
Information
Psychometrika , Volume 72 , Issue 2 , June 2007 , pp. 199 - 225
Copyright
Copyright © 2007 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bartlett, M.S. (1953). Factor analysis in psychology as a statistician sees it. In Uppsala symposium on psychological factor analysis, pp. 23–24. Stockholm: Almqvist and Wiksell.Google Scholar
Chen, L., & Buja, A. (2006). Local multidimensional scaling for nonlinear dimension reduction, graph layout and proximity analysis. Unpublished thesis, University of Pennsylvania, http://www-stat.wharton.upenn.edu/~buja/ PAPERS/lmds-chen-buja.pdf.Google Scholar
de Leeuw, J. (2005). Nonlinear principal component analysis. Department of Statistics, UCLA. Department of Statistics Papers. Paper 2005103001. http://repositories.cdlib.org/uclastat/papers/2005103001Google Scholar
Delicado, P., Huerta, M. (2003). Principal curves of oriented points: Theoretical and computational improvements. Computational Statistics, 18, 293–315.CrossRefGoogle Scholar
Donoho, D.L., & Grimes, C. (2003). Hessian eigenmaps: New locally linear embedding techniques for high-dimensional data. Technical Report TR-2003-08, Department of Statistics, Stanford University.CrossRefGoogle Scholar
Green, P.J., & Silverman, B.W. (1994). Nonparametric regression and generalized linear models: A roughness penality approach. London: Chapmman and Hall.CrossRefGoogle Scholar
Greenacre, M.J. (1984). Theory and applications of correspondence analysis. London: Academic Press.Google Scholar
Hill, M.O., Gauch, H.G. (1980). Detrended corresponsence analysis: An improved ordination technique. Vegetatio, 42, 47–58.CrossRefGoogle Scholar
Kammann, E.E., Wand, M.P. (2003). Geoadditive models. Journal of the Royal Statistical Society, Series C, 51, 1–18.CrossRefGoogle Scholar
Kaufman, L., & Rousseeuw, P.J. (1990). Finding groups in data: An introduction to cluster analysis. New York: Wiley.CrossRefGoogle Scholar
Kramer, M.A. (1991). Nonlinear principal components analysis using autoassociative neural networks. AIChE Journal, 37(2), 233–243.CrossRefGoogle Scholar
McDonald, R.P. (1965). Difficulty factors and nonlinear factor analysis. British Journal of Mathematical and Statistical Psychology, 18, 11–23.CrossRefGoogle Scholar
McDonald, R.P. (1967a). Numerical methods for polynomial models in nonlinear factor analysis. Psychometrika, 32, 77–112.CrossRefGoogle Scholar
McDonald, R.P. (1967b). Factor interaction in nonlinear factor analysis. British Journal of Mathematical and Statistical Psychology, 20, 205–215.CrossRefGoogle ScholarPubMed
Ngo, L., & Wand, M.P. (2004). Smoothing with mixed model software. Journal of Statistical Software, 9.CrossRefGoogle Scholar
Ramsay, J.O., & Silverman, S.W. (2002). Applied functional data analysis. New York: Springer-Verlag.Google Scholar
Reyment, R.A., & Joreskog, K.G. (1993). Applied factor analysis in the natural sciences. New York: Cambridge University Press.CrossRefGoogle Scholar
Roweis, S.T., Saul, L.K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326.CrossRefGoogle ScholarPubMed
Ruppert, D., Wand, M.P., & Carroll, R.J. (2003). Semiparametric regression. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Saul, L.K., Roweis, S.T. (2003). Think globally, fit locally: Unsupervised learning of nonlinear manifolds. Journal of Machine Learning Research, 4, 119–155.Google Scholar
Tenenbaum, J.B., de Silva, V., Langford, J.C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290, 2319–2323.CrossRefGoogle ScholarPubMed
Wand, M.P., French, J.L., Ganguli, B., Kammann, E.E., Staudenmayer, J., & Zanobetti, A. (2005). {SemiPar 1.0}. R package. http://cran.r-project.org.Google Scholar
Wang, H. (2005). Internet site: http://www.stat.colostate.edu/~wanghn/nlfa.htm.Google Scholar
Yalcin, I., Amemiya, Y. (2001). Nonlinear factor analysis as a statistical method. Statistical Sciences, 16, 275–294.Google Scholar