Hostname: page-component-745bb68f8f-lrblm Total loading time: 0 Render date: 2025-01-08T04:12:53.671Z Has data issue: false hasContentIssue false
  • English
  • Français

Regularized Multiple-Set Canonical Correlation Analysis

Published online by Cambridge University Press:  01 January 2025

Yoshio Takane ,
Heungsun Hwang  and
Hervé Abdi
Yoshio Takane*
Affiliation:
McGill University
Heungsun Hwang
Affiliation:
McGill University
Hervé Abdi
Affiliation:
University of Texas at Dallas
*
Requests for reprints should be sent to Yoshio Takane, Department of Psychology, McGill University, 1205 Dr. Penfield Avenue, Montreal, QC, H3A 1B1 Canada. E-mail: takane@psych.mcgill.ca
Get access Rights & Permissions [Opens in a new window]

Abstract

Multiple-set canonical correlation analysis (Generalized CANO or GCANO for short) is an important technique because it subsumes a number of interesting multivariate data analysis techniques as special cases. More recently, it has also been recognized as an important technique for integrating information from multiple sources. In this paper, we present a simple regularization technique for GCANO and demonstrate its usefulness. Regularization is deemed important as a way of supplementing insufficient data by prior knowledge, and/or of incorporating certain desirable properties in the estimates of parameters in the model. Implications of regularized GCANO for multiple correspondence analysis are also discussed. Examples are given to illustrate the use of the proposed technique.

Keywords

information integrationprior informationridge regressiongeneralized singular value decomposition (GSVD)G-fold cross validationpermutation teststhe Bootstrap methodmultiple correspondence analysis (MCA)
Type
Original Paper
Information
Psychometrika , Volume 73 , Issue 4 , December 2008 , pp. 753 - 775
Copyright
Copyright © 2008 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The work reported in this paper is supported by Grants 10630 and 290439 from the Natural Sciences and Engineering Research Council of Canada to the first and the second authors, respectively. The authors would like to thank the two editors (old and new), the associate editor, and four anonymous reviewers for their insightful comments on earlier versions of this paper. Matlab programs that carried out the computations reported in the paper are available upon request.

References

Abdi, H. (2007). Singular value decomposition (SVD) and generalized singular decomposition (GSVD). In Salkind, N.J. (Eds.), Encyclopedia of measurement and statistics (pp. 907–912). Thousand Oak: Sage.Google Scholar
Abdi, H., & Valentin, D. (2007). The STATIS method. In Salkind, N.J. (Eds.), Encyclopedia of measurement and statistics (pp. 955–962). Thousand Oaks: Sage.Google Scholar
Adachi, K. (2002). Homogeneity and smoothness analysis for quantifying a longitudinal categorical variable. In Nishisato, S., Baba, Y., Bozdogan, H., & Kanefuji, K. (Eds.), Measurement and multivariate analysis (pp. 47–56). Tokyo: Springer.CrossRefGoogle Scholar
Beck, A. (1996). BDI-II, San Antonio: Psychological Corporation.Google Scholar
Carroll, J.D. (1968). A generalization of canonical correlation analysis to three or more sets of variables. In Proceedings of the 76th annual convention of the American psychological association (pp. 227–228).Google Scholar
Dahl, T., & Næs, T. (2006). A bridge between Tucker-1 and Carroll’s generalized canonical analysis. Computational Statistics and Data Analysis, 50, 3086–3098.CrossRefGoogle Scholar
de Leeuw, J. (1982). Generalized eigenvalue problems with posivite semi-definite matrices. Psychometrika, 47, 87–93.CrossRefGoogle Scholar
Devaux, M.-F., Courcoux, P., Vigneau, E., & Novales, B. (1998). Generalized canonical correlation analysis for the interpretation of fluorescence spectral data. Analusis, 26, 310–316.CrossRefGoogle Scholar
DiPillo, P.J. (1976). The application of bias to discriminant analysis. Communications in Statistics—Theory and Methods, 5, 843–859.CrossRefGoogle Scholar
Efron, B. (1979). Bootstrap methods: another look at the jackknife. Annals of Statistics, 7, 1–26.CrossRefGoogle Scholar
Fischer, B., Ross, V., & Buhmann, J.M. (2007). Time-series alignment by non-negative multiple generalized canonical correlation analysis. In Massuli, F., & Mitra, S. (Eds.), Applications of fuzzy set theory (pp. 505–511). Berlin: Springer.CrossRefGoogle Scholar
Friedman, J. (1989). Regularized discriminant analysis. Journal of the American Statistical Association, 84, 165–175.CrossRefGoogle Scholar
Gardner, S., Gower, J.C., & le Roux, N.J. (2006). A synthesis of canonical variate analysis, generalized canonical correlation and Procrustes analysis. Computational Statistics and Data Analysis, 50, 107–134.CrossRefGoogle Scholar
Gifi, A. (1990). Nonlinear multivariate analysis, Chichester: Wiley.Google Scholar
Greenacre, M.J. (1984). Theory and applications of correspondence analysis, London: Academic Press.Google Scholar
Hoerl, A.F., & Kennard, R.W. (1970). Ridge regression: biased estimation for nonorthgonal problems. Technometrics, 12, 55–67.CrossRefGoogle Scholar
Horst, P. (1961). Generalized canonical correlations and their applications to experimental data. Journal of Clinical Psychology, 17, 331–347.3.0.CO;2-D>CrossRefGoogle ScholarPubMed
Kroonenberg, P.M. (2008). Applied multiway data analysis, New York: Wiley.CrossRefGoogle Scholar
Legendre, P., & Legendre, L. (1998). Numerical ecology, Amsterdam: North Holland.Google Scholar
Maraun, M., Slaney, K., & Jalava, J. (2005). Dual scaling for the analysis of categorical data. Journal of Personality Assessment, 85, 209–217.CrossRefGoogle ScholarPubMed
Meredith, W. (1964). Rotation to achieve factorial invariance. Psychometrika, 29, 187–206.CrossRefGoogle Scholar
Poggio, T., & Girosi, F. (1990). Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247, 978–982.CrossRefGoogle ScholarPubMed
Ramsay, J.O., & Silverman, B.W. (2005). Functional data analysis, (2nd ed.). New York: Springer.CrossRefGoogle Scholar
Rao, C.R., & Mitra, S.K. (1971). Generalized inverse of matrices and its applications, New York: Wiley.Google Scholar
Smilde, A., Bro, R., & Geladi, P. (2004). Multi-way analysis: applications in the chemical sciences, New York: Wiley.CrossRefGoogle Scholar
Sun, Q.-S., Heng, P.-A., Jin, Z., & Xia, D.-S. (2005). Face recognition based on generalized canonical correlation analysis. In Huang, D.S., Zhang, X.-P., & Huang, G.-B. (Eds.), Advances in intelligent computing (pp. 958–967). Berlin: Springer.CrossRefGoogle Scholar
Takane, Y. (1980). Analysis of categorizing behavior by a quantification method. Behaviormetrika, 8, 75–86.CrossRefGoogle Scholar
Takane, Y., & Hunter, M.A. (2001). Constrained principal component analysis: a comprehensive theory. Applicable Algebra in Engineering, Communication and Computing, 12, 391–419.CrossRefGoogle Scholar
Takane, Y., & Hwang, H. (2002). Generalized constrained canonical correlation analysis. Multivariate Behavioral Research, 37, 163–195.CrossRefGoogle Scholar
Takane, Y., & Hwang, H. (2006). Regularized multiple correspondence analysis. In Greenacre, M.J., & Blasius, J. (Eds.), Multiple correspondence analysis and related methods (pp. 259–279). London: Chapman and Hall.CrossRefGoogle Scholar
Takane, Y., & Hwang, H. (2007). Regularized linear and kernel redundancy analysis. Computational Statistics and Data Analysis, 52, 392–405.CrossRefGoogle Scholar
Takane, Y., & Jung, S. (in press). Regularized partial and/or constrained redundancy analysis. Psychometrika. DOI: 10.1007/s11336-008-9067-y.CrossRefGoogle Scholar
Takane, Y., & Oshima-Takane, Y. (2002). Nonlinear generalized canonical correlation analysis by neural network models. In Nishisato, S., Baba, Y., Bozdogan, H., & Kanefuji, K. (Eds.), Measurement and multivariate analysis (pp. 183–190). Tokyo: Springer.CrossRefGoogle Scholar
Takane, Y., & Yanai, H. (2008). On ridge operators. Linear Algebra and Its Applications, 428, 1778–1790.CrossRefGoogle Scholar
ten Berge, J.M.F. (1979). On the equivalence of two oblique congruence rotation methods, and orthogonal approximations. Psychometrika, 44, 359–364.CrossRefGoogle Scholar
ter Braak, C.J.F. (1990). Update notes: CANOCO Version 3.10, Wageningen: Agricultural Mathematics Group.Google Scholar
Tikhonov, A.N., & Arsenin, V.Y. (1977). Solutions of ill-posed problems, Washington: Winston.Google Scholar
Tillman, B., Dowling, J., & Abdi, H. (2008). Bach, Mozart or Beethoven? Indirect investigations of musical style perception with subjective judgments and sorting tasks. In preparation.Google Scholar
van de Velden, M., & Bijmolt, T.H.A. (2006). Generalized canonical correlation analysis of matrices with missing rows: a simulation study. Psychometrika, 71, 323–331.CrossRefGoogle Scholar
van der Burg, E. (1988). Nonlinear canonical correlation and some related techniques, Leiden: DSWO Press.Google Scholar
Vinod, H.D. (1976). Canonical ridge and econometrics of joint production. Journal of Econometrics, 4, 47–166.CrossRefGoogle Scholar