Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-08T16:17:27.504Z Has data issue: false hasContentIssue false
  • English
  • Français

Constrained Candecomp/Parafac via the Lasso

Published online by Cambridge University Press:  01 January 2025

Paolo Giordani  and
Roberto Rocci
Paolo Giordani*
Affiliation:
Dipartimento di Scienze Statistiche, Sapienza Università di Roma
Roberto Rocci
Affiliation:
Dipartimento di Economia e Finanza, Università “Tor Vergata”
*
Requests for reprints should be sent to Paolo Giordani, Dipartimento di Scienze Statistiche, Sapienza Università di Roma, P.le A. Moro, 5, 00185 Rome, Italy. E-mail: paolo.giordani@uniroma1.it
Get access Rights & Permissions [Opens in a new window]

Abstract

The Candecomp/Parafac (CP) model is a well-known tool for summarizing a three-way array by extracting a limited number of components. Unfortunately, in some cases, the model suffers from the so-called degeneracy, that is a solution with diverging and uninterpretable components. To avoid degeneracy, orthogonality constraints are usually applied to one of the component matrices. This solves the problem only from a technical point of view because the existence of orthogonal components underlying the data is not guaranteed. For this purpose, we consider some variants of the CP model where the orthogonality constraints are relaxed either by constraining only a pair, or a subset, of components or by stimulating the CP solution to be possibly orthogonal. We theoretically clarify that only the latter approach, based on the least absolute shrinkage and selection operator and named the CP-Lasso, is helpful in solving the degeneracy problem. The results of the application of CP-Lasso on simulated and real life data show its effectiveness.

Keywords

Candecomp/ParafacdegeneracyLassoorthogonality constraints
Type
Original Paper
Information
Psychometrika , Volume 78 , Issue 4 , October 2013 , pp. 669 - 684
Copyright
Copyright © 2013 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bini, D. (1980). Border rank of a p×q×2 tensor and the optimal approximation of a pair of bilinear forms. In de Bakker, J.W., & van Leeuwen, J. (Eds.), Automata, languages and programming (pp. 98–108). New York: Springer.CrossRefGoogle Scholar
Carroll, J.D., & Chang, J.J. (1970). Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart–Young decomposition. Psychometrika, 35, 283–319.CrossRefGoogle Scholar
Cliff, N. (1966). Orthogonal rotation to congruence. Psychometrika, 31, 33–42.CrossRefGoogle Scholar
De Silva, V., & Lim, L.-H. (2008). Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM Journal on Matrix Analysis and Applications, 30, 1084–1127.CrossRefGoogle Scholar
Gill, P.E., Murray, W., & Wright, M.H. (1981). Practical optimization. London: Academic Press.Google Scholar
Guggenheimer, H.W., Edelman, A.S., & Johnson, C.R. (1995). A simple estimate of the condition number of a linear system. The College Mathematics Journal, 26, 2–5.CrossRefGoogle Scholar
Harshman, R.A. (1970). Foundations of the Parafac procedure: models and conditions for an “explanatory” multimodal factor analysis (UCLA Working Papers in Phonetics, 16, pp. 1–84).Google Scholar
Harshman, R.A., & De Sarbo, W.S. (1984). An application of PARAFAC to a small sample problem, demonstrating preprocessing, orthogonality constraints, and split-half diagnostic techniques. In Law, H.G., Snyder, C.W. Jr., Hattie, J.A., & McDonald, R.P. (Eds.), Research methods for multimode data analysis (pp. 602–642). New York: Praeger.Google Scholar
Harshman, R.A., & Lundy, M.E. (1984). Data preprocessing and the extended PARAFAC model. In Law, H.G., Snyder, C.W. Jr., Hattie, J.A., & McDonald, R.P. (Eds.), Research methods for multimode data analysis (pp. 216–284). New York: Praeger.Google Scholar
Jiang, T., & Sidiropoulos, N.D. (2004). Kruskal’s permutation lemma and the identification of Candecomp/Parafac and bilinear models with constant modulus constraints. IEEE Transactions on Signal Processing, 52, 2625–2636.CrossRefGoogle Scholar
Krijnen, W.P. (2006). Convergence of the sequence of parameters generated by alternating least squares algorithms. Computational Statistics & Data Analysis, 51, 481–489.CrossRefGoogle Scholar
Krijnen, W.P., Dijkstra, T.K., & Stegeman, A. (2008). On the non-existence of optimal solutions and the occurrence of “degeneracy” in the Candecomp/Parafac model. Psychometrika, 73, 431–439.CrossRefGoogle ScholarPubMed
Kruskal, J.B. (1977). Three-way arrays: rank and uniqueness of trilinear decompositions, with applications to arithmetic complexity and statistics. Linear Algebra and Its Applications, 18, 95–138.CrossRefGoogle Scholar
Kruskal, J.B., Harshman, R.A., & Lundy, M.E. (1989). How 3-MFA data can cause degenerate PARAFAC solutions, among other relationships. In Coppi, R., & Bolasco, S. (Eds.), Multiway data analysis (pp. 115–122). Amsterdam: Elsevier.Google Scholar
Lawson, C.L., & Hanson, R.J. (1995). Solving least squares problems. Philadelphia: SIAM.CrossRefGoogle Scholar
Louwerse, D.J., Smilde, A.K., & Kiers, H.A.L. (1999). Cross-validation of multiway component models. Journal of Chemometrics, 13, 491–510.3.0.CO;2-2>CrossRefGoogle Scholar
Lundy, M.E., Harshman, R.A., & Kruskal, J.B. (1989). A two stage procedure incorporating good features of both trilinear and quadrilinear models. In Coppi, R., & Bolasco, S. (Eds.), Multiway data analysis (pp. 123–130). Amsterdam: Elsevier.Google Scholar
Merikoski, J.K., Urpala, U., Virtanen, A., Tam, T.-Y., & Uhlig, F. (1997). A best upper bound for the 2-norm condition number of a matrix. Linear Algebra and Its Applications, 254, 355–365.CrossRefGoogle Scholar
Mitchell, B.C., & Burdick, D.S. (1994). Slowly converging Parafac sequences: swamps and two-factor degeneracies. Journal of Chemometrics, 8, 155–168.CrossRefGoogle Scholar
Ortega, J.M., & Rheinboldt, W.C. (1970). Iterative solution of nonlinear equations in several variables. San Diego: Academic Press.Google Scholar
Paatero, P. (2000). Construction and analysis of degenerate Parafac models. Journal of Chemometrics, 14, 285–299.3.0.CO;2-1>CrossRefGoogle Scholar
Rayens, W.S., & Mitchell, B.C. (1997). Two-factor degeneracies and a stabilization of Parafac. Chemometrics and Intelligent Laboratory Systems, 38, 173–181.CrossRefGoogle Scholar
Rocci, R., & Giordani, P. (2010). A weak degeneracy revealing decomposition for the CANDECOMP/PARAFAC model. Journal of Chemometrics, 24, 57–66.CrossRefGoogle Scholar
Stegeman, A. (2006). Degeneracy in Candecomp/Parafac explained for p×p×2 arrays of rank p+1 or higher. Psychometrika, 71, 483–501.CrossRefGoogle Scholar
Stegeman, A. (2007). Degeneracy in Candecomp/Parafac and Indscal explained for several three-sliced arrays with a two-valued typical rank. Psychometrika, 72, 601–619.CrossRefGoogle ScholarPubMed
Stegeman, A. (2008). Low-rank approximation of generic p×q×2 arrays and diverging components in the Candecomp/Parafac model. SIAM Journal on Matrix Analysis and Applications, 30, 988–1007.CrossRefGoogle Scholar
Stegeman, A. (2009). On uniqueness conditions for Candecomp/Parafac and Indscal with full column rank in one mode. Linear Algebra and Its Applications, 431, 211–227.CrossRefGoogle Scholar
Stegeman, A. (2009). Using the simultaneous generalized Schur decomposition as a Candecomp/Parafac algorithm for ill-conditioned data. Journal of Chemometrics, 23, 385–392.CrossRefGoogle Scholar
Stegeman, A., Ten Berge, J.M.F., & De Lathauwer, L. (2006). Sufficient conditions for uniqueness in Candecomp/Parafac and Indscal with random component matrices. Psychometrika, 71, 219–229.CrossRefGoogle ScholarPubMed
Stegeman, A. (2012). Candecomp/Parafac: from diverging components to a decomposition in block terms. SIAM Journal on Matrix Analysis and Applications, 30, 1614–1638.CrossRefGoogle Scholar
Stegeman, A., & De Lathauwer, L. (2009). A method to avoid diverging components in the Candecomp/Parafac model for generic I×J×2 arrays. SIAM Journal on Matrix Analysis and Applications, 30, 1614–1638.CrossRefGoogle Scholar
Styan, G.P.H. (1973). Hadamard products and multivariate statistical analysis. Linear Algebra and Its Applications, 6, 217–240.CrossRefGoogle Scholar
ten Berge, J.M.F., Kiers, H.A.L., & De Leeuw, J. (1988). Explicit Candecomp/Parafac solutions for a contrived 2×2×2 array of rank three. Psychometrika, 53, 579–584.CrossRefGoogle Scholar
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B, 58, 267–288.CrossRefGoogle Scholar
Tomasi, G., & Bro, G. (2006). A comparison of algorithms for fitting the PARAFAC model. Computational Statistics & Data Analysis, 50, 1700–1734.CrossRefGoogle Scholar
Tucker, L.R. (1951). A method for synthesis of factor analysis studies (Personnel Research Section Report No. 984). Department of the Army, Washington.Google Scholar