Hostname: page-component-745bb68f8f-5r2nc Total loading time: 0 Render date: 2025-01-08T13:59:26.009Z Has data issue: false hasContentIssue false
  • English
  • Français

Heuristic Implementation of Dynamic Programming for Matrix Permutation Problems in Combinatorial Data Analysis

Published online by Cambridge University Press:  01 January 2025

Michael J. Brusco ,
Hans-Friedrich Köhn  and
Stephanie Stahl
Michael J. Brusco*
Affiliation:
Florida State University, Tallahassee, FL
Hans-Friedrich Köhn
Affiliation:
University of Missouri-Columbia, Columbia, MO
Stephanie Stahl
Affiliation:
Tallahassee, FL
*
Requests for reprints should be sent to Michael J. Brusco, Department of Marketing, College of Business, Florida State University, Tallahassee, FL 32306-1110, USA. E-mail: mbrusco@cob.fsu.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Dynamic programming methods for matrix permutation problems in combinatorial data analysis can produce globally-optimal solutions for matrices up to size 30×30, but are computationally infeasible for larger matrices because of enormous computer memory requirements. Branch-and-bound methods also guarantee globally-optimal solutions, but computation time considerations generally limit their applicability to matrix sizes no greater than 35×35. Accordingly, a variety of heuristic methods have been proposed for larger matrices, including iterative quadratic assignment, tabu search, simulated annealing, and variable neighborhood search. Although these heuristics can produce exceptional results, they are prone to converge to local optima where the permutation is difficult to dislodge via traditional neighborhood moves (e.g., pairwise interchanges, object-block relocations, object-block reversals, etc.). We show that a heuristic implementation of dynamic programming yields an efficient procedure for escaping local optima. Specifically, we propose applying dynamic programming to reasonably-sized subsequences of consecutive objects in the locally-optimal permutation, identified by simulated annealing, to further improve the value of the objective function. Experimental results are provided for three classic matrix permutation problems in the combinatorial data analysis literature: (a) maximizing a dominance index for an asymmetric proximity matrix; (b) least-squares unidimensional scaling of a symmetric dissimilarity matrix; and (c) approximating an anti-Robinson structure for a symmetric dissimilarity matrix.

Keywords

Combinatorial data analysismatrix permutationdynamic programmingheuristics
Type
Theory and Methods
Information
Psychometrika , Volume 73 , Issue 3 , September 2008 , pp. 503 - 522
Copyright
Copyright © 2007 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We are extremely grateful to the Associate Editor and two anonymous reviewers for helpful suggestions and corrections.

References

Aarts, E., & Korst, J. (1989). Simulated annealing and Boltzmann machines: A stochastic approach to combinatorial optimization and neural computing, New York: Wiley.Google Scholar
Baker, F.B., & Hubert, L.J. (1977). Applications of combinatorial programming to data analysis: Seriation using asymmetric proximity measures. British Journal of Mathematical and Statistical Psychology, 30, 154–164.CrossRefGoogle Scholar
Bellman, R. (1962). Dynamic programming treatment of the traveling salesman problem. Journal of the Association for Computing Machinery, 9, 61–63.CrossRefGoogle Scholar
Blin, J.M., & Whinston, A.B. (1974). A note on majority rule under transitivity constraints. Management Science, 20, 1439–1440.CrossRefGoogle Scholar
Bowman, V.J., & Colantoni, C.S. (1973). Majority rule under transitivity constraints. Management Science, 19, 1029–1041.CrossRefGoogle Scholar
Brusco, M.J. (2001). Seriation of asymmetric proximity matrices using integer linear programming. British Journal of Mathematical and Statistical Psychology, 54, 367–375.CrossRefGoogle ScholarPubMed
Brusco, M.J. (2001). A simulated annealing heuristic for unidimensional and multidimensional (city-block) scaling of symmetric proximity matrices. Journal of Classification, 18, 3–33.CrossRefGoogle Scholar
Brusco, M.J. (2002). Identifying a reordering of the rows and columns of multiple proximity matrices using multiobjective programming. Journal of Mathematical Psychology, 46, 731–745.CrossRefGoogle Scholar
Brusco, M.J. (2006). On the performance of simulated annealing for large-scale L 2 unidimensional scaling. Journal of Classification, 23, 255–268.CrossRefGoogle Scholar
Brusco, M.J., & Stahl, S. (2000). Using quadratic assignment methods to generate initial permutations for least-squares unidimensional scaling of symmetric proximity matrices. Journal of Classification, 17, 197–223.CrossRefGoogle Scholar
Brusco, M.J., & Stahl, S. (2001). An interactive approach to multiobjective combinatorial data analysis. Psychometrika, 66, 5–24.CrossRefGoogle Scholar
Brusco, M.J., & Stahl, S. (2005). Optimal least-squares unidimensional scaling: Improved branch-and-bound procedures and comparison to dynamic programming. Psychometrika, 70, 253–270.CrossRefGoogle Scholar
Brusco, M.J., & Stahl, S. (2005). Branch-and-bound applications in combinatorial data analysis, New York: Springer.Google Scholar
Brusco, M.J., & Stahl, S. (2005). Bicriterion seriation methods for skew-symmetric matrices. British Journal of Mathematical and Statistical Psychology, 58, 333–343.CrossRefGoogle ScholarPubMed
Brusco, M.J., & Steinley, D. (in press). A comparison of heuristic procedures for minimum within-cluster sums of squares partitioning. Psychometrika.Google Scholar
Chenery, H.R., & Watanabe, T. (1958). International comparisons of the structure of production. Econometrica, 26, 487–521.CrossRefGoogle Scholar
DeCani, J.S. (1969). Maximum likelihood paired comparison ranking by linear programming. Biometrika, 56, 537–545.CrossRefGoogle Scholar
DeCani, J.S. (1972). A branch and bound algorithm for maximum likelihood paired comparison ranking by linear programming. Biometrika, 59, 131–135.CrossRefGoogle Scholar
Defays, D. (1978). A short note on a method of seriation. British Journal of Mathematical and Statistical Psychology, 31, 49–53.CrossRefGoogle Scholar
de Leeuw, J., & Heiser, W.J. (1977). Convergence of correction-matrix algorithms for multidimensional scaling. In Lingoes, J.C. (Eds.), Geometric representations of relational data: Readings in multidimensional scaling (pp. 735–752). Ann Arbor: Mathesis Press.Google Scholar
De Soete, G., Hubert, L., & Arabie, P. (1988). The comparative performance of simulated annealing on two problems of combinatorial data analysis. In Diday, E. (Eds.), Data analysis and informatics (pp. 489–496). Amsterdam: North-Holland.Google Scholar
Flueck, J.A., & Korsh, J.F. (1974). A branch search algorithm for maximum likelihood paired comparison ranking. Biometrika, 61, 621–626.CrossRefGoogle Scholar
Fukui, Y. (1986). A more powerful method for triangularizing input-output matrices and the similarity of production structures. Econometrica, 54, 1425–1433.CrossRefGoogle Scholar
Garcia, C.G., Pérez-Brito, D., Campos, V., & Marti, R. (2006). Variable neighborhood search for the linear ordering problem. Computers & Operations Research, 33, 3549–3565.CrossRefGoogle Scholar
Garey, M.R., & Johnson, D.S. (1979). Computers and intractability: A guide to the theory of NP-completeness, San Francisco: Freeman.Google Scholar
Glover, F., & Laguna, M. (1993). Tabu search. In Reeves, C. (Eds.), Modern heuristic techniques for combinatorial problems (pp. 70–141). Oxford: Blackwell.Google Scholar
Goldberg, D.E. (1989). Genetic algorithms in search, optimization, and machine learning, New York: Addison-Wesley.Google Scholar
Groenen, P.J.F. (1993). The majorization approach to multidimensional scaling: Some problems and extensions, Leiden: DSWO Press.Google Scholar
Groenen, P.J.F., & Heiser, W.J. (1996). The tunneling method for global optimization in multidimensional scaling. Psychometrika, 61, 529–550.CrossRefGoogle Scholar
Groenen, P.J.F., Heiser, W.J., & Meulman, J.J. (1999). Global optimization in least-squares multidimensional scaling by distance smoothing. Journal of Classification, 16, 225–254.CrossRefGoogle Scholar
Grötschel, M., Jünger, M., & Reinelt, G. (1984). A cutting plane algorithm for the linear ordering problem. Operations Research, 32, 1195–1220.CrossRefGoogle Scholar
Hansen, P., & Mladenoviĉ, N. (2003). Variable neighborhood search. In Glover, F.W., & Kochenberger, G.A. (Eds.), Handbook of metaheuristics (pp. 145–184). Norwell: Kluwer Academic.CrossRefGoogle Scholar
Held, M., & Karp, R.M. (1962). A dynamic programming approach to sequencing problems. Journal of the Society for Industrial and Applied Mathematics, 10, 196–210.CrossRefGoogle Scholar
Holman, E. (1979). Monotonic models for asymmetric proximities. Journal of Mathematical Psychology, 20, 1–15.CrossRefGoogle Scholar
Howe, E.C. (1991). A more powerful method for triangularizing input-output matrices: A comment. Econometrica, 59, 521–523.CrossRefGoogle Scholar
Hubert, L. (1976). Seriation using asymmetric proximity measures. British Journal of Mathematical and Statistical Psychology, 29, 32–52.CrossRefGoogle Scholar
Hubert, L., & Arabie, P. (1986). Unidimensional scaling and combinatorial optimization. In de Leeuw, J., Heiser, W., Meulman, J., & Critchley, F. (Eds.), Multidimensional data analysis (pp. 181–196). Leiden: DSWO Press.Google Scholar
Hubert, L., & Arabie, P. (1994). The analysis of proximity matrices through sums of matrices having (anti-)Robinson forms. British Journal of Mathematical and Statistical Psychology, 47, 1–40.CrossRefGoogle Scholar
Hubert, L., & Arabie, P. (1995). Iterative projection strategies for the least-squares fitting of tree structures to proximity data. British Journal of Mathematical and Statistical Psychology, 48, 281–317.CrossRefGoogle Scholar
Hubert, L.J., & Golledge, R.G. (1981). Matrix reorganization and dynamic programming: Applications to paired comparisons and unidimensional seriation. Psychometrika, 46, 429–441.CrossRefGoogle Scholar
Hubert, L., & Schultz, J. (1976). Quadratic assignment as a general data analysis strategy. British Journal of Mathematical and Statistical Psychology, 29, 190–241.CrossRefGoogle Scholar
Hubert, L., Arabie, P., & Meulman, J. (1997). Linear and circular unidimensional scaling for symmetric proximity matrices. British Journal of Mathematical and Statistical Psychology, 50, 253–284.CrossRefGoogle Scholar
Hubert, L., Arabie, P., & Meulman, J. (1998). Graph-theoretic representations for proximity matrices through strongly anti-Robinson or circular strongly anti-Robinson matrices. Psychometrika, 63, 341–358.CrossRefGoogle Scholar
Hubert, L., Arabie, P., & Meulman, J. (1998). The representation of symmetric proximity data: Dimensions and classifications. The Computer Journal, 41, 566–577.CrossRefGoogle Scholar
Hubert, L., Arabie, P., & Meulman, J. (2001). Combinatorial data analysis: Optimization by dynamic programming, Philadelphia: SIAM.CrossRefGoogle Scholar
Hubert, L.J., Arabie, P., & Meulman, J.J. (2002). Linear unidimensional scaling in the L2-Norm: Basic optimization methods using MATLAB. Journal of Classification, 19, 303–328.CrossRefGoogle Scholar
Hubert, L., Arabie, P., & Meulman, J. (2006). The structural representation of proximity matrices with MATLAB, Philadelphia: SIAM.CrossRefGoogle Scholar
Laguna, M., Marti, R., & Campos, V. (1999). Intensification and diversification with elite tabu search solutions for the linear ordering problem. Computers & Operations Research, 26, 1217–1230.CrossRefGoogle Scholar
Lawler, E.L. (1964). A comment on minimum feedback arc sets. IEEE Transactions on Circuit Theory, 11, 296–297.CrossRefGoogle Scholar
Murillo, A., Vera, J.F., & Heiser, W.J. (2005). A permutation-translation simulated annealing algorithm for L 1 and L 2 unidimensional scaling. Journal of Classification, 22, 119–138.CrossRefGoogle Scholar
Phillips, J.P.N. (1967). A procedure for determining Slater’s i and all nearest adjoining orders. British Journal of Mathematical and Statistical Psychology, 20, 217–225.CrossRefGoogle Scholar
Phillips, J.P.N. (1969). A further procedure for determining Slater’s i and all nearest adjoining orders. British Journal of Mathematical and Statistical Psychology, 22, 97–101.CrossRefGoogle Scholar
Pliner, V. (1996). Metric unidimensional scaling and global optimization. Journal of Classification, 13, 3–18.CrossRefGoogle Scholar
Ranyard, R.H. (1976). An algorithm for maximum likelihood ranking and Slater’s i from paired comparisons. British Journal of Mathematical and Statistical Psychology, 29, 242–248.CrossRefGoogle Scholar
Reinelt, G. (1997). LOLIB. http://www.iwr.uni-heidelberg.de/groups/comopt/software/LOLIB.Google Scholar
Robinson, W.S. (1951). A method for chronologically ordering archaeological deposits. American Antiquity, 16, 293–301.CrossRefGoogle Scholar
Rothkopf, E.Z. (1957). A measure of stimulus similarity and errors in some paired-associate learning tasks. Journal of Experimental Psychology, 53, 94–101.CrossRefGoogle ScholarPubMed
Schiavinotto, T., & Stützle, T. (2004). The linear ordering problem: Instances, search space analysis and algorithms. Journal of Mathematical Modelling and Algorithms, 3, 367–402.CrossRefGoogle Scholar
Shepard, R.N., Kilpatrick, D.W., & Cunningham, J.P. (1975). The internal representation of numbers. Cognitive Psychology, 7, 82–138.CrossRefGoogle Scholar
Slater, P. (1961). Inconsistencies in a schedule of paired comparisons. Biometrika, 48, 303–312.CrossRefGoogle Scholar
Szczotka, F. (1972). On a method of ordering and clustering of objects. Zastosowania Mathematyki, 13, 23–33.Google Scholar
van Os, B.J. (2000). Dynamic programming for partitioning in multivariate data analysis, Leiden: Leiden University Press.Google Scholar
van Os, B.J., & Meulman, J.J. (2004). Improving dynamic programming strategies for partitioning. Journal of Classification, 21, 207–230.CrossRefGoogle Scholar