Hostname: page-component-6bf8c574d5-r4mrb Total loading time: 0 Render date: 2025-03-07T03:19:39.102Z Has data issue: false hasContentIssue false
  • English
  • Français

A Geometric Analysis of When Fixed Weighting Schemes Will Outperform Ordinary Least Squares

Published online by Cambridge University Press:  01 January 2025

Clintin P. Davis-Stober
Clintin P. Davis-Stober*
Affiliation:
University of Missouri
*
Requests for reprints should be sent to Clintin P. Davis-Stober, Department of Psychological Sciences, University of Missouri, 219 McAlester Hall, Columbia, MO, USA. E-mail: stoberc@missouri.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Many researchers have demonstrated that fixed, exogenously chosen weights can be useful alternatives to Ordinary Least Squares (OLS) estimation within the linear model (e.g., Dawes, Am. Psychol. 34:571–582, 1979; Einhorn & Hogarth, Org. Behav. Human Perform. 13:171–192, 1975; Wainer, Psychol. Bull. 83:213-217, 1976). Generalizing the approach of Davis-Stober, Dana, and Budescu (Psychometrika 75:521–541, 2010b), I present an analytic method to determine when a choice of fixed weights will incur less mean squared error than OLS as a function of sample size, error variance, and model predictability. Geometrically, I solve for the region of population β that favors a choice of fixed weights over OLS. I derive closed-form upper and lower bounds on the volume of this region, giving tight bounds on the proportion of population β favoring a choice of fixed weights. I illustrate this methodology with several examples and provide a MATLAB© (The MathWorks, Matlab software, version 2009b, 2010) programming implementation of the major results.

Keywords

improper modelshyper-cylindersmean squared errorequal weightsalternative weightsfixed weights
Type
Original Paper
Information
Psychometrika , Volume 76 , Issue 4 , October 2011 , pp. 650 - 669
Copyright
Copyright © 2011 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Azen, R., Budescu, D.V. (2009). Applications of multiple regression in psychological research. In Millsap, R., Olivares, A.M. (Eds.), Handbook of quantitative methods in psychology (pp. 285–310). Thousand Oaks: Sage.CrossRefGoogle Scholar
Barron, F.H., Barrett, B.E. (1996). Decision quality using ranked attribute weights. Management Science, 42, 1515–1523.CrossRefGoogle Scholar
Baucells, M., Carrasco, J.A., Hogarth, R.M. (2008). Cumulative dominance and heuristic performance in binary multiattribute choice. Operations Research, 56, 1289–1304.CrossRefGoogle Scholar
Ben-Haim, Z., Eldar, Y.C. (2005). Minimax estimators dominating the least-squares estimator. ICASSP’05: Proceedings of the 30th IEEE international conference on acoustics, speech, and signal processing (pp. 53–56). Washington: IEEE Computer Society.Google Scholar
Bickel, P.J., Doksum, K.A. (2001). Mathematical statistics: basic ideas and selected topics, (2nd ed.). Upper Saddle River: Prentice Hall.Google Scholar
Bobko, P., Roth, P.L., Buster, M.A. (2007). The usefulness of unit weights in creating composite scores—a literature review, application to content validity, and meta-analysis. Organizational Research Methods, 10, 689–709.CrossRefGoogle Scholar
Boothby, W.M. (2003). An introduction to differentiable manifolds and Riemannian geometry, San Diego: Academic Press.Google Scholar
Borda, J.C. (1781). Mémoire sur les élections au scrutin, Paris: Histoire de l’Académie Royale des Sciences.Google Scholar
Czerlinski, J., Gigerenzer, G., Goldstein, D.G. (1999). How good are simple heuristics. In Gigerenzer, G., Todd, P.M.the ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 97–118). New York: Oxford University Press.Google Scholar
Dana, J. (2008). What makes improper linear models tick. In Krueger, J. (Eds.), Rationality and social responsibility: essays in honor of Robyn Mason Dawes, Mahwah: Lawrence Erlbaum Associates.Google Scholar
Dana, J., Dawes, R.M. (2004). The superiority of simple alternatives to regression for social science predictions. Journal of Educational and Behavioral Statistics, 3, 317–331.CrossRefGoogle Scholar
Davis-Stober, C.P., Dana, J., Budescu, D.V. (2010). Why recognition is rational: Optimality results on single-variable decision rules. Judgment and Decision Making, 5, 216–229.CrossRefGoogle Scholar
Davis-Stober, C.P., Dana, J., Budescu, D.V. (2010). A constrained linear estimator for multiple regression. Psychometrika, 75, 521–541.CrossRefGoogle Scholar
Dawes, R.M. (1971). A case study of graduate admissions: Applications of three principles of human decision making. American Psychologist, 26, 180–188.CrossRefGoogle Scholar
Dawes, R.M. (1979). The robust beauty of improper linear models. American Psychologist, 34, 571–582.CrossRefGoogle Scholar
Dawes, R.M., Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.CrossRefGoogle Scholar
Dorans, N., Drasgow, F. (1978). Alternative weighting schemes for linear prediction. Organizational Behavior and Human Performance, 21, 316–345.CrossRefGoogle Scholar
Einhorn, H.J., Hogarth, R.M. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13, 171–192.CrossRefGoogle Scholar
Eldar, Y.C., Ben-Tal, A., Nemirovski, A. (2005). Robust mean-squared error estimation in the presence of model uncertainties. IEEE Transactions on Signal Processing, 53, 168–181.CrossRefGoogle Scholar
Fasolo, B., McClelland, G.H., Todd, P.M. (2007). Escaping the tyranny of choice: When fewer attributes make choice easier. Marketing Theory, 7, 13–26.CrossRefGoogle Scholar
Gigerenzer, G., Goldstein, D.G. (1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103, 650–669.CrossRefGoogle ScholarPubMed
Gigerenzer, G., Todd, P.M.the ABC Research Group (1999). Simple heuristics that make us smart, New York: Oxford University Press.Google Scholar
Goldstein, D.G., Gigerenzer, G. (2002). Models of ecological rationality: The recognition heuristic. Psychological Review, 109, 75–90.CrossRefGoogle ScholarPubMed
Goldstein, D.G., Gigerenzer, G. (2009). Fast and frugal forecasting. International Journal of Forecasting, 25, 760–772.CrossRefGoogle Scholar
Guttman, L. (1984). A corollary of least-squares: For every unbiased estimator there is generally a better biased estimator. Unpublished manuscript.Google Scholar
Hoerl, A.E., Kennard, R.W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55–67.CrossRefGoogle Scholar
Hogarth, R.M., Karelaia, N. (2005). Ignoring information in binary choice with continuous variables: When is less “more”. Journal of Mathematical Psychology, 49, 115–124.CrossRefGoogle Scholar
Hogarth, R.M., Karelaia, N. (2005). Simple models for multiattribute choice with many alternatives: When it does and does not pay to face tradeoffs with binary attributes?. Management Science, 51, 1860–1872.CrossRefGoogle Scholar
Hogarth, R.M., Karelaia, N. (2006). “Take-The-Best” and other simple strategies: Why and when they work “well” with binary cues. Theory and Decision, 61, 205–249.CrossRefGoogle Scholar
Hogarth, R.M., Karelaia, N. (2006). Regions of rationality: Maps for bounded agents. Decision Analysis, 3, 124–144.CrossRefGoogle Scholar
Hogarth, R.M., Karelaia, N. (2007). Heuristic and linear models of judgment: Matching rules and environments. Psychological Review, 114, 733–758.CrossRefGoogle ScholarPubMed
James, W., Stein, C. (1961). Estimation with quadratic loss. Proceedings of the 4th Berkeley symposium on mathematical statistics and probability (pp. 361–379). Berkeley: University of California Press.Google Scholar
Katsikopoulos, K.V., Martignon, L. (2006). Naïve heuristics for paired comparisons: Some results on their relative accuracy. Journal of Mathematical Psychology, 50, 488–494.CrossRefGoogle Scholar
Katsikopoulos, K.V., Schooler, L.J., Hertwig, R. (2010). The robust beauty of ordinary information. Psychological Review, 117, 1259–1266.CrossRefGoogle ScholarPubMed
Keren, G., Newman, J.R. (1978). Additional considerations with regard to multiple regression and equal weighting. Organizational Behavior and Human Performance, 22, 143–164.CrossRefGoogle Scholar
Koopman, R.F. (1988). On the sensitivity of a composite to its weights. Psychometrika, 53, 547–552.CrossRefGoogle Scholar
Lehmann, E.L., Casella, G. (1998). Theory of point estimation, (2nd ed.). New York: Springer.Google Scholar
Li, S. (2011). Concise formulas for the area and volume of a hyperspherical cap. Asian Journal of Mathematics and Statistics, 4, 66–70.CrossRefGoogle Scholar
Manning, H.P. (1914). Geometry of four dimensions, New York: The MacMillan.CrossRefGoogle Scholar
Martignon, L., Hoffrage, U. (1999). Why does one-reason decision making work? A case study in ecological rationality. In Gigerenzer, G., Todd, P.M.the ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 119–140). New York: Oxford University Press.Google Scholar
O’Neill, B. (1997). Elementary differential geometry, San Diego: Academic Press.Google Scholar
R Development Core Team (2010). R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. ISBN 3-900051-07-0. URL www.R-project.org.Google Scholar
Payne, J.W., Bettman, J.R., Johnson, E.J. (1993). The adaptive decision maker, New York: Cambridge University Press.CrossRefGoogle Scholar
Regenwetter, M., Grofman, B., Marley, A.A.J., Tsetlin, I.M. (2006). Behavioral social choice, New York: Cambridge University Press.Google Scholar
Schmidt, F.L. (1971). The relative efficiency of regression and simple unit predictor weights in applied differential psychology. Educational and Psychological Measurement, 31, 699–714.CrossRefGoogle Scholar
Stillwell, W.G., Seaver, D.A., Edwards, W. (1981). A comparison of weight approximation techniques in multiattribute utility decision making. Organizational Behavior and Human Performance, 28, 62–77.CrossRefGoogle Scholar
The Mathworks (2010). Matlab software, version 2009b. Natick: USA.Google Scholar
Wainer, H. (1976). Estimating coefficients in linear models: It don’t make no nevermind. Psychological Bulletin, 83, 213–217.CrossRefGoogle Scholar
Wainer, H. (1978). On the sensitivity of regression and regressors. Psychological Bulletin, 85, 267–273.CrossRefGoogle Scholar
Wainer, H., Thissen, D. (1976). Three steps toward robust regression. Psychometrika, 41, 9–34.CrossRefGoogle Scholar
Waller, N.G. (2008). Fungible weights in multiple regression. Psychometrika, 73, 691–703.CrossRefGoogle Scholar
Waller, N.G., Jones, J.A. (2009). Locating the extrema of fungible regression weights. Psychometrika, 74, 589–602.CrossRefGoogle Scholar
Waller, N.G., Jones, J.A. (2010). Correlation weights in multiple regression. Psychometrika, 75, 58–69.CrossRefGoogle Scholar
Waller, N.G., Jones, J.A. (2011). Investigating the performance of alternate regression weights by studying all possible criteria in regression models with a fixed set of predictors. Psychometrika, 76, 410–439.CrossRefGoogle Scholar
Wilks, S.S. (1938). Weighting systems for linear functions of correlated variables when there is no dependent variable. Psychometrika, 3, 23–40.CrossRefGoogle Scholar