Hostname: page-component-5f745c7db-sbzbt Total loading time: 0 Render date: 2025-01-06T06:43:09.980Z Has data issue: true hasContentIssue false
  • English
  • Français

A Constrained Linear Estimator for Multiple Regression

Published online by Cambridge University Press:  01 January 2025

Clintin P. Davis-Stober ,
Jason Dana  and
David V. Budescu
Clintin P. Davis-Stober*
Affiliation:
University of Missouri
Jason Dana
Affiliation:
University of Pennsylvania
David V. Budescu
Affiliation:
Fordham University
*
Requests for reprints should be sent to Clintin P. Davis-Stober, University of Missouri at Columbia, 210 McAlester Hall., Columbia, MO, 65211, USA. E-mail: stoberc@missouri.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

“Improper linear models” (see Dawes, Am. Psychol. 34:571–582, 1979), such as equal weighting, have garnered interest as alternatives to standard regression models. We analyze the general circumstances under which these models perform well by recasting a class of “improper” linear models as “proper” statistical models with a single predictor. We derive the upper bound on the mean squared error of this estimator and demonstrate that it has less variance than ordinary least squares estimates. We examine common choices of the weighting vector used in the literature, e.g., single variable heuristics and equal weighting, and illustrate their performance in various test cases.

Keywords

least squaresinconsistent estimatorsimproper linear modelsunit weightsequal weightstake the bestconstrained models
Type
Original Paper
Information
Psychometrika , Volume 75 , Issue 3 , September 2010 , pp. 521 - 541
Copyright
Copyright © 2010 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Much of this work was made possible by a predoctoral trainee fellowship awarded to the first author from the National Institutes of Mental Health under Training Grant Award Nr. PHS 2 T32 MH014257 entitled “Quantitative Methods for Behavioral Research” (to M. Regenwetter, PI). The first author was also supported by a Dissertation Completion Fellowship awarded by the University of Illinois. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Institutes of Mental Health or the University of Illinois. We would like to thank Lawrence Hubert and Konstantinos Katsikopoulos for comments on an earlier draft. We would also like to thank the editor, action editor, and three anonymous referees for their comments and suggestions.

References

Azen, R., Budescu, D.V. (2009). Applications of multiple regression in psychological research. In Millsap, R., Olivares, A.M. (Eds.), Handbook of quantitative methods in psychology (pp. 285–310). Thousand Oaks: Sage.CrossRefGoogle Scholar
Ben-Haim, Z., & Eldar, Y.C. (2005). Minimax estimators dominating the least-squares estimator. In Acoustics, speech, and signal processing, 2005. Proceedings (ICASSP ’05). IEEE international conference on (Vol. 4, pp. 53–56).CrossRefGoogle Scholar
Camerer, C.F. (1981). General conditions for the success of bootstrapping models. Organizational Behavior and Human Performance, 27, 411–422.CrossRefGoogle Scholar
Dana, J. (2008). What makes improper linear models tick?. In Krueger, J. (Eds.), Rationality and social responsibility: essays in honor of Robyn Mason Dawes, Mahwah: Lawrence Erlbaum Associates.Google Scholar
Dana, J., Dawes, R.M. (2004). The superiority of simple alternatives to regression for social science predictions. Journal of Educational and Behavioral Statistics, 3, 317–331.CrossRefGoogle Scholar
Dawes, R.M. (1979). The robust beauty of improper linear models. American Psychologist, 34, 571–582.CrossRefGoogle Scholar
Dawes, R.M., Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.CrossRefGoogle Scholar
Dorans, N., Drasgow, F. (1978). Alternative weighting schemes for linear prediction. Organizational Behavior and Human Performance, 21, 316–345.CrossRefGoogle Scholar
Draper, N.R., van Nostrand, R.C. (1979). Ridge regression and James–Stein estimation: Review and comments. Technometrics, 21, 451–466.CrossRefGoogle Scholar
Einhorn, H.J., Hogarth, R.M. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13, 171–192.CrossRefGoogle Scholar
Eldar, Y.C., Ben-Tal, A., Nemirovski, A. (2005). Robust mean-squared error estimation in the presence of model uncertainties. IEEE Transactions on Signal Processing, 53, 168–181.CrossRefGoogle Scholar
Gigerenzer, G., Goldstein, D.G. (1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103, 650–669.CrossRefGoogle ScholarPubMed
Gigerenzer, G., Todd, P.M.the ABC Research Group (1999). Simple heuristics that make us smart, New York: Oxford University Press.Google Scholar
Goldstein, D.G., Gigerenzer, G. (2009). Fast and frugal forecasting. International Journal of Forecasting, 25, 760–772.CrossRefGoogle Scholar
Guttman, L. (1984). A corollary of least-squares: For every unbiased estimator there is generally a better biased estimator. Unpublished manuscript.Google Scholar
Hocking, R.R. (1983). Developments in linear regression methodology: 1959–1982. Technometrics, 25, 219–230.Google Scholar
Hoerl, A.E., Kennard, R.W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55–67.CrossRefGoogle Scholar
James, W., Stein, C. (1961). Estimation with quadratic loss. Proceedings of the 4th Berkeley symposium on mathematical statistics and probability (pp. 361–379). Berkeley: Univ. California Press.Google Scholar
Kendall, M.G. (1957). A course in multivariate analysis, London: Griffin.Google Scholar
Keren, G., Newman, J.R. (1978). Additional considerations with regard to multiple regression and equal weighting. Organizational Behavior and Human Performance, 22, 143–164.CrossRefGoogle Scholar
Marquardt, D.W. (1970). Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation. Technometrics, 12, 591–612.CrossRefGoogle Scholar
Pruzek, R.M., Frederick, B.C. (1978). Weighting predictors in linear models: Alternatives to least squares and limitations of equal weights. Psychological Bulletin, 85, 254–266.CrossRefGoogle Scholar
Simon, H.A. (1956). Rational choice and the structure of the environment. Psychological Review, 63, 129–138.CrossRefGoogle ScholarPubMed
Simon, H.A. (1982). Models of bounded rationality, Cambridge: MIT Press.Google Scholar
Wainer, H. (1976). Estimating coefficients in linear models: It don’t make no nevermind. Psychological Bulletin, 83, 213–217.CrossRefGoogle Scholar
Waller, N.G. (2008). Fungible weights in multiple regression. Psychometrika, 73, 691–703.CrossRefGoogle Scholar
Wang, M.W., Stanley, J.C. (1970). Differential weighting: A review of methods and empirical studies. Review of Educational Research, 40, 663–705.CrossRefGoogle Scholar
Wilks, S.S. (1938). Weighting systems for linear functions of correlated variables when there is no dependent variable. Psychometrika, 3, 23–40.CrossRefGoogle Scholar
Vinod, H.D. (1978). A survey of ridge regression and related techniques for improvements over ordinary least squares. The Review of Economics and Statistics, 60, 121–131.CrossRefGoogle Scholar