Hostname: page-component-5f745c7db-j9pcf Total loading time: 0 Render date: 2025-01-06T06:57:22.680Z Has data issue: true hasContentIssue false
  • English
  • Français

Robust Nonorthogonal Analyses Revisited: An Update Based on Trimmed Means

Published online by Cambridge University Press:  01 January 2025

H. J. Keselman ,
Rhonda K. Kowalchuk  and
Lisa M. Lix
H. J. Keselman*
Affiliation:
University of Manitoba
Rhonda K. Kowalchuk
Affiliation:
University of Manitoba
Lisa M. Lix
Affiliation:
University of Manitoba
*
Requests for reprints should be sent to H. J. Keselman, Department of Psychology, University of Manitoba, Winnipeg, Manitoba R3T 2N2, CANADA.
Get access Rights & Permissions [Opens in a new window]

Abstract

Three approaches to the analysis of main and interaction effect hypotheses in nonorthogonal designs were compared in a 2 × 2 design for data that was neither normal in form nor equal in variance. The approaches involved either least squares or robust estimators of central tendency and variability and/or a test statistic that either pools or does not pool sources of variance. Specifically, we compared the ANOVA F test which used trimmed means and Winsorized variances, the Welch-James test with the usual least squares estimators for central tendency and variability and the Welch-James test using trimmed means and Winsorized variances. As hypothesized, we found that the latter approach provided excellent Type I error control, whereas the former two did not.

Keywords

nonorthogonal analysesrobust estimation testsType I errorstrimmed means
Type
Original Paper
Information
Psychometrika , Volume 63 , Issue 2 , June 1998 , pp. 145 - 163
Copyright
Copyright © 1998 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Financial support for this research was provided by grants to the first author from the National Sciences and Engineering Research Council of Canada (#OGP0015855) and the Social Sciences and Humanities Research Council (#410-95-0006). The authors would like to express their appreciation to the Associate Editor as well as the reviewers who provided valuable comments on an earlier version of this paper.

References

Alexander, R. A., & Govern, D. M. (1994). A new and simpler approximation for ANOVA under variance heterogeneity. Journal of Educational Statistics, 19, 91–101.CrossRefGoogle Scholar
Algina, J., Oshima, T. C., & Tang, K. L. (1991). Robustness of Yao's, James', and Johansen's tests under variance-covariance heteroscedasticity and nonnormality. Journal of Educational Statistics, 16, 125–140.Google Scholar
Behrens, W. V. (1929). Ein beitrag zur fehlerberechnung bei wenigen beobachtungen. Landwirtsch Jahrbucher, 68, 807–837 [An essay on calculating mistakes when having free observations]Google Scholar
Box, G. E. P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems, I. Effect of inequality of variance in the one-way classification. Annals of Mathematical Statistics, 25, 290–302.CrossRefGoogle Scholar
Bradley, J. V. (1978). Robustness?. British Journal of Mathematical and Statistical Psychology, 31, 144–152.CrossRefGoogle Scholar
Brown, M. B., & Forsythe, A. B. (1974). The small sample behavior of some statistics which test the equality of several means. Technometrics, 16, 129–132.CrossRefGoogle Scholar
Cressie, N. A. C., Whitford, H. J. (1986). How to use the two sample t-test. Biometrical Journal, 28, 131–148.CrossRefGoogle Scholar
D'Agostino, R. B., Belanger, A., D'Agostino, R. B. Jr. (1990). A suggestion for using powerful and informative tests of normality. The American Statistician, 44, 316–321.CrossRefGoogle Scholar
Fenstad, G. U. (1983). A comparison between U and V tests in the Behrens-Fisher problem. Biometrika, 70, 300–302.CrossRefGoogle Scholar
Fisher, R. A. (1935). The design of experiments, Edinburgh and London: Oliver & Boyd.Google Scholar
Gross, A. M. (1976). Confidence interval robustness with long-tailed symmetric distributions. Journal of the American Statistical Association, 71, 409–416.CrossRefGoogle Scholar
Hastings, N. A. J., Peacock, J. B. (1975). Statistical distributions: A handbook for students and practitioners, New York: Wiley.Google Scholar
Hoaglin, D. C. (1985). Summarizing shape numerically: The g- and h-distributions. In Hoaglin, D., Mosteller, F., Tukey, J. (Eds.), Exploring data tables, trends, and shapes (pp. 461–513). New York: Wiley.Google Scholar
James, G. S. (1951). The comparison of several groups of observations when the ratios of the population variances are unknown. Biometrika, 38, 324–329.CrossRefGoogle Scholar
James, G. S. (1954). Tests of linear hypotheses in univariate and multivariate analysis when the ratios of the population variances are unknown. Biometrika, 41, 19–43.Google Scholar
Johansen, S. (1980). The Welch-James approximation to the distribution of the residual sum of squares in a weighted linear regression. Biometrika, 67, 85–92.CrossRefGoogle Scholar
Keren, G. (1982). A balanced approach to unbalanced designs. In Keren, G. (Eds.), Statistical and methodological issues in psychology and social science research (pp. 155–186). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Keselman, H. J., Carriere, K. C., & Lix, L. M. (1993). Testing repeated measures hypotheses when covariance matrices are heterogeneous. Journal of Educational Statistics, 18, 305–319.CrossRefGoogle Scholar
Keselman, H. J., Carriere, K. C., & Lix, L. M. (1995). Robust and powerful nonorthogonal analyses. Psychometrika, 60, 395–418.CrossRefGoogle Scholar
Keselman, H. J., Carriere, K. C., & Lix, L. M. (1996). Errata to “Robust and powerful nonorthogonal analyses. Psychometrika, 61, 191–191.CrossRefGoogle Scholar
Keselman, H. J., Lix, L. M., & Kowalchuk, R. K. (1998). Multiple comparison procedures for trimmed means. Psychological Methods, 3, 123–141.CrossRefGoogle Scholar
Lewis, C., Keren, G. (1977). You can't have your cake and eat it too: Some considerations of the error term. Psychological Bulletin, 84, 1150–1154.CrossRefGoogle Scholar
Lix, L. M., Cribbie, R., & Keselman, H. J. (1996). The analysis of completely randomized univariate designs. Paper presented at the annual meeting of the Psychometric Society, Banff, AB, Canada.Google Scholar
Lix, L. M., & Keselman, H. J. (1995). To trim or not to trim: Tests of mean equality under heteroscedasticity and nonnormality. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.Google Scholar
Lix, L. M., & Keselman, H. J. (in press). To trim or not to trim: Tests of mean equality under heteroscedasticity and nonnormality, Educational and Psychological Measurement.Google Scholar
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166.CrossRefGoogle Scholar
Milligan, G. W., Wong, D. S., & Thompson, P. A. (1987). Robustness properties of nonorthogonal analysis of variance. Psychological Bulletin, 101, 464–470.CrossRefGoogle Scholar
Sawilowsky, S. S., & Blair, R. C. (1992). A more realistic look at the robustness and Type II error probabilities of the t test to departures from population normality. Psychological Bulletin, 111, 352–360.CrossRefGoogle Scholar
SAS Institute (1989). SAS/IML software: Usage and reference, version 6 1st ed.,, Cary, NC: Author.Google Scholar
Scheffe, H. (1959). The analysis of variance, New York: Wiley.Google Scholar
Tukey, J. W. (1960). A survey of sampling from contaminated normal distributions. In Olkin, I., Hoeffding, W., Ghurye, S., Madow, W., Mann, H. (Eds.), Contributions to probability and statistics, Stanford, CA: Stanford University Press.Google Scholar
Welch, B. L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika, 29, 350–362.CrossRefGoogle Scholar
Welch, B. L. (1947). The generalization of Student's problem when several different population variances are involved. Biometrika, 34, 28–35.Google ScholarPubMed
Welch, B. L. (1951). On the comparison of several mean values: An alternative approach. Biometrika, 38, 330–336.CrossRefGoogle Scholar
Wilcox, R. R. (1987). New designs in the analysis of variance. Annual Review of Psychology, 38, 29–60.CrossRefGoogle Scholar
Wilcox, R. R. (1990). Comparing the means of two independent groups. Biometrical Journal, 32, 771–780.CrossRefGoogle Scholar
Wilcox, R. R. (1994). A one-way random effects model for trimmed means. Psychometrika, 59, 289–306.CrossRefGoogle Scholar
Wilcox, R. R. (1994). Some results on the Tukey-McLaughlin and Yuen methods for trimmed means when distributions are skewed. Biometrical Journal, 36, 259–273.CrossRefGoogle Scholar
Wilcox, R. R. (1995). ANOVA: A paradigm for low power and misleading measures of effect size?. Review of Educational Research, 65, 51–77.CrossRefGoogle Scholar
Wilcox, R. R. (1995). ANOVA: The practical importance of heteroscedastic methods, using trimmed means versus means, and designing simulation studies. British Journal of Mathematical and Statistical Psychology, 48, 99–114.CrossRefGoogle Scholar
Wilcox, R. R. (1995). Three multiple comparison procedures for trimmed means. Biometrical Journal, 37, 643–656.CrossRefGoogle Scholar
Wilcox, R. R. (1996). Statistics for the social Sciences, New York: Academic Press.Google Scholar
Wilcox, R. R., Charlin, V. L., & Thompson, K. L. (1986). New Monte Carlo results on the robustness of the ANOVA F, W and F* statistics. Communications in Statistics, Simulation and Computation, 15(4), 933–943.CrossRefGoogle Scholar
Yuen, K. K. (1974). The two-sample trimmed t for unequal population variances. Biometrika, 61, 165–170.CrossRefGoogle Scholar