Hostname: page-component-5f745c7db-tvc9f Total loading time: 0 Render date: 2025-01-06T06:53:30.889Z Has data issue: true hasContentIssue false
  • English
  • Français

Robust and Powerful Nonorthogonal Analyses

Published online by Cambridge University Press:  01 January 2025

H. J. Keselman ,
K. C. Carriere  and
Lisa M. Lix
H. J. Keselman*
Affiliation:
Department of Community Health Sciences
K. C. Carriere
Affiliation:
Department of Community Health Sciences
Lisa M. Lix
Affiliation:
University of Manitoba, Winnipeg
*
Requests for reprints should be addressed to H. J. Keselman, Department of Psychology, University of Manitoba, Winnipeg, MB, R3T 2N2 CANADA.
Get access Rights & Permissions [Opens in a new window]

Abstract

Numerous types of analyses for factorial designs having unequal cell frequencies have been discussed in the literature. These analyses test either weighted or unweighted marginal means which, in turn, correspond to different model comparisons. Previous research has indicated, however, that these analyses result in biased (liberal or conservative) tests when cell variances are heterogeneous. We show how to obtain a generally robust and powerful analysis with any of the recommended nonorthogonal solutions by adapting a modification of the Welch-James procedure for comparing means when population variances are heterogeneous.

Keywords

nonorthogonal designsweighted and unweighted meansmodel comparisonsrobust and powerful solution
Type
Original Paper
Information
Psychometrika , Volume 60 , Issue 3 , September 1995 , pp. 395 - 418
Copyright
Copyright © 1995 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was supported by a Social Sciences and Humanities Research Council (SSHRC) grant (# 410-92-0430) to the first author, a Manitoba Health Research Council Scholar Award and a grant from Natural Sciences and Engineering Research Council to the second author, and a SSHRC Doctoral Fellowship (# 752-92-1628) to the third author. The authors would like to express their gratitude to Joanne Keselman and three anonymous reviewers for their many helpful substantive comments on earlier drafts of this paper.

References

Akritas, M. G. (1991). Limitations of the rank transform procedure: A study of repeated measures designs, Part I. Journal of the American Statistical Association, 86, 457–460.CrossRefGoogle Scholar
Algina, J., Olejnik, S. F. (1984). Implementing the Welch-James procedure with factorial designs. Educational and Psychological Measurement, 44, 39–48.CrossRefGoogle Scholar
Algina, J., Oshima, T. C., Tang, K. L. (1991). Robustness of Yao's, James', and Johansen's tests under variance-covariance heteroscedasticity and nonnormality. Journal of Educational Statistics, 16, 125–140.Google Scholar
Algina, J., Tang, K. L. (1988). Type I error rates for Yao's and James' tests of equality of mean vector under variance-covariance heteroscedasticity. Journal of Educational Statistics, 13, 281–290.Google Scholar
Appelbaum, M. I., Cramer, E. M. (1974). Some problems in nonorthogonal analysis of variance. Psychological Bulletin, 81, 335–343.CrossRefGoogle Scholar
Blair, R. C., Sawilowsky, S., Higgins, J. J. (1987). Limitations of the rank transform statistic in tests for interactions. Communications in Statistics B, Simulation and Computation, 16, 1133–1145.CrossRefGoogle Scholar
Box, G. E. P., Cox, D. R. (1964). An analysis of transformations. Journal of the Royal Statistical Society B, 26, 211–243.CrossRefGoogle Scholar
Bradley, J. V. (1978). Robustness?. British Journal of Mathematical and Statistical Psychology, 31, 144–152.CrossRefGoogle Scholar
Brown, M. B., Forsythe, A. B. (1974). The ANOVA and multiple comparisons for data with heterogeneous variances. Biometrics, 30, 719–724.CrossRefGoogle Scholar
Brown, M. B., Forsythe, A. B. (1974). The small sample behavior of some statistics which test the equality of several means. Technometrics, 16, 129–132.CrossRefGoogle Scholar
Brunner, E., Dette, H. (1992). Rank procedures for the two-factor mixed model. Journal of the American Statistical Association, 87, 884–888.CrossRefGoogle Scholar
Budescu, D. V., Appelbaum, M. I. (1981). Variance stabilizing transformations and the power of the F-test. Journal of Educational Statistics, 6, 55–74.CrossRefGoogle Scholar
Burdick, D. S., Herr, D. G. (1980). Counterexamples in unbalanced two-analysis of variance. Communications in Statistics-Theory and Methods, A9, 231–241.CrossRefGoogle Scholar
Carlson, J. E., Timm, N. H. (1974). Analysis of nonorthogonal fixed-effects designs. Psychological Bulletin, 81, 563–570.CrossRefGoogle Scholar
Cramer, E. M., Appelbaum, M. I. (1980). Nonorthogonal analysis of variance—Once again. Psychological Bulletin, 87, 51–57.CrossRefGoogle Scholar
Emerson, J. D., Stoto, M. A. (1983). Transforming data. In Hoaglin, D. C., Mosteller, F., Tukey, J. W. (Eds.), Understanding robust and exploratory data analysis (pp. 97–128). New York: Wiley.Google Scholar
Flinger, M. A. (1981). Comment. Journal of the American Statistical Association, 35, 131–132.Google Scholar
Freund, R. J., Littell, R. C. (1981). SAS for linear models: A guide to the ANOVA and GLM procedures, Cary, NC: SAS Institute.Google Scholar
Games, P. A. (1983). Curvilinear transformation of the dependent variable. Psychological Bulletin, 93, 382–387.CrossRefGoogle Scholar
Hamer, R. M., Hosking, J. D. (1977). The nonorthogonal analysis of variance. Representative Research in Social Psychology, 8, 71–87.Google Scholar
Harwell, M. R., Rubinstein, E. N., Hayes, W. S., Olds, C. C. (1992). Summarizing Monte Carlo results in methodological research: The one- and two-factor fixed effects ANOVA cases. Journal of Educational Statistics, 17, 315–340.CrossRefGoogle Scholar
Herr, D. G., Gaebelein, J. (1978). Nonorthogonal two way analysis of variance. Psychological Bulletin, 85, 207–216.CrossRefGoogle Scholar
Hocking, R. R., Speed, F. M. (1975). A full rank analysis of some linear model problems. Journal of the American Statistical Association, 70, 706–712.Google Scholar
Howell, D. C., McConaughy, S. H. (1982). Nonorthogonal analysis of variance: Putting the question before the answer. Educational and Psychological Measurement, 42, 9–24.CrossRefGoogle Scholar
Hsiung, T., & Olejnik, S. (1994, April). Type I error rates and statistical power for the James second-order test and the univariate F test in two-way fixed-effects ANOVA models under heteroscedasticity and/or nonnormality. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.Google Scholar
James, G. S. (1951). The comparison of several groups of observations when the ratios of the population variances are unknown. Biometrika, 38, 324–329.CrossRefGoogle Scholar
James, G. S. (1954). Tests of linear hypotheses in univariate and multivariate analysis when the ratios of the population variances are unknown. Biometrika, 41, 19–43.Google Scholar
Johansen, S. (1980). The Welch-James approximation of the distribution of the residual sum of squares in weighted linear regression. Biometrika, 67, 85–92.CrossRefGoogle Scholar
Katz, B. M., McSweeney, M. (1983). Some non-parametric tests for analysing ranked data in multi-group repeated measures designs. British Journal of Mathematical and Statistical Psychology, 36, 145–156.CrossRefGoogle Scholar
Keren, G. (1982). A balanced approach to unbalanced designs. In Keren, G. (Eds.), Statistical and methodological issues in psychology and social science research (pp. 155–186). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Keselman, H. J., Carriere, K. C., Lix, L. M. (1993). Testing repeated measures hypotheses when covariance matrices are heterogeneous. Journal of Educational Statistics, 18, 305–319.CrossRefGoogle Scholar
Kleijnen, J. P. C., Brent, R., Bowers, R. (1981). Small-sample behavior of weighted least squares in experimental design applications. Communication in Statistics B, Simulation and Computation, 10, 303–313.CrossRefGoogle Scholar
Kleijnen, J. P. C., Cremers, P., van Belle, F. (1985). The power of weighted and ordinary least squares with estimated unequal variances in experimental design. Communications in Statistics B, Simulation and Computation, 14, 85–102.CrossRefGoogle Scholar
Koch, G. G., Amara, I. A., Stokes, M. E., Gillings, D. B. (1980). Some views on parametric and non-parametric analysis for repeated measurements and selected bibliography. International Statistical Review, 48, 249–265.CrossRefGoogle Scholar
Kutner, M. H. (1974). Hypothesis testing in linear models (Eisenhart Model I). The American Statistician, 28, 98–100.CrossRefGoogle Scholar
Lewis, C., Keren, G. (1977). You can't have your cake and eat it too: Some considerations of the error term. Psychological Bulletin, 84, 1150–1154.CrossRefGoogle Scholar
Lix, L. M., Keselman, H. J. (1995). Approximate degrees of freedom tests: A unified perspective on testing for mean equality. Psychological Bulletin, 117, 547–560.CrossRefGoogle Scholar
Maxwell, S. E., Delaney, H. D. (1990). Designing experiments and analyzing data. A model comparison perspective, Belmont, CA: Wadsworth.Google Scholar
Micceri, T. (1989). The unicorn, the normal distribution, and other improbable creatures. Psychological Bulletin, 105, 156–166.CrossRefGoogle Scholar
Milligan, G. W., Wong, D. S., Thompson, P. A. (1987). Robustness properties of nonorthogonal analysis of variance. Psychological Bulletin, 101, 464–470.CrossRefGoogle Scholar
Nozari, A. (1984). Generalized and ordinary least squares with estimated and unequal variances. Communications in Statistics B, Simulation and Computation, 13, 521–537.CrossRefGoogle Scholar
Overall, J. E., Spiegel, D. K. (1969). Concerning least squares analysis of experimental data. Psychological Bulletin, 72, 311–322.CrossRefGoogle Scholar
SAS Institute (1989). SAS language: Reference, Version 6 1st ed.,, Cary, NC: Author.Google Scholar
SAS Institute (1989). SAS/STAT user's guide, Version 6 4th ed.,, Cary, NC: Author.Google Scholar
SAS Institute (1989). SAS/IML software: Usage and reference, version 6 1st ed.,, Cary, NC: Author.Google Scholar
Sawilowsky, S. S., Blair, R. C. (1992). A more realistic look at the robustness and Type II error probabilities of the t test to departures from population normality. Psychological Bulletin, 111, 352–360.CrossRefGoogle Scholar
Scheffé, H. (1959). The analysis of variance, New York: Wiley.Google Scholar
Searle, S. R. (1987). Linear models for unbalanced data, Toronto: Wiley.Google Scholar
Speed, F. M., Hocking, R. R. (1976). The use of the R( )-notation with unbalanced data. The American Statistician, 30, 30–33.Google Scholar
Speed, F. M., Hocking, R. R., Hackney, O. P. (1978). Methods of analysis of linear models with unbalanced data. Journal of the American Statistical Association, 73, 105–112.CrossRefGoogle Scholar
Timm, N. H., Carlson, J. E. (1975). Analysis of variance through full rank models. Multivariate Behavioral Research Monographs, 75–1, 1–120.Google Scholar
Tomarken, A. J., Serlin, R. C. (1986). Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures. Psychological Bulletin, 99, 90–99.CrossRefGoogle Scholar
Welch, B. L. (1947). The generalization of Students' problem when several different population variances are involved. Biometrika, 34, 28–35.Google ScholarPubMed
Welch, B. L. (1951). On the comparison of several mean values. An alternative approach. Biometrika, 38, 330–336.CrossRefGoogle Scholar
Wilcox, R. R. (1987). New statistical procedures for the social sciences. Modern solutions to basic problems, New Jersey: Lawrence Erlbaum.Google Scholar
Wilcox, R. R., Charlin, V. L., Thompson, K. L. (1986). New Monte Carlo results on the robustness of the ANOVAF, W and F* statistics. Communications in Statistics B, Simulation and Computation, 15, 933–943.CrossRefGoogle Scholar

A correction has been issued for this article:

Errata