Hostname: page-component-745bb68f8f-cphqk Total loading time: 0 Render date: 2025-01-07T16:41:47.323Z Has data issue: false hasContentIssue false
  • English
  • Français

Tests for Equality of Several Alpha Coefficients when their Sample Estimates are Dependent

Published online by Cambridge University Press:  01 January 2025

David J. Woodruff  and
Leonard S. Feldt
David J. Woodruff*
Affiliation:
The American College Testing Program
Leonard S. Feldt
Affiliation:
The University of Iowa
*
Requests for reprints should be addressed to David Woodruff, Measurement Research Department, The American College Testing Program, PO Box 168, Iowa City, IA 52243.
Get access Rights & Permissions [Opens in a new window]

Abstract

In a variety of measurement situations, the researcher may wish to compare the reliabilities of several instruments administered to the same sample of subjects. This paper presents eleven statistical procedures which test the equality of m coefficient alphas when the sample alpha coefficients are dependent. Several of the procedures are derived in detail, and numerical examples are given for two. Since all of the procedures depend on approximate asymptotic results, Monte Carlo methods are used to assess the accuracy of the procedures for sample sizes of 50, 100, and 200. Both control of Type I error and power are evaluated by computer simulation. Two of the procedures are unable to control Type I errors satisfactorily. The remaining nine procedures perform properly, but three are somewhat superior in power and Type I error control.

Keywords

coefficient alphatest reliabilityinternal consistencyhypothesis testingrepeated measures
Type
Original Paper
Information
Psychometrika , Volume 51 , Issue 3 , September 1986 , pp. 393 - 413
Copyright
Copyright © 1986 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

A more detailed version of this paper is also available.

Acknowledgments: Shin-Ichi Mayekawa provided valuable advice and assistance with certain statistical derivations and computational algorithms. Suggestions by anonymous reviewers significantly improved the clarity of the paper.

References

Adegboye, O. S., & Gupta, A. K. (1981). On a multivariateF distribution and its applications. Proceedings of the American Statistical Asociation (Social Statistics Section).Google Scholar
Bartlett, M. S. (1937). Properties of sufficiency and statistical tests. Proceedings of the Royal Society (Series A), 160, 268–282.Google Scholar
Bentler, P. M. (1983). Some contributions to efficient statistics in structural models: Specification and estimation of moment structures. Psychometrika, 48, 493–517.CrossRefGoogle Scholar
Browne, M. W. (1968). A comparison of factor analytic techniques. Psychometrika, 33, 267–334.CrossRefGoogle ScholarPubMed
Browne, M. W. (1974). Generalized least squares estimators in the analysis of covariance structures. South African Statistical Journal, 8, 1–24.Google Scholar
Browne, M. W. (1984). Asymptotic distribution-free methods for the analysis of covariance structure. British Journal of Mathematical and Statistical Psychology, 37, 62–83.CrossRefGoogle Scholar
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.CrossRefGoogle Scholar
Dunn, O. J. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56, 52–64.CrossRefGoogle Scholar
Dykstra, R. L., Hewet, J. E., Bulgren, W. (1979). A note on multivariateF distributions. Sankhya: The Indian Journal of Statistics (Series B), 41, 31–40.Google Scholar
Everitt, B. S. (1979). A Monte Carlo investigation of the robustness of Hotelling's one- and two-sample T2 tests. Journal of the American Statistical Association, 74, 48–51.Google Scholar
Feldt, L. S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30, 357–370.CrossRefGoogle ScholarPubMed
Feldt, L. S. (1969). A test of the hypothesis that Cronbach's alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363–373.CrossRefGoogle Scholar
Feldt, L. S. (1980). A test of the hypothesis that Cronbach's alpha reliability coefficient is the same for two tests administered to the same sample. Psychometrika, 45, 99–105.CrossRefGoogle Scholar
Ferger, W. F. (1931). The nature and uses of the harmonic mean. Journal of the American Statistical Association, 26, 36–40.CrossRefGoogle Scholar
Hakstian, A. R., Whalen, T. E. (1976). Ak-sample significance test for independent alpha coefficients. Psychometrika, 41, 219–231.CrossRefGoogle Scholar
Han, C. P. (1968). Testing the homogeneity of a set of correlated variances. Biometrika, 55, 317–326.CrossRefGoogle Scholar
Hartley, H. O. (1950). The maximumF-ratio as a short cut test for heterogeneity of variance. Biometrika, 37, 308–312.Google Scholar
Hogg, R. V., Craig, A. T. (1978). Introduction to mathematical statistics 4th ed.,, New York: Macmillan.Google Scholar
Hoyt, C. (1941). Test reliability estimated by analysis of variance. Psychometrika, 6, 153–160.CrossRefGoogle Scholar
Hsu, T. C., Feldt, L. S. (1969). The effect of limitation on the number of criterion score values on the significance level of theF-test. American Educational Research Journal, 6, 515–527.Google Scholar
IMSL (1980). International mathematical and statistical libraries 8th ed.,, Houston: Author.Google Scholar
Jensen, D. R. (1970). The joint distributions of the traces of Wishart matrices and some applications. Annals of Mathematical Statistics, 41, 133–146.CrossRefGoogle Scholar
Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109–133.CrossRefGoogle Scholar
Jöreskog, K. G., Sörbom, D. (1981). LISREL V, Chicago: International Educational Services.Google Scholar
Kendall, M. G., Stuart, A. (1977). The advanced theory of statistics 4th ed.,, London: Griffin.Google Scholar
Kenney, J. F. (1939). Mathematics of statistics, New York: D. Van Nostrand.Google Scholar
Kraemer, H. C. (1981). Extension of Feldt's approach to testing homogeneity of coefficients of reliability. Psychometrika, 46, 41–45.CrossRefGoogle Scholar
Krishnaiah, P. R., Armitage, J. V.et al. (1970). On a multivariateF distribution. In Bose, R. C.et al. (Eds.), Essays in Probability and Statistics, Chapell Hill: University of North Carolina Press.Google Scholar
Kristof, W. (1963). The statistical theory of stepped-up reliability coefficients when a test has been divided into several equivalent parts. Psychometrika, 28, 221–238.CrossRefGoogle Scholar
Kristof, W. (1964). Testing differences between reliability coefficients. The British Journal of Mathematical and Statistical Psychology, 17, 105–111.CrossRefGoogle Scholar
Kuder, G. F., Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151–160.CrossRefGoogle Scholar
Lord, F. M. (1955). A survey of observed test-score distributions with respect to skewness and kurtosis. Educational and Psychological Measurement, 15, 383–389.CrossRefGoogle Scholar
Myers, J. L. (1979). Fundamentals of experimental design, Boston: Allyn and Bacon.Google Scholar
Odell, P. L., Feiveson, A. H. (1966). A numerical procedure to generate a sample covariance matrix. Journal American Statistical Association, 61, 199–203.CrossRefGoogle Scholar
Paulson, E. (1942). An approximate normalization of the analysis of variance distribution. Annals of Mathematical Statistics, 13, 233–235.CrossRefGoogle Scholar
Pearson, E. S., Hartley, H. O. (1970). Biometrika tables for statisticians 3rd ed.,, Cambridge: University Press.Google Scholar
Pitman, E. J. G. (1939). A note on normal correlation. Biometrika, 31, 9–12.CrossRefGoogle Scholar
Seeger, P., Gabrielsson, A. (1968). Applicability of the CochranQ test and theF test for statistical analysis of dichotomous data for dependent samples. Psychological Bulletin, 69, 269–277.CrossRefGoogle Scholar
Scheffé, H. (1959). The analysis of variance, New York: Wiley.Google Scholar
Werts, C. E., Grandy, J., Schabacker, W. H. (1980). A confirmatory approach to calibrating congeneric measures. Multivariate Behavioral Research, 15, 109–122.CrossRefGoogle ScholarPubMed
Wilson, E. B., Hilferty, M. M. (1931). The distribution of chi-square. Proceedings of the Natural Academy of Sciences, USA, 17, 684–684.CrossRefGoogle ScholarPubMed