Hostname: page-component-745bb68f8f-f46jp Total loading time: 0 Render date: 2025-01-07T18:47:37.722Z Has data issue: false hasContentIssue false
  • English
  • Français

Some Developments in Multivariate Generalizability

Published online by Cambridge University Press:  01 January 2025

George W. Joe  and
J. Arthur Woodward
George W. Joe*
Affiliation:
Texas Christian University
J. Arthur Woodward
Affiliation:
University of Texas Medical Branch, Galveston
*
Requests for reprints should be sent to George W. Joe, Institute of Behavioral Research, Texas Christian University, Fort Worth, Texas 76129.
Get access Rights & Permissions [Opens in a new window]

Abstract

This article is concerned with estimation of components of maximum generalizability in multifacet experimental designs involving multiple dependent measures. Within a Type II multivariate analysis of variance framework, components of maximum generalizability are defined as those composites of the dependent measures that maximize universe score variance for persons relative to observed score variance. The coefficient of maximum generalizability, expressed as a function of variance component matrices, is shown to equal the squared canonical correlation between true and observed scores. Emphasis is placed on estimation of variance component matrices, on the distinction between generalizability- and decision-studies, and on extension to multifacet designs involving crossed and nested facets. An example of a two-facet partially nested design is provided.

Keywords

reliabilityvariance component matricesmaximally reliable composite
Type
Original Paper
Information
Psychometrika , Volume 41 , Issue 2 , June 1976 , pp. 205 - 217
Copyright
Copyright © 1976 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Appreciation is expressed to the Office of Research in Medical Education, University of Texas Medical Branch, for permitting use of their data.

References

Bock, R. D. Contributions of multivariate experimental designs to educational research. In Cattell, R. B. (Eds.), Handbook of multivariate experimental psychology, 1966, Chicago: Rand McNally.Google Scholar
Bock, R. D. and Petersen, A. C. A multivariate correction for attentuation. Biometrika, 1975 (in press).CrossRefGoogle Scholar
Conger, A. J. Estimating profile reliability and maximally reliable composites. Multivariate Behavioral Research, 1974, 9, 85–104.CrossRefGoogle ScholarPubMed
Conger, A. J. and Lipshitz, R. Measures of reliability for profiles and test batteries. Psychometrika, 1973, 38, 411–427.CrossRefGoogle Scholar
Cronbach, L. J., Gleser, G. C., and Nanda, H., Rajaratnam, N. The dependability of behavioral measurements, 1972, New York: Wiley.Google Scholar
Graybill, F. A. An introduction to linear statistical models, 1961, New York: McGraw-Hill.Google Scholar
Green, B. F. Jr. A note on the calculation of weights for maximum battery reliability. Psychometrika, 1950, 15, 57–61.CrossRefGoogle Scholar
Levine, H. G., Bryan, G. T., Travis, L. B., Daeschner, C. W., and Bosshart, D. A. Handbook on the development of a system of instruction for medical students: Using examples from pediatrics education, 1974, Galveston: Department of Pediatrics and Office of Research in Medical Education: University of Texas Medical Branch.Google Scholar
Loevinger, J. Person and population as psychometric concepts. Psychological Review, 1965, 72, 143–155.CrossRefGoogle ScholarPubMed
Mosier, C. I. On the reliability of weighted composites. Psychometrika, 1943, 8, 161–168.CrossRefGoogle Scholar
Nelder, J. A. The interpretation of negative components of variance. Biometrika, 1954, 41, 544–548.CrossRefGoogle Scholar
Peel, E. A. Prediction of a complex criterion and battery reliability. British Journal of Psychology, 1948, 1, 84–94.Google Scholar
Scheffé, H. The analysis of variance, 1959, New York: Wiley.Google Scholar
Thompson, G. H. Weighting for battery reliability and prediction. British Journal of Psychology, 1940, 30, 357–366.Google Scholar
Thorndike, R. L. Reliability. In Anastasi, A. (Eds.), Testing Problems in Perspective, 1967, Washington: American Council on Education.Google Scholar