Hostname: page-component-745bb68f8f-hvd4g Total loading time: 0 Render date: 2025-01-07T19:18:35.774Z Has data issue: false hasContentIssue false
  • English
  • Français

The Estimation of True Score Variance and Error Variance in the Classical Test Theory Model

Published online by Cambridge University Press:  01 January 2025

Paul H. Jackson
Paul H. Jackson*
Affiliation:
University College of Wales
Get access Rights & Permissions [Opens in a new window]

Abstract

The use of one-way analysis of variance tables for obtaining unbiased estimates of true score variance and error score variance in the classical test theory model is discussed. Attention is paid to both balanced (equal numbers of observations on each person) and unbalanced designs, and estimates provided for both homoscedastic (common error variance for all persons) and heteroscedastic cases.

It is noted that optimality properties (minimum variance) can be claimed for estimates derived from analysis of variance tables only in the balanced, homoscedastic case, and that there they are essentially a reflection of the symmetry inherent in the situation. Estimates which might be preferable in other cases are discussed. An example is given where a natural analysis of variance table leads to estimates which cannot be derived from the set of statistics which is sufficient under normality assumptions. Reference is made to Bayesian studies which shed light on the difficulties encountered.

Type
Original Paper
Information
Psychometrika , Volume 38 , Issue 2 , June 1973 , pp. 183 - 201
Copyright
Copyright © 1973 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Work on this paper was carried out at the headquarters of the American College Testing Program, Iowa City, Iowa, while the author was on leave from the University College of Wales.

References

Graybill, F. A., & Hultquist, F. A. Theorems concerning Eisenhart's Model II. Annals of Mathematical Statistics, 1961, 32, 261–269CrossRefGoogle Scholar
Harville, D. A. Quadratic unbiased estimation of variance components for the one-way classification. Biometrika, 1969, 56, 313–326CrossRefGoogle Scholar
Harville, D. A. Variance component estimation for the unbalanced one-way random classification-a critique. Aerospace Research Laboratories Report ARL 69-0180, 1969(b).Google Scholar
Hewitt, E., & Savage, L. J. Symmetric measures on cartesian products. Transactions of the American Mathematical Society, 1955, 80, 470–501CrossRefGoogle Scholar
Hill, B. M. Inference about variance components in the one-way model. Journal of the American Statistical Association, 1965, 60, 806–825CrossRefGoogle Scholar
Jackson, P. H. Simple approximations in the estimation of many parameters, 1972, Iowa City, Iowa: The American College Testing ProgramCrossRefGoogle Scholar
Lindley, D. V. The estimation of many parameters. In Godambe, V. P. and Sprott, D. A. (Eds.), foundations of Statistical Inference. Toronto: Holt, Rinehart, and Winston. 1971, 435–455Google Scholar
Novick, M. R. Bayesian considerations in educational information systems. In Glass, G. V. (Eds.), Proceedings of the 1970 Invitational Conference on Testing Problems, 1971, Princeton, New Jersey: Educational Testing ServiceGoogle Scholar
Novick, M. R., Jackson, P. H., and Thayer, D. T. Bayesian inference and the classical test theory model: Reliability and true scores. Psychometrika, 1971, 36, 261–288CrossRefGoogle Scholar
Tiao, G. C., & Tan, W. Y. Bayesian analysis of random-effect models in the analysis of variance: I Posterior distribution of variance components. Biometrika, 1965, 52, 37–53CrossRefGoogle Scholar
Tukey, J. W. Variances of variance components: II The unbalanced single classification. Annals of Mathematical Statistics, 1957, 28, 43–56CrossRefGoogle Scholar