Hostname: page-component-745bb68f8f-grxwn Total loading time: 0 Render date: 2025-01-08T15:14:14.729Z Has data issue: false hasContentIssue false
  • English
  • Français

Estimation of Reliability and True Score Variance from a Split of a Test into Three Arbitrary Parts

Published online by Cambridge University Press:  01 January 2025

Walter Kristof
Walter Kristof*
Affiliation:
Educational Testing Service
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper gives a method of estimating the reliability of a test which has been divided into three parts. The parts do not have to satisfy any statistical criteria like parallelism or τ-equivalence. If the parts are homogeneous in content (congeneric), i.e., if their true scores are linearly related and if sample size is large then the method described in this paper will give the precise value of the reliability parameter. If the homogeneity condition is violated then underestimation will typically result. However, the estimate will always be at least as accurate as coefficient α and Guttman's lower bound λ3 when the same data are used. An application to real data is presented by way of illustration. Seven different splits of the same test are analyzed. The new method yields remarkably stable reliability estimates across splits as predicted by the theory. One deviating value can be accounted for by a certain unsuspected peculiarity of the test composition. Both coefficient α and λ3 would not have led to the same discovery.

Type
Original Paper
Information
Psychometrika , Volume 39 , Issue 4 , December 1974 , pp. 491 - 499
Copyright
Copyright © 1974 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

Expanded version of a paper given at the Psychometric Society Meeting in Stanford, California, March 1974.

*

Present affiliation: Seminar für Sozialwissenschaften, University of Hamburg, Germany.

References

Cronbach, L. J. Coefficient alpha and the internal structure of tests. Psychometrika, 1951, 16, 297–334.CrossRefGoogle Scholar
Feldt, L. S. The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 1965, 30, 357–370.CrossRefGoogle ScholarPubMed
Guttman, L. A basis for analyzing test-retest reliability. Psychometrika, 1945, 10, 255–282.CrossRefGoogle ScholarPubMed
Kristof, W. The statistical theory of stepped-up reliability coefficients when a test has been divided into several equivalent parts. Psychometrika, 1963, 28, 221–238.CrossRefGoogle Scholar
Kristof, W. On a statistic arising in testing correlation. Psychometrika, 1972, 37, 377–384.CrossRefGoogle Scholar
Lord, F. M. and Novick, M. R. Statistical theories of mental test scores, 1968, Reading, Mass.: Addison-Wesley.Google Scholar
Novick, M. R. and Lewis, C. Coefficient alpha and the reliability of composite measurements. Psychometrika, 1967, 32, 1–13.CrossRefGoogle ScholarPubMed
Olkin, I. and Pratt, J. W. Unbiased estimation of certain correlation coefficients. Annals of Mathematical Statistics, 1958, 29, 201–211.CrossRefGoogle Scholar
Payne, W. H. and Anderson, D. E. Significance levels for the Kuder-Richardson twenty: An automated sampling experiment approach. Educational and Psychological Measurement, 1968, 28, 23–39.CrossRefGoogle Scholar
Werts, C. E., Linn, R. L. and Jöreskog, K. G. Intraclass reliability estimates: testing structural assumptions. Educational and Psychological Measurement, 1974, 34, 25–33.CrossRefGoogle Scholar