Hostname: page-component-745bb68f8f-grxwn Total loading time: 0 Render date: 2025-01-07T18:32:09.607Z Has data issue: false hasContentIssue false
  • English
  • Français

The Concept of Test and Item Reliability in Relation to Factor Pattern

Published online by Cambridge University Press:  01 January 2025

Robert J. Wherry  and
Richard H. Gaylord
Robert J. Wherry
Affiliation:
University of North Carolina
Richard H. Gaylord
Affiliation:
University of North Carolina
Get access Rights & Permissions [Opens in a new window]

Abstract

It is shown that approaches other than the internal consistency method of estimating test reliability are either less satisfactory or lead to the same general results. The commonly attendant assumption of a single factor throughout the test items is challenged, however. The consideration of a test made up of K sub-tests each composed of a different orthogonal factor disclosed that the assumption of a single factor produced an erroneous estimate of reliability with a ratio of (n−K)/(n−1) to the correct estimate. Special difficulties arising from this error in application of current techniques to short tests or to test batteries are discussed. Application of this same multi-factor concept to item-analysis discloses similar difficulties in that field. The item-test coefficient approaches √1/K as an upper limit rather than 1.00 and approaches √1/n as a lower limit rather than .00. This latter finding accounts for an over-estimation error in the Kuder-Richardson formula (8). A new method of isolating sub-tests based upon the item-test coefficient is proposed and tentatively outlined. Either this new method or a complete factor analysis is regarded as the only proper approach to the problem of test reliability, and the item-sub-est coefficient is similarly recommended as the proper approach for item analysis.

Type
Original Paper
Information
Psychometrika , Volume 8 , Issue 4 , December 1943 , pp. 247 - 264
Copyright
Copyright © 1943 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Brown, W. Some experimental results in the correlation of mental abilities. Brit. J. Psychol., 1909, 3, 296–322.Google Scholar
Edgerton, H. A., and Kolbe, L. E. The method of minimum variation for the combination of criteria. Psychometrika, 1936, 1, 183–187.CrossRefGoogle Scholar
Edgerton, H. A. Thomson, K. F. Test scores examined with the lexis ratio. Psychometrika, 1942, 7, 281–288.CrossRefGoogle Scholar
Horst, P. Obtaining a composite measure from a number of different measures of the same attribute. Psychometrika, 1936, 1, 53–60.CrossRefGoogle Scholar
Hotelling, H. The most predictable criterion. J. educ. Psychol., 1935, 26, 139–142.CrossRefGoogle Scholar
Hoyt, C. Test reliability estimated by analysis of variance. Psychometrika, 1941, 6, 153–160.CrossRefGoogle Scholar
Jackson, R. W. B. Reliability of mental tests. Brit. J. Psychol., 1938, 29, 267–287.Google Scholar
Kelley, T. L. The reliability coefficient. Psychometrika, 1942, 7, 75–83.CrossRefGoogle Scholar
Kuder, G. F., and Richardson, M. W. The theory of the estimation of reliability. Psychometrika, 1937, 2, 151–160.CrossRefGoogle Scholar
Lentz, T. F., and Whitmer, E. F. Item synonymization: a method for determining the total meaning of pencil-paper reactions. Psychometrika, 1941, 6, 131–139.CrossRefGoogle Scholar
Mosier, C. A note on item analysis and the criterion of internal consistency. Psychometrika, 1936, 1, 275–282.CrossRefGoogle Scholar
Paulsen, G. B. A coefficient of trait variability. Psychol. Bull., 1931, 28, 218–218.Google Scholar
Richardson, M. W. Note on the rationale of item analysis. Psychometrika, 1936, 1, 69–76.CrossRefGoogle Scholar
Royer, E. B. Some recent developments in test construction. Proc. Okla. Acad. Sci., 1936, 16, 107–109.Google Scholar
Spearman, C. Correlation calculated from faulty data. Brit. J. Psychol., 1909, 3, 271–295.Google Scholar
Thompson, G. A. Weighting for battery reliability and prediction. Brit. J. Psychol., 1939, 30, 357–366.Google Scholar
Thouless, R. H. Test unreliability and functional fluctuation. Brit. J. Psychol., 1935, 26, 325–343.Google Scholar
Woodrow, H. Quotidian variability. Psychol. Rev., 1932, 32, 245–256.CrossRefGoogle Scholar