Hostname: page-component-745bb68f8f-5r2nc Total loading time: 0 Render date: 2025-01-07T19:27:36.208Z Has data issue: false hasContentIssue false
  • English
  • Français

The Relation of the Reliability of Multiple-Choice Tests to the Distribution of Item Difficulties

Published online by Cambridge University Press:  01 January 2025

Frederic M. Lord
Frederic M. Lord*
Affiliation:
Educational Testing Service
Get access Rights & Permissions [Opens in a new window]

Abstract

Under certain assumptions an expression, in terms of item difficulties and intercorrelations, is derived for the curvilinear correlation of test score on the “ability underlying the test,” this ability being defined as the common factor of the item tetrachoric intercorrelations corrected for guessing. It is shown that this curvilinear correlation is equal to the square root of the test reliability. Numerical values for these curvilinear correlations are presented for a number of hypothetical tests, defined in terms of their item parameters. These numerical results indicate that the reliability and the curvilinear correlation will be maximized by (1) minimizing the variability of item difficulty and (2) making the level of item difficulty somewhat easier than the halfway point between a chance percentage of correct answers and 100 per cent correct answers.

Type
Original Paper
Information
Psychometrika , Volume 17 , Issue 2 , June 1952 , pp. 181 - 194
Copyright
Copyright © 1952 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Brogden, H. E. Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation. Psychometrika, 1946, 11, 197–214.CrossRefGoogle ScholarPubMed
Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests. Psychometrika, 1945, 10, 1–20.CrossRefGoogle Scholar
Cronbach, L. J., and Warrington, W. G. Design study for sonar pitch memory test, Urbana, Ill.: Bureau of Research and Service, College of Education, Univ. of Illinois, 1951.Google Scholar
Gulliksen, H. The relation of item difficulty and inter-item correlation to test variance and reliability. Psychometrika, 1945, 10, 79–91.CrossRefGoogle Scholar
Kuder, G. F., and Richardson, M. W. The theory of the estimation of test reliability. Psychometrika, 1937, 2, 151–160.CrossRefGoogle Scholar
Lord, F. M. A theory of test scores. Psychometric Monograph No. 7, 1952.Google Scholar
Pearson, K. Tables for statisticians and biometricians, London: Cambridge Univ. Press, 1924.Google Scholar
Plumlee, L. B. The effect of difficulty and chance success on item-test correlations and test reliability. Psychometrika, 1952, 17, 69–86.CrossRefGoogle Scholar
Tucker, L. R. Maximum validity of a test with equivalent items. Psychometrika, 1946, 11, 1–13.CrossRefGoogle ScholarPubMed
Wherry, R. J., and Gaylord, R. H. Factor pattern of test items and tests as a function of the correlation coefficient: content, difficulty, and constant error factors. Psychometrika, 1944, 9, 237–244.CrossRefGoogle Scholar
Yule, G. U., and Kendall, M. G. An introduction to the theory of statistics, London: Charles Griffin and Company, 1940.Google Scholar