Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-07T19:16:24.973Z Has data issue: false hasContentIssue false
  • English
  • Français

Efficiency of Multiple-Choice Tests as a Function of Spread of Item Difficulties

Published online by Cambridge University Press:  01 January 2025

Lee J. Cronbach  and
Willard G. Warrington
Lee J. Cronbach
Affiliation:
University of Illinois
Willard G. Warrington
Affiliation:
University of Illinois
Get access Rights & Permissions [Opens in a new window]

Abstract

The validity of a univocal multiple-choice test is determined for varying distributions of item difficulty and varying degrees of item precision. Validity is a function of σd2 + σy2, where σd measures item unreliability and σy measures the spread of item difficulties. When this variance is very small, validity is high for one optimum cutting score, but the test gives relatively little valid information for other cutting scores. As this variance increases, eta increases up to a certain point, and then begins to decrease. Screening validity at the optimum cutting score declines as this variance increases, but the test becomes much more flexible, maintaining the same validity for a wide range of cutting scores. For items of the type ordinarily used in psychological tests, the test with uniform item difficulty gives greater over-all validity, and superior validity for most cutting scores, compared to a test with a range of item difficulties. When a multiple-choice test is intended to reject the poorest F per cent of the men tested, items should on the average be located at or above the threshold for men whose true ability is at the Fth percentile.

Type
Original Paper
Information
Psychometrika , Volume 17 , Issue 2 , June 1952 , pp. 127 - 147
Copyright
Copyright © 1952 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

This research was performed under contract Nop 536 with the Bureau of Naval Personnel, and received additional support from the Bureau of Research and Service, College of Education, University of Illinois.

References

Brogden, H. E. Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation. Psychometrika, 1946, 11, 197–214.CrossRefGoogle ScholarPubMed
Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests. Psychometrika, 1945, 10, 1–19.CrossRefGoogle Scholar
Gulliksen, H. The relation of item difficulty and interitem correlation to test variance and reliability. Psychometrika, 1945, 10, 79–91.CrossRefGoogle Scholar
Lord, F. M. A theory of test scores and their relation to the trait measured. Res. Bull. 51–13, Educational Testing Service, 1951. See also A theory of test scores. Psychometric Monograph No. 7, 1952.CrossRefGoogle Scholar
Richardson, M. W. The relation between the difficulty and the differential validity of a test. Psychometrika, 1936, 1, 33–49.CrossRefGoogle Scholar
Tucker, L. R. Maximum validity of a test with equivalent items. Psychometrika, 1946, 11, 1–13.CrossRefGoogle ScholarPubMed