Hostname: page-component-745bb68f8f-b6zl4 Total loading time: 0 Render date: 2025-01-07T19:24:57.641Z Has data issue: false hasContentIssue false
  • English
  • Français

The Effect of Difficulty and Chance Success on Item-Test Correlation and on Test Reliability

Published online by Cambridge University Press:  01 January 2025

Lynnette B. Plumlee
Lynnette B. Plumlee*
Affiliation:
Educational Testing Service
Get access Rights & Permissions [Opens in a new window]

Abstract

An equation is derived for predicting the effect of chance success, relative to item difficulty, on item-test correlation. The values predicted by this equation and by equations derived by Guilford and Carroll for predicting the effect of chance success on item difficulty and test reliability are compared with empirical values in an experiment which used identical test items in multiple-choice and answer-only form.

Type
Original Paper
Information
Psychometrika , Volume 17 , Issue 1 , March 1952 , pp. 69 - 86
Copyright
Copyright © 1952 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Condensation of a dissertation presented in partial fulfillment of the requirements for the Ph.D. degree to the University of Chicago. Grateful acknowledgment is made to Professor Harold Gulliksen for his guidance as thesis advisor and to Professor L. L. Thurstone and Dr. D. W. Fiske of the University of Chicago who served as members of the thesis committee. The author is also indebted to Professor S. S. Wilks for review of the derivations and development of statistical tests used in the thesis, to Dr. L. R Tucker for technical advice, and to Dr. W. G. Mollenkopf for critical comments on the derivations and interpretations. The writer expresses appreciation to the Educational Testing Service for making available its technical facilities, and to the University of Chicago for the flexible administrative arrangement which made this thesis possible.

References

Carroll, J. B. The effect of difficulty and chance success on correlations between items or between tests. Psychometrika, 1945, 10, 1–19.CrossRefGoogle Scholar
Denney, H. R., and Remmers, H. H. Reliability of multiple-choice measuring instruments as a function of the Spearman-Brown prophecy formula, II. J. educ. Psychol., 1940, 31, 699–704.Google Scholar
Guilford, J. P. The determination of item difficulty when chance success is a factor. Psychometrika, 1936, 1, 259–264.CrossRefGoogle Scholar
Horst, Paul. The chance element in the multiple choice test item. J. gen. Psychol., 1932, 6, 209–211.CrossRefGoogle Scholar
Horst, Paul. The difficulty of a multiple-choice test item. J. educ. Psychol., 1933, 24, 229–232.Google Scholar
Horst, Paul. The difficulty of multiple choice test item alternatives. J. exp. Psychol., 1932, 15, 469–472.CrossRefGoogle Scholar
Johnson, A. P. An index of item validity providing a correction for chance success. Psychometrika, 1947, 12, 51–58.Google ScholarPubMed
Lord, F. M. Reliability of multiple-choice tests as a function of number of choices per item. J. educ. Psychol., 1944, 35, 175–180.CrossRefGoogle Scholar
Remmers, H. H., and Adkins, R. M. Reliability of multiple-choice measuring instruments as a function of the Spearman-Brown prophecy formula, VI. J. educ. Psychol., 1942, 33, 385–390.CrossRefGoogle Scholar
Remmers, H. H., and Ewart, Edwin. Reliability of multiple-choice measuring instruments as a function of the Spearman-Brown prophecy formula, III. J. educ. Psychol., 1941, 32, 61–66.Google Scholar
Remmers, H. H., and House, J. M. Reliability of multiple-choice measuring instruments as a function of the Spearman-Brown prophecy formula, IV. J. educ. Psychol., 1941, 32, 372–376.CrossRefGoogle Scholar
Remmers, H. H., Karslake, Ruth, and Gage, N. L. Reliability of multiple-choice measuring instruments as a function of the Spearman-Brown prophecy formula, I.. J. educ. Psychol., 1940, 31, 583–590.CrossRefGoogle Scholar
Remmers, H. H., and Sageser, H. W. Reliability of multiple-choice measuring instruments as a function of the Spearman-Brown formula, V. J. educ. Psychol., 1941, 32, 445–451.CrossRefGoogle Scholar
Rider, P. R. An introduction to modern statistical methods, New York: John Wiley & Sons, Inc., 1939.Google Scholar
Votaw, D. F. Notes on validation of test items by comparison of widely spaced groups. J. educ. Psychol., 1934, 25, 185–191.CrossRefGoogle Scholar
Wilks, S. S. Elementary statistical analysis, Princeton, New Jersey: Princeton Univ. Press, 1948.CrossRefGoogle Scholar
Wilks, S. S. Unpublished notes on the derivations of the confidence limits of the regression intercept.Google Scholar