Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-07T18:42:36.408Z Has data issue: false hasContentIssue false
  • English
  • Français

A Subset Selection Technique for Scoring Items on a Multiple Choice Test

Published online by Cambridge University Press:  01 January 2025

Jean D. Gibbons ,
Ingram Olkin  and
Milton Sobel
Jean D. Gibbons
Affiliation:
University of Alabama
Ingram Olkin*
Affiliation:
Stanford University
Milton Sobel
Affiliation:
University of California, Santa Barbara
*
Requests for reprints should be sent to Ingram Olkin, Department of Statistics, Stanford University, Stanford, CA 94305.
Get access Rights & Permissions [Opens in a new window]

Abstract

On a multiple-choice test in which each item has k alternative responses, the test taker is permitted to choose any subset which he believes contains the one correct answer. A scoring system is devised that depends on the size of the subset and on whether or not the correct answer is eliminated. The mean and variance of the score per item are obtained. Methods are derived for determining the total number of items that should be included on the test so that the average score on all items can be regarded as a good measure of the subject's knowledge. Efficiency comparisons between conventional and the subset selection scoring procedures are made. The analogous problem of r > 1 correct answers for each item (with r fixed and known) is also considered.

Keywords

Public PolicyStatistical TheoryCorrect AnswerAverage ScoreGood Measure
Type
Original Paper
Information
Psychometrika , Volume 44 , Issue 3 , September 1979 , pp. 259 - 270
Copyright
Copyright © 1979 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The authors are grateful to M. Aitkin, C. Coombs, F. Lord, and the reviewers for their comments and suggestions.

References

Bechhofer, R. E. A single-sample multiple decision procedure for ranking means of normal populations with known variances. Annals of Mathematical Statistics, 1954, 25, 16–39.CrossRefGoogle Scholar
Collet, L. S. Elimination scoring: An empirical evaluation. Journal of Educational Measurement, 1971, 8, 209–214.CrossRefGoogle Scholar
Coombs, C. H. On the use of objective examination. Educational and Psychological Measurement, 1953, 13, 308–310.CrossRefGoogle Scholar
Coombs, C. H., Milholland, J. E. & Womer, F. B. The assessment of partial knowledge. Educational and Psychological Measurement, 1956, 16, 13–37.CrossRefGoogle Scholar
Dressel, P. L. & Schmid, J. Some modifications of the multiple-choice item. Educational and Psychological Measurement, 1953, 13, 574–595.CrossRefGoogle Scholar
de Finetti, D. Probability, induction and statistics, 1972, New York: John Wiley & Sons.Google Scholar
Gibbons, J. D., Olkin, I. & Sobel, M. Selecting and ordering populations: A new statistical methodology, 1977, New York: John Wiley & Sons.Google Scholar
Gupta, S. S. On a decision rule for a problem in ranking means, 1956, Chapel Hill, North Carolina: University of North Carolina.Google Scholar
Gupta, S. S., Huyett, M. & Sobel, M. Selecting and ranking problem with binomial populations. Transactions of the American Society for Quality Control, 1957, 635–644.Google Scholar
Gupta, S. S. & Sobel, M. Selecting a subset containing the best of several binomial populations. In Olkin, I., Ghurye, S. G., Hoeffding, W., Madow, W. G. & Mann, H. B. (Eds.), Contributions to probability and statistics, Essays in honor of Harold Hotelling. California: Stanford University Press. 1960, 224–248.Google Scholar
Olkin, I. Inference for a normal population when the parameters exhibit some structure. In Proschan, F. & Serfling, R. J. (Eds.), Reliability and biometry: Statistical analysis of lifelength. Philadelphia: Society for Industrial and Applied Mathematics. 1974, 759–773.Google Scholar
Roy, S. N., Greenberg, B. G. & Sarhan, A. E. Evaluation of determinants, characteristic equations and their roots for a class of patterned matrices. Journal of the Royal Statistical Society, 1960, 22, 348–359.CrossRefGoogle Scholar