Hostname: page-component-5f745c7db-j9pcf Total loading time: 0 Render date: 2025-01-06T07:36:40.886Z Has data issue: true hasContentIssue false
  • English
  • Français

A Modified Beta Binomial Model with Applications to Multiple Choice and Taste Tests

Published online by Cambridge University Press:  01 January 2025

Donald G. Morrison  and
George Brockway
Donald G. Morrison*
Affiliation:
Columbia University
George Brockway
Affiliation:
Columbia University
*
Requests for reprints should be sent to Donald G. Morrison, Graduate School of Business, Columbia University, New York, NY 10027.
Get access Rights & Permissions [Opens in a new window]

Abstract

A modified beta binomial model is presented for use in analyzing ramdom guessing multiple choice tests and certain forms of taste tests. Detection probabilities for each item are distributed beta across the population subjects. Properties for the observable distribution of correct responses are derived. Two concepts of true score estimates are presented. One, analogous to Duncan’s empirical Bayes posterior mean score, is appropriate for assessing the subject’s performance on that particular test. The second is more suitable for predicting outcomes on similar tests.

Keywords

true score estimatesrandom guessingdetection probabilitiesBayesian updating
Type
Original Paper
Information
Psychometrika , Volume 44 , Issue 4 , December 1979 , pp. 427 - 442
Copyright
Copyright © 1979 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was made possible by a grant from the Center for Food Policy Research, Graduate School of Business, Columbia University.

References

Algina, J. & Noe, M. J. Accuracy of Subkoviak’s coefficient of agreement. Journal of Educational Measurement, 1978, 15, 101–110.CrossRefGoogle Scholar
Birnbaum, A. Statistical theory for logistic mental test models with a prior distribution on ability. Journal of Mathematical Psychology, 1969, 6, 258–276.CrossRefGoogle Scholar
Calandra, A. Scoring problems and probability considerations. Psychometrika, 1941, 6, 1–9.CrossRefGoogle Scholar
Chernoff, H. The scoring of multiple choice questionnaires. Annals of Mathematical Statistics, 1962, 33, 375–393.CrossRefGoogle Scholar
Duncan, G. T. An empirical Bayes approach to scoring multiple-choice tests in the misinformation model. Journal of the American Statistical Association, 1974, 69, 50–57.CrossRefGoogle Scholar
Ferris, G. E. The k-visit method of consumer testing. Biometrics, 1958, 14, 39–49.CrossRefGoogle Scholar
Frary, R. B. Reliability of multiple-choice test scores is not the proportion of variance which is true variance. Educational and Psychological Measurement, 1969, 29, 359–365.CrossRefGoogle Scholar
Guilford, J. P. The determination of item difficulty when chance success is a factor. Psychometrika, 1936, 1, 159–264.CrossRefGoogle Scholar
Hamilton, C. H. Bias and error in multiple choice tests. Psychometrika, 1950, 15, 151–168.CrossRefGoogle ScholarPubMed
Horsnell, G. Paired comparison product testing when individual preferences are stochastic. An alternative model. Applied Statistics, 1977, 26, 162–172.CrossRefGoogle Scholar
Huynh, H. Two simple classes of mastery scores based on the beta-binomial model. Psychometrika, 1977, 42, 601–608.CrossRefGoogle Scholar
Huynh, H. Statistical considerations of mastery scores. Psychometrika, 1976, 41, 65–78.CrossRefGoogle Scholar
Johnson, N. L. & Kotz, S. Discrete distributions, 1969, New York: Wiley.Google Scholar
Johnson, N. L. & Kotz, S. Continuous distributions (Vols. I & II), 1970, New York: Wiley.Google Scholar
Keats, J. A. & Lord, F. M. A theoretical distribution for mental test scores. Psychometrika, 1962, 27, 59–72.CrossRefGoogle Scholar
Krutchkoff, R. G. & Rutherford, J. R. The empirical Bayes approach: Estimating the prior distribution. Biometrika, 1967, 54, 326–328.Google Scholar
Lemon, G. H. & Krutchkoff, R. G. An empirical Bayes smoothing technique. Biometrika, 1969, 56, 361–365.CrossRefGoogle Scholar
Lord, F. M. An approach to mental test theory. Psychometrika, 1959, 24, 283–302.CrossRefGoogle Scholar
Lord, F. M. Use of true-score theory to predict moments of univariate and bivariate observed score distributions. Psychometrika, 1960, 25, 325–342.CrossRefGoogle Scholar
Lord, F. M. A strong true-score theory with applications. Psychometrika, 1965, 30, 239–270.CrossRefGoogle Scholar
Lord, F. M. Estimating true-score distributions in psychological testing: An empirical Bayes estimation problem. Psychometrika, 1969, 34, 259–299.CrossRefGoogle Scholar
Lord, F. M. Formula scoring and number-right scoring. Journal of Educational Measurement, 1975, 12, 7–11.CrossRefGoogle Scholar
Lord, F. M. & Novick, M. R. Statistical theories of mental test scores (with contributions by Allen Birnbaum), 1968, Reading, Mass.: Addison Wesley.Google Scholar
Lord, F. M. & Stocking, M. L. An interval estimate for making statistical inferences about true scores. Psychometrika, 1976, 41, 79–87.CrossRefGoogle Scholar
Lyerly, S. B. A note on correcting for chance success in objective tests. Psychometrika, 1951, 16, 21–30.CrossRefGoogle Scholar
Molenaar, W. On bayesian formula scores for random guessing in multiple choice tests. British Journal of Mathematical and Statistical Psychology, 1977, 30, 79–89.CrossRefGoogle Scholar
Morrison, D. G. A probability model for forced binary choices. The American Statistician, 1978, 32, 23–25.CrossRefGoogle Scholar
Robbins, H. An empirical Bayes approach to statistics. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1955, 1, 157–164.Google Scholar
Robbins, H. Prediction and estimation for the compound Poisson distribution. Proceedings of the National Academy of Sciences, 1977, 74, 2670–2671.CrossRefGoogle ScholarPubMed
Rolph, J. E. Bayesian estimation of mixing distributions. Annals of Mathematical Statistics, 1961, 32, 448–465.Google Scholar
Skellam, J. G. A probability derived from the binomial distribution by regarding the probability of success as a variable between sets of trials. Journal of the Royal Statistical Society, Series B, 1948, 10, 257–65.CrossRefGoogle Scholar
Wierenga, B. Paired comparison product testing when individual preferences are stochastic. Applied Statistics, 1974, 23, 284–296.CrossRefGoogle Scholar
Wilcox, R. R. Estimating true score in the compound binomial error model. Psychometrika, 1978, 43, 245–258.CrossRefGoogle Scholar