Hostname: page-component-745bb68f8f-b6zl4 Total loading time: 0 Render date: 2025-01-07T18:09:21.121Z Has data issue: false hasContentIssue false
  • English
  • Français

Analysis of Distractor Difficulty in Multiple-Choice Items

Published online by Cambridge University Press:  01 January 2025

Javier Revuelta
Javier Revuelta*
Affiliation:
Autónoma University de Madrid
*
Requests for reprints should be sent to Javier Revuelta, Departamento de Psicologia Social y Metodologia, Universidad Autdnoma de Madrid, Cantoblanco 28049, Madrid, SPAIN.javier.revuelta@uam.es
Get access Rights & Permissions [Opens in a new window]

Abstract

Two psychometric models are presented for evaluating the difficulty of the distractors in multiple-choice items. They are based on the criterion of rising distractor selection ratios, which facilitates interpretation of the subject and item parameters. Statistical inferential tools are developed in a Bayesian framework: modal a posteriori estimation by application of an EM algorithm and model evaluation by monitoring posterior predictive replications of the data matrix. An educational example with real data is included to exemplify the application of the models and compare them with the nominal categories model.

Keywords

multiple-choice itemsitem-response modelsEM algorithmMCMC simulationPosterior Predictive Checks
Type
Theory And Methods
Information
Psychometrika , Volume 69 , Issue 2 , June 2004 , pp. 217 - 234
Copyright
Copyright © 2004 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was supported by the DGI grant BSO2002-01485.

I would like to thank Eric Maris and Vicente Ponsoda for their advice, Juan Botella for providing the data for the empirical application, and three anonymous reviewers for their comments that were essential for improving the quality of the paper.

References

Abramowitz, M., & Stegun, I.A. (1965). Handbook of Mathematical Functions. New York: Dover Publications Inc.Google Scholar
Bock, R.D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29–51CrossRefGoogle Scholar
Bock, R.D. (1997). The nominal categories model. In van der Linden, W.J. & Hambleton, R.K. (Eds.), Handbook of Modern Item Response Theory. New York: SpringerGoogle Scholar
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459CrossRefGoogle Scholar
Bock, R.D., & Jones, L.V. (1968). The Measurement and Prediction of Judgement and Choice. San Francisco: Holden-DayGoogle Scholar
Brooks, S., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulation. Journal of Computational and Graphical Statistics, 7, 434–455CrossRefGoogle Scholar
Chib, S., & Greenberg, E. (1995). Understanding the Metropolis-Hastings algorithm. The American Statistician, 49, 327–335CrossRefGoogle Scholar
Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, series B, 39, 1–38CrossRefGoogle Scholar
Fahrmeir, L., & Tutz, G. (1994). Multivariate Statistical Modelling Based on Generalized Linear Models. New York: Springer-VerlagCrossRefGoogle Scholar
Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (1995). Bayesian Data Analysis. London: Chapman and HallCrossRefGoogle Scholar
Gelman, A., Meng, X.L., & Stern, H.S. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6, 733–807Google Scholar
Gill, P.E., Murray, W., & Wright, M.H. (1981). Practical Optimization. London: Academic PressGoogle Scholar
Goodman, L.A., & Kruskal, W.H. (1979). Measures of Associations for Cross Classifications. New York: Springer-VerlagCrossRefGoogle Scholar
Hoijtink, H., & Molenaar, I. W. (1997). A multidimensional item response model: Constrained latent class analysis using the Gibbs sampler and Posterior Predictive Checks. Psychometrika, 62, 171–189CrossRefGoogle Scholar
Louis, T.A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society, series B, 44, 226–233CrossRefGoogle Scholar
Love, T.E. (1997). Distractor selection ratios. Psychometrika, 62, 51–62CrossRefGoogle Scholar
Luce, R.D. (1959). Individual Choice Behavior: A Theoretical Analysis. New York: WileyGoogle Scholar
McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In Zarembka, P. (Eds.), Frontiers in Econometrics. New York: Academic PressGoogle Scholar
Mellenberg, G.J. (1995). Conceptual notes on models for polytomous item responses. Applied Psychological Measurement, 19, 91–100CrossRefGoogle Scholar
Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement, 14, 3–19CrossRefGoogle Scholar
Press, W.H., Teukolsky, S.A., Veterling, W.T., & Flannery, B.P. (1992). Numerical Recipes in C 2nd ed., New York: Cambridge University PressGoogle Scholar
Ramsay, J.O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611–630CrossRefGoogle Scholar
Ramsay, J.O. (2000). TestGraf. A Program for the Graphical Analysis of Multiple Choice Test and Questionnaire Data. McGill University. Available on-line at http://www.psych.mcgill.ca/faculty/ramsay.htmlGoogle Scholar
Tierney, L. (1994). Markov chains for exploring posterior distributions. The Annals of Statistics, 22, 1701–1762Google Scholar
van Onna, M.J.H. (2002). Bayesian estimation and model selection in ordered latent class models for polytomous items. Psychometrika, 67, 519–538CrossRefGoogle Scholar