Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-02-06T20:31:29.277Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal Scores: An Alternative to Parametric Item Response Theory and Sum Scores

Published online by Cambridge University Press:  01 January 2025

Marie Wiberg Open the ORCID record for Marie Wiberg [Opens in a new window] ,
James O. Ramsay  and
Juan Li
Marie Wiberg*
Affiliation:
Umeå University
James O. Ramsay
Affiliation:
McGill University
Juan Li
Affiliation:
McGill University
*
Correspondence should be made to Marie Wiberg, USBE, Department of Statistics, Umeå University, 90187 Umeå, Sweden. Email: marie.wiberg@umu.se
Get access Rights & Permissions [Opens in a new window]

Abstract

The aim of this paper is to discuss nonparametric item response theory scores in terms of optimal scores as an alternative to parametric item response theory scores and sum scores. Optimal scores take advantage of the interaction between performance and item impact that is evident in most testing data. The theoretical arguments in favor of optimal scoring are supplemented with the results from simulation experiments, and the analysis of test data suggests that sum-scored tests would need to be longer than an optimally scored test in order to attain the same level of accuracy. Because optimal scoring is built on a nonparametric procedure, it also offers a flexible alternative for estimating item characteristic curves that can fit items that do not show good fit to item response theory models.

Type
Original Paper
Information
Psychometrika , Volume 84 , Issue 1 , 15 March 2019 , pp. 310 - 322
Copyright
Copyright © 2018 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11336-018-9639-4) contains supplementary material, which is available to authorized users.

References

Birnbaum, A., Lord, F. M. & Novick, M. R. (1968). Some latent trait models and their use in inferring an examinee’s ability. Statistical theories of mental tests, Reading, MA: Addison-Wesley. 395– 479. Google Scholar
Lee, Y. -S. (2007). A comparison of methods for nonparametric estimation of item characteristic curves for binary items. Applied Psychological Measurement, 37, 121– 134. CrossRefGoogle Scholar
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum. Google Scholar
Mokken, R. J., van der Linden, W. J. & Hambleton, R. K. (1997). Nonparametric models for dichotomous responses. Handbook of modern item response theory, 351– 367. New York: Springer. CrossRefGoogle Scholar
Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611– 630. CrossRefGoogle Scholar
Ramsay, J. O., van der Linden, W. J. & Hambleton, R. K. (1997). A functional approach to modeling test data. Handbook of modern item response theory, 351– 367. New York: Springer. Google Scholar
Ramsay, J. O. (2000). TestGraf: A program for the graphical analysis of multiple choice test and questionnaire data [Computer software and manual]. Montreal: Department of Psychology, McGill University. Google Scholar
Ramsay, J. O., Hooker, G., & Graves, S. (2009). Functional data analysis in R and Matlab. New York: Springer. CrossRefGoogle Scholar
Ramsay, J. O., & Silverman, B. W., Ramsay, J. O. & Silverman, B. W. (2002). Functional models for test items. In Applied functional data analysis. New York: Springer-Verlag.Google Scholar
Ramsay, J. O., & Silverman, B. W. (2005). Functional data analysis. New York: Springer. CrossRefGoogle Scholar
Ramsay, J. O., & Wiberg, M. (2017a). A strategy for replacing sum scores. Journal of Educational and Behavioral Statistics, 42 (3), 282– 307. CrossRefGoogle Scholar
Ramsay, J. O., & Wiberg, M., van der Ark, L. A., Wiberg, M., Culpepper, S. A., Douglas, J. A. & Wang, W. -C. (2017b). Breaking through the sum scoring barrier. Quantitative psychology—81st annual meeting of the psychometric society, Asheville, North Carolina, 2016, 151– 158. New York: Springer. Google Scholar
Rossi, N., Wang, X., & Ramsay, J. O. (2002). Nonparametric item response function estimates with the EM algorithm. Journal of the Behavioral and Educational Sciences, 27, 291– 317. Google Scholar
Wiberg, M., & Bränberg, K. (2015). Kernel equating under the non-equivalent groups with covariates design. Applied Psychological Measurement, 39, 5 349– 361. CrossRefGoogle ScholarPubMed
Woods, C. M. (2006). Ramsay-curve item response theory (RC-IRT) to detect and correct for nonnormal latent variables. Psychological Methods, 11, 253– 270. CrossRefGoogle ScholarPubMed
Woods, C. M., & Thissen, D. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71, 281– 301. CrossRefGoogle ScholarPubMed
Supplementary material: File

Wiberg et al. supplementary material

Wiberg et al. supplementary material
Download Wiberg et al. supplementary material(File)
File 43.4 KB