Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-08T16:09:47.456Z Has data issue: false hasContentIssue false
  • English
  • Français

To Weight or Not to Weight? Balancing Influence of Initial Items in Adaptive Testing

Published online by Cambridge University Press:  01 January 2025

Hua-Hua Chang  and
Zhiliang Ying
Hua-Hua Chang*
Affiliation:
University of Illinois
Zhiliang Ying
Affiliation:
Columbia University
*
Requests for reprints should be sent to Hua-Hua Chang, University of Illinois, Urbana, USA. E-mail: hhchang@uiuc.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

It has been widely reported that in computerized adaptive testing some examinees may get much lower scores than they would normally if an alternative paper-and-pencil version were given. The main purpose of this investigation is to quantitatively reveal the cause for the underestimation phenomenon. The logistic models, including the 1PL, 2PL, and 3PL models, are used to demonstrate our assertions. Our analytical derivation shows that, under the maximum information item selection strategy, if an examinee failed a few items at the beginning of the test, easy but more discriminating items are likely to be administered. Such items are ineffective to move the estimate close to the true θ, unless the test is sufficiently long or a variable-length test is used. Our results also indicate that a certain weighting mechanism is necessary to make the algorithm rely less on the items administered at the beginning of the test.

Keywords

computerized adaptive testingMLEFisher informationa-stratified methoditem selection algorithm
Type
Theory and Methods
Information
Psychometrika , Volume 73 , Issue 3 , September 2008 , pp. 441 - 450
Copyright
Copyright © 2007 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was partially supported by the NSF Grants SES0241020 and SES0613025. The authors thank the Editor, Associate Editor and two anonymous reviewers for their comments and suggestions. Send further information to Hua-Hua Chang, Department of Psychology, 603 E. Daniel Street, M/C 716, Champaign, IL 61820.

References

Carlson, S. (2000). ETS finds flaws in the way online GRE rates some students. Chronicle of Higher Education, 47(8), A47.Google Scholar
Chang, H.H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213–229.CrossRefGoogle Scholar
Chang, H.H., & Ying, Z. (1999). A-stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23(3), 211–222.CrossRefGoogle Scholar
Hau, K.-T., & Chang, H. (2001). Item selection in computerized adaptive testing: should more discriminating items be used first?. Journal of Educational Measurement, 28, 249–266.CrossRefGoogle Scholar
Kaplan, . (2004). GRE Exam (2005 ed., p. 198). New York: Kaplan Publishing.Google Scholar
Lord, M.F. (1980). Applications of item response theory to practical testing problems, Hillsdale: Laurence Erlbaum Associates, Publishers.Google Scholar
Merritt, J. (2003). Why the folks at ETS flunked the course—a tech-savvy service will soon be giving B-school applicants their GMATs. Business Week, December 29, 2003.Google Scholar