Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-08T16:06:22.023Z Has data issue: false hasContentIssue false
  • English
  • Français

Modern Sequential Analysis and Its Applications to Computerized Adaptive Testing

Published online by Cambridge University Press:  01 January 2025

Jay Bartroff ,
Matthew Finkelman  and
Tze Leung Lai
Jay Bartroff*
Affiliation:
University of Southern California
Matthew Finkelman
Affiliation:
Harvard University
Tze Leung Lai
Affiliation:
Stanford University
*
Requests for reprints should be sent to Jay Bartroff, Department of Mathematics, University of Southern California, 3620 S Vermont Ave., KAP 108, Los Angeles, CA 90089, USA. E-mail: bartroff@usc.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

After a brief review of recent advances in sequential analysis involving sequential generalized likelihood ratio tests, we discuss their use in psychometric testing and extend the asymptotic optimality theory of these sequential tests to the case of sequentially generated experiments, of particular interest in computerized adaptive testing. We then show how these methods can be used to design adaptive mastery tests, which are asymptotically optimal and are also shown to provide substantial improvements over currently used sequential and fixed length tests.

Keywords

sequential analysiscomputerized adaptive testingmastery testinggeneralized likelihood ratio statisticsitem response theory
Type
Theory and Methods
Information
Psychometrika , Volume 73 , Issue 3 , September 2008 , pp. 473 - 486
Copyright
Copyright © 2008 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bickel, P., Buyske, S., Chang, H., & Ying, Z. (2001). On maximizing item information and matching difficulty with ability. Psychometrika, 66, 69–77.CrossRefGoogle Scholar
Chang, Y. (2004). Application of sequential probability ratio test to computerized criterion-referenced testing. Sequential Analysis, 23, 45–61.CrossRefGoogle Scholar
Chang, Y. (2005). Application of sequential interval estimation to adaptive mastery testing. Psychometrika, 70, 685–713.CrossRefGoogle Scholar
Chang, Y., & Ying, Z. (2003). Sequential estimation in variable length computerized adaptive testing. Journal of Statistical Planning and Inference, 121, 249–264.CrossRefGoogle Scholar
Durrett, R. (2005). Probability: theory and examples, (3rd ed.). Belmont: Thomson.Google Scholar
Eggen, T. (1999). Item selection in adaptive testing with the sequential probability ratio test. Applied Psychological Measurement, 23, 249–261.CrossRefGoogle Scholar
Fedorov, V.V. (1997). Model-oriented design of experiments, New York: Springer.CrossRefGoogle Scholar
Hoeffding, W. (1960). Lower bounds for the expected sample size and the average risk of a sequential procedure. Annals of Mathematical Statistics, 31, 352–368.CrossRefGoogle Scholar
Jennison, C., & Turnbull, B.W. (2000). Group sequential methods with applications to clinical trials, New York: Chapman & Hall/CRC.Google Scholar
Kingsbury, G., & Zara, A. (1989). Procedures for selecting items for computerized adaptive testing. Applied Measurement in Education, 2, 359–375.CrossRefGoogle Scholar
Lai, T.L. (1981). Asymptotic optimality of invariant sequential probability ratio tests. Annals of Statistics, 9, 318–333.CrossRefGoogle Scholar
Lai, T.L. (1997). On optimal stopping problems in sequential hypothesis testing. Statistica Sinica, 7, 33–52.Google Scholar
Lai, T.L. (2001). Sequential analysis: some classical problems and new challenges (with discussion). Statistica Sinica, 11, 303–408.Google Scholar
Lai, T.L., & Shih, M.C. (2004). Power, sample size and adaptation considerations in the design of group sequential clinical trials. Biometrika, 91, 507–528.CrossRefGoogle Scholar
Lai, T.L., & Zhang, L. (1994). A modification of Schwarz’s sequential likelihood ratio tests in multivariate sequential analysis. Sequential Analysis, 13, 79–96.CrossRefGoogle Scholar
Lewis, C., & Sheehan, K. (1990). Using Bayesian decision theory to design a computerized mastery test. Applied Psychological Measurement, 14, 367–386.CrossRefGoogle Scholar
Lin, C., & Spray, J. (2000). Effects of item-selection criteria on classification testing with the sequential probability ratio test (Research Report 2000-8). Iowa City, IA: American College Testing.Google Scholar
Lord, F. (1980). Applications of item response theory to practical testing problems, Hillsdale: Erlbaum.Google Scholar
Reckase, M. (1983). A procedure for decision making using tailored testing. In Weiss, D. (Eds.), New horizons in testing—latent trait test theory and computerized adaptive testing (pp. 238–257). New York: Academic Press.Google Scholar
Siegmund, D. (1985). Sequential analysis: tests and confidence intervals, New York: Springer.CrossRefGoogle Scholar
Spray, J., & Reckase, M. (1996). Comparison of SPRT and sequential Bayes procedures for classifying examinees into two categories using a computerized test. Journal of Educational and Behavioral Statistics, 21, 405–414.CrossRefGoogle Scholar
Stocking, M.L., & Swanson, L. (1993). A method for severely constrained item selection in adaptive testing. Applied Psychological Measurement, 17, 277–292.CrossRefGoogle Scholar
Sympson, J.B., & Hetter, R.D. (1985). Controlling item-exposure rates in computerized adaptive testing. In Proceedings of the 27th annual meeting of the military testing association (pp. 973–977). San Diego, CA. Navy Personnel Research and Development Center.Google Scholar
van der Linden, W.J., & Pashley, P. (2000). Item selection and ability estimation in adaptive testing. In van der Linden, W.J., & Glas, C.A.W. (Eds.), Computerized adaptive testing: theory and practice (pp. 1–25). Norwell: Kluwer.CrossRefGoogle Scholar
Vos, H. (2000). A Bayesian procedure in the context of sequential mastery testing. Psicológica, 21, 191–211.Google Scholar
Wald, A. (1947). Sequential analysis, New York: Wiley. Reprinted by Dover (1973)Google Scholar