Hostname: page-component-745bb68f8f-hvd4g Total loading time: 0 Render date: 2025-01-07T18:49:22.983Z Has data issue: false hasContentIssue false
  • English
  • Français

Kullback–Leibler Information and Its Applications in Multi-Dimensional Adaptive Testing

Published online by Cambridge University Press:  01 January 2025

Chun Wang ,
Hua-Hua Chang  and
Keith A. Boughton
Chun Wang*
Affiliation:
University of Illinois at Urbana-Champaign
Hua-Hua Chang
Affiliation:
University of Illinois at Urbana-Champaign
Keith A. Boughton
Affiliation:
CTB/McGraw-Hill
*
Requests for reprints should be sent to Chun Wang, Department of Psychology, University of Illinois at Urbana-Champaign, 603 E. Daniel St., Champaign, IL 61820, USA. E-mail: cwang49@uiuc.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper first discusses the relationship between Kullback–Leibler information (KL) and Fisher information in the context of multi-dimensional item response theory and is further interpreted for the two-dimensional case, from a geometric perspective. This explication should allow for a better understanding of the various item selection methods in multi-dimensional adaptive tests (MAT) which are based on these two information measures. The KL information index (KI) method is then discussed and two theorems are derived to quantify the relationship between KI and item parameters. Due to the fact that most of the existing item selection algorithms for MAT bear severe computational complexity, which substantially lowers the applicability of MAT, two versions of simplified KL index (SKI), built from the analytical results, are proposed to mimic the behavior of KI, while reducing the overall computational intensity.

Keywords

Kullback–Leibler informationFisher informationmulti-dimensional adaptive testing
Type
Original Paper
Information
Psychometrika , Volume 76 , Issue 1 , January 2011 , pp. 13 - 39
Copyright
Copyright © 2010 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adams, R.J., Wilson, M., Wang, W.C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1–23.CrossRefGoogle Scholar
Allen, D.D., Ni, P.S., Haley, S.M. (2008). Efficiency and sensitivity of multidimensional computerized adaptive testing of pediatric physical functioning. Disability and Rehabilitation, 30(6), 479–484.CrossRefGoogle ScholarPubMed
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In Lord, F.M., Novick, M.R. (Eds.), Statistical theories of mental test scores (pp. 379–479). Reading: Addison-Welsey.Google Scholar
Bloxom, B.M., & Vale, C.D. (1987). Multidimensional adaptive testing: a procedure for sequential estimation of the posterior centroid and dispersion of theta. Paper presented at the meeting of the Psychometric Society, Montreal, Canada.Google Scholar
Chang, H.H. (1996). The asymptotic posterior normality of the latent trait for polytomous IRT models. Psychometrika, 61(3), 445–463.CrossRefGoogle Scholar
Chang, H.H., Qian, J.H., Ying, Z.L. (2001). a-stratified multistage computerized adaptive testing with b blocking. Applied Psychological Measurement, 25(4), 333–341.CrossRefGoogle Scholar
Chang, H.H., Stout, W. (1993). The asymptotic posterior normality of the latent trait in an IRT model. Psychometrika, 58(1), 37–52.CrossRefGoogle Scholar
Chang, H.H., Ying, Z.L. (1996). A global information approach to computerized adoptive testing. Applied Psychological Measurement, 20(3), 213–229.CrossRefGoogle Scholar
Chang, H.H., Ying, Z.L. (1999). a-stratified multistage computerized adaptive testing. Applied Psychological Measurement, 23(3), 211–222.CrossRefGoogle Scholar
Chang, H.H., Ying, Z.L. (2008). To weight or not to weight? Balancing influence of initial items in adaptive testing. Psychometrika, 73(3), 441–450.CrossRefGoogle Scholar
Chang, H.H., Zhang, J.M. (2002). Hypergeometric family and item overlap rates in computerized adaptive testing. Psychometrika, 67(3), 387–398.CrossRefGoogle Scholar
Cover, T., Thomas, J. (1991). Elements of information theory, New York: Wiley.Google Scholar
Finkelman, M., Nering, M.L., Roussos, L.A. (2009). A conditional exposure control method for multidimensional adaptive testing. Journal of Educational Measurement, 46(1), 84–103.CrossRefGoogle Scholar
Hambleton, R.K., Swaminathan, H. (1985). Item response theory: principles and applications, Boston: Kluwer/Nijhoff.CrossRefGoogle Scholar
Henson, R., Douglas, J. (2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29(4), 262–277.CrossRefGoogle Scholar
Haley, S.M., Ni, P.S., Ludlow, L.H., Fragala-Pinkham, M.A. (2006). Measurement precision and efficiency of multidimensional computer adaptive testing of physical functioning using the pediatric evaluation of disability inventory. Archives of Physical Medicine and Rehabilitation, 87(9), 1223–1229.CrossRefGoogle ScholarPubMed
Lehmann, E.L. (1986). Testing statistical hypotheses, (2nd ed.). New York: Wiley.CrossRefGoogle Scholar
Lehmann, E.L., Casella, G. (1998). Theory of point estimation, (2nd ed.). Berlin: Springer.Google Scholar
Lehmann, E.L. (1999). Elements of large-sample theory, New York: Springer.CrossRefGoogle Scholar
Li, Y.H., Schafer, W.D. (2005). Trait parameter recovery using multidimensional computerized adaptive testing in reading and mathematics. Applied Psychological Measurement, 29(1), 3–25.CrossRefGoogle Scholar
Lord, F.M. (1980). Applications of item response theory to practical testing problems, Hillsdale: Erlbaum.Google Scholar
Lord, F.M., Novick, M.R. (1968). Statistical theories of mental test scores, Reading: Addison-Welsley.Google Scholar
Luecht, R.M. (1996). Multidimensional computerized adaptive testing in a certification or licensure context. Applied Psychological Measurement, 20(4), 389–404.CrossRefGoogle Scholar
Mulder, J., van der Linden, W.J. (2009). Multidimensional adaptive testing with optimal design criteria for item selection. Psychometrika, 74(2), 273–296.CrossRefGoogle ScholarPubMed
Owen, R.J. (1969). A Bayesian approach to tailored testing (Research Report 69-92). Princeton, NJ: Educational Testing Service.Google Scholar
Owen, R.J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351–356.CrossRefGoogle Scholar
Passos, V.L., Berger, M.P.F., Tan, F.E.S. (2008). The d-optimality item selection criterion in the early stage of CAT: a study with the graded response model. Journal of Educational and Behavioral Statistics, 33(1), 88–110.CrossRefGoogle Scholar
Reckase, M.D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9(4), 401–412.CrossRefGoogle Scholar
Reckase, M.D. (1997). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21(1), 25–36.CrossRefGoogle Scholar
Reckase, M.D. (2009). Multidimensional item response theory, New York: Springer.CrossRefGoogle Scholar
Reckase, M.D., McKinley, R.L. (1991). The discrimination power of items that measure more than one dimension. Applied Psychological Measurement, 15(4), 361–373.CrossRefGoogle Scholar
Renyi, A. (1961). On measurs of entropy and information. Proceeding fourth Berkeley symposium on mathematical statististics and probability, 547–561.Google Scholar
Renyi, A. (1970). Probability theory, Amsterdam: North-Holland.Google Scholar
Segall, D.O. (1996). Multidimensional adaptive testing. Psychometrika, 61(2), 331–354.CrossRefGoogle Scholar
Segall, D.O. (2001). General ability measurement: an application of multidimensional item response theory. Psychometrika, 66(1), 79–97.CrossRefGoogle Scholar
Stocking, M.L. (1994). Three practical issues for modern adaptive testing item pools (Research Report 94-5). Princeton, NJ: Educational Testing Service.Google Scholar
Stroud, A.H., Secrest, D. (1966). Gaussian quadrature formulas, Englewood Cliffs: Prentice-Hall.Google Scholar
Thissen, D., Mislevy, R.J. (2000). Testing algorithms. In Wainer, H. (Eds.), Computerized adaptive testing: a primer (pp. 101–133). Hillsdale: Erlbaum.Google Scholar
van der Linden, W.J. (1996). Assembling tests for the measurement of multiple traits. Applied Psychological Measurement, 20, 373–388.CrossRefGoogle Scholar
van der Linden, W.J. (1998). Bayesian item selection criteria for adaptive testing.CrossRefGoogle Scholar
van der Linden, W.J. (1999). Multidimensional adaptive testing with a minimum error-variance criterion. Journal of Educational and Behavioral Statistics, 24(4), 398–412.CrossRefGoogle Scholar
van Rijn, P.W., Eggen, T., Hemker, B.T., Sanders, P.F. (2002). Evaluation of selection procedures for computerized adaptive testing with polytomous items. Applied Psychological Measurement, 26(4), 393–411.CrossRefGoogle Scholar
Veerkamp, W.J.J., Berger, M.P.F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22(2), 203–226.CrossRefGoogle Scholar
Veldkamp, B.P., van Der Linden, W.J. (2002). Multidimensional adaptive testing with constraints on test content. Psychometrika, 67(4), 575–588.CrossRefGoogle Scholar
Wang, C., Chang, H. (2009). Kullback–Leibler information in multidimensional adaptive testing: theory and application. In Weiss, D.J. (Eds.), Proceedings of the 2009 GMAC conference on computerized adaptive testing, Retrieved from www.psych.umn.edu/psylabs/CATCentral/Google Scholar
Wang, C., & Chang, H. (2010). Item selection in multidimensional computerized adaptive testing—the new application of Kullback–Leibler information. Paper presented at 2010 International Meeting of Psychometric Society, Athens, Georgia.Google Scholar
Yi, Q., Chang, H.H. (2003). alpha-Stratified CAT design with content blocking. British Journal of Mathematical & Statistical Psychology, 56, 359–378.CrossRefGoogle ScholarPubMed