Hostname: page-component-745bb68f8f-f46jp Total loading time: 0 Render date: 2025-01-07T18:59:30.511Z Has data issue: false hasContentIssue false
  • English
  • Français

A New Procedure for Detection of Crossing DIF

Published online by Cambridge University Press:  01 January 2025

Hsin-Hung Li  and
William Stout
Hsin-Hung Li
Affiliation:
Department of Statistics, University of Illinois at Urbana-Champaign
William Stout*
Affiliation:
Department of Statistics, University of Illinois at Urbana-Champaign
*
Requests for reprints should be sent to William Stout, Department of Statistics, University of Illinois at Urbana-Champaign, 725 South Wright Street, Champaign, IL 61820.
Get access Rights & Permissions [Opens in a new window]

Abstract

The purpose of this paper is to present a hypothesis testing and estimation procedure, Crossing SIBTEST, for detecting crossing DIF. Crossing DIF exists when the difference in the probabilities of a correct answer for the two examinee groups changes signs as ability level is varied. In item response theory terms, crossing DIF is indicated by two crossing item characteristic curves. Our new procedure, denoted as Crossing SIBTEST, first estimates the matching subtest score at which crossing occurs using least squares regression analysis. A Crossing SIBTEST statistic then is used to test the hypothesis of crossing DIF. The performance of Crossing SIBTEST is evaluated in this study.

Keywords

DIFbiasunidirectional DIFcrossing DIFmultidimensional IRTrandomization testsSIBTESTMantel-Haenszellogistic regression procedure crossingSIBTEST
Type
Original Paper
Information
Psychometrika , Volume 61 , Issue 4 , December 1996 , pp. 647 - 677
Copyright
Copyright © 1996 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was partially supported by a grant from the Law School Admission Council and by National Science Foundation Mathematics Grant NSF-DMS-94-04327. The research reported here is collaborative in every respect and the order of authorship is alphabetical. The authors thank Jeff Douglas and Louis Roussos for their useful comments and discussions.

References

Bennett, R. E., Rock, D. A., Kaplan, B. A. (1987). SAT differential item performance for nine handicapped groups. Journal of Educational Measurement, 24(1), 56–64.CrossRefGoogle Scholar
Bock, R. D. (1975). Multivariate statistical methods, New York: McGraw-Hill.Google Scholar
Dorans, N. J., Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the scholastic aptitude test. Journal of Educational Measurement, 23, 355–368.CrossRefGoogle Scholar
Douglas, J., Roussos, L., & Stout, W. F. (in press). Item bundle DIF hypothesis testing: identifying suspect bundles and assessing their DIF. Journal of Educational Measurement.Google Scholar
Drasgow, F. (1987). A study of measurement bias of two standard psychological tests. Journal of Applied Psychology, 72, 19–30.CrossRefGoogle Scholar
Edgington, E. S. (1987). Randomization tests, New York, NY: Maecel Dekker.Google Scholar
Ellis, B. (1989). Differential item functioning: Implications for test translations. Journal of Applied Psychology, 74(6), 912–921.CrossRefGoogle Scholar
Hambleton, R. K., Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods. Applied Measurement in Education, 2(4), 313–334.CrossRefGoogle Scholar
Holland, P. W., Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In Wainer, H., Braun, H. I. (Eds.), Test validity (pp. 129–145). Hillsadle, NJ: Lawrence Erlbaum.Google Scholar
Li, H., & Stout, W. (1993, June). A new procedure for detection of crossing bias/DIF. Paper presented at the 58th Annual Meeting of Psychometric Society, Berkeley, CA.Google Scholar
Li, H., & Stout, W. (1994, April). Detecting crossing item bias/DIF: Comparison of logistic regression and crossing SIBTEST procedures. Paper presented at the 1994 AERA Annual Meeting, New Orleans, LA.Google Scholar
Mellenbergh, G. J. (1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7(2), 105–118.CrossRefGoogle Scholar
Mislevy, R. J., Bock, R. D. (1984). Item operating characteristics of the Armed Services Aptitute Battery (ASVAB), Form 8A, Washington, DC: Office of Naval Research.Google Scholar
Nandakumar, R. (1992). Simultaneous DIF amplification and cancellation: Shealy-Stout's test for DIF. Journal of Educational Measurement, 30, 293–311.CrossRefGoogle Scholar
Oshima, T., & Miller, D. (1991, April). Multidimensionality and item bias in item response theory. Presented at 1991 AERA Annual Meeting, Chicago, IL.Google Scholar
Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611–630.CrossRefGoogle Scholar
Ramsay, J. O. (1993). TESTGRAF: A program for the graphical analysis of multiple choice test and questionnaire data. TESTGRAF user's guide.Google Scholar
Roussos, L., & Stout, W. F. (1991). BIASDEM: A program for simulating models of unidirectional and crossing DIF. Unpublished manuscript.Google Scholar
Roussos, L., & Stout, W. F. (in press). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement.Google Scholar
Shealy, R., Stout, W. (1993). An item response theory model for test bias. In Holland, P. W., Wainer, H. (Eds.), Differential item functioning (pp. 197–239). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Shealy, R., Stout, W. (1993). A model-based standardization approach that separate true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF. Psychometrika, 58, 159–194.CrossRefGoogle Scholar
Swaminathan, H., Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361–370.CrossRefGoogle Scholar