Hostname: page-component-745bb68f8f-g4j75 Total loading time: 0 Render date: 2025-01-07T19:32:27.607Z Has data issue: false hasContentIssue false
  • English
  • Français

The Theoretical Detect Index of Dimensionality and its Application to Approximate Simple Structure

Published online by Cambridge University Press:  01 January 2025

Jinming Zhang  and
William Stout
Jinming Zhang*
Affiliation:
Educational Testing Service
William Stout
Affiliation:
Department of Statistics, University of Illinois at Urbana-Champaign
*
Requests for reprints should be sent to Jinming Zhang, Educational Testing Service, MS 02-T, Rosedale Road, Princeton NJ 08541. E-mail: jmzhang@ets.org
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, a theoretical index of dimensionality, called the theoretical DETECT index, is proposed to provide a theoretical foundation for the DETECT procedure. The purpose of DETECT is to assess certain aspects of the latent dimensional structure of a test, important to practitioner and research alike. Under reasonable modeling restrictions referred to as “approximate simple structure”, the theoretical DETECT index is proven to be maximized at the correct dimensionality-based partition of a test, where the number of item clusters in this partition corresponds to the number of substantively separate dimensions present in the test and by “correct” is meant that each cluster in this partition contains only items that correspond to the same separate dimension. It is argued that the separation into item clusters achieved by DETECT is appropriate from the applied perspective of desiring a partition into clusters that are interpretable as substantively distinct between clusters and substantively homogeneous within cluster. Moreover, the maximum DETECT index value is a measure of the amount of multidimensionality present. The estimation of the theoretical DETECT index is discussed and a genetic algorithm is developed to effectively execute DETECT. The study of DETECT is facilitated by the recasting of two factor analytic concepts in a multidimensional item response theory setting: a dimensionally homogeneous item cluster and an approximate simple structure test.

Keywords

item response theorydimensionalitymultidimensionalitygeneralized compensatory modelsimple structureapproximate simple structuregenetic algorithmdimensionally homogeneous item clusterlatent spacetest space
Type
Original Paper
Information
Psychometrika , Volume 64 , Issue 2 , June 1999 , pp. 213 - 249
Copyright
Copyright © 1999 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Portions of this article were presented at the annual meeting of the National Council on Measurement in Education, New York, April 1996.

The authors would like to thank Ting Lu, Ming-mei Wang, and four anonymous reviewers for their comments and suggestions on an earlier version of this manuscript. The research of the first author was partially supported by an ETS/GREB Psychometric Fellowship, and by Educational Testing Service Research Allocation Projects 794-33 and 883-05. The research of the second author was partially supported by NSF grants DMS 94-04327 and DMS 97-04474.

References

Ackerman, T. A. (1996). Graphical representation of multidimensional item response theory analyses. Applied Psychological Measurement, 20, 311–329.CrossRefGoogle Scholar
Harman, H. H. (1976). Modern factor analysis 3rd ed.,, Chicago: University of Chicago Press.Google Scholar
Johnson, E. G., & Carlson, J. E. (1994). The NAEP 1992 technical report, Washington, DC: National Center for Education Statistics.Google Scholar
Johnson, R.A., & Wichern, D.W. (1992). Applied Multivariate Statistical Analysis, New Jersey: Prentice Hall.Google Scholar
Junker, B., & Stout, W. F. (1994). Robustness of ability estimation when multiple traits are present with one trait dominant. In Laveault, D., Zumbo, B., Gessaroli, M., & Boss, M. (Eds.), Modern Theories of Measurement: Problems and Issues (pp. 31–61). Ottawa, Canada: University of Ottawa Press.Google Scholar
Kim, H.R. (1994). New techniques for the dimensionality assessment of standardized test data. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Department of Statistics.Google Scholar
Lord, F. M. (1980). Applications of item response theory to practical testing problems, Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Lord, F. M., & Novick, M. (1968). Statistical theories of mental test scores, Reading, MA: Addison Weley.Google Scholar
McDonald, R. P. (1994). Testing for approximate dimensionality. In Laveault, D., Zumbo, B., Gessaroli, M., & Boss, M. (Eds.), Modern theories of measurement: Problems and issues (pp. 63–85). Ottawa, Canada: University of Ottawa Press.Google Scholar
Michalewicz, Z. (1994). Genetic algorithms + Data structures = Genetic programs, Berlin: Springer-Verlag.Google Scholar
Miller, T. R., & Hirsch, T. M. (1992). Cluster analysis of angular data in applications of multidimensional item response theory. Applied Measurement in Education, 5, 193–211.CrossRefGoogle Scholar
Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401–412.CrossRefGoogle Scholar
Reckase, M. D., & McKinley, R. L. (1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 15, 361–373.CrossRefGoogle Scholar
Roussos, L.A., Stout, W., & Marden, J. (1998). Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35, 1–30.CrossRefGoogle Scholar
Sijtsma, K. (1998). Methodology review: Nonparametric IRT approaches to the analysis of dichotomous item scores. Applied Psychological Measurement, 22, 3–31.CrossRefGoogle Scholar
Stout, W. F. (1987). A nonparametric approach for assessing latent trait dimensionality. Psychometrika, 52, 589–617.CrossRefGoogle Scholar
Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145.CrossRefGoogle Scholar
Zhang, J. (1996). Some fundamental issues in item response theory with applications. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Department of Statistics.Google Scholar
Zhang, J. (1997). A new approach for assessing the dimensionality of NAEP data. Paper presented at the annual meeting of American Educational Research Association, Chicago, Illinois, March 1997.Google Scholar
Zhang, J., & Stout, W. F. (1996). A theoretical index of dimensionality and its estimation. Paper presented at the annual meeting of the National Council on Measurement in Education, New York, April 1996.Google Scholar
Zhang, J., Stout, W. F. (1999). Conditional covariance structure of generalized compensatory multidimensional items. Psychometrika, 64, 129–152.CrossRefGoogle Scholar