Hostname: page-component-745bb68f8f-l4dxg Total loading time: 0 Render date: 2025-01-07T19:00:14.694Z Has data issue: false hasContentIssue false
  • English
  • Français

Modeling Item Responses When Different Subjects Employ Different Solution Strategies

Published online by Cambridge University Press:  01 January 2025

Robert J. Mislevy  and
Norman Verhelst
Robert J. Mislevy*
Affiliation:
Educational Testing Service
Norman Verhelst
Affiliation:
CITO (National Institute for Educational Measurement), Arnhem, The Netherlands
*
Requests for reprints should be sent to Robert J. Mislevy, Educational Testing Service, Princeton, NJ 08541.
Get access Rights & Permissions [Opens in a new window]

Abstract

A model is presented for item responses when different subjects employ different strategies, but only responses, not choice of strategy, can be observed. Using substantive theory to differentiate the likelihoods of response vectors under a fixed set of strategies, we model response probabilities in terms of item parameters for each strategy, proportions of subjects employing each strategy, and distributions of subject proficiency within strategies. The probabilities that an individual subject employed the various strategies can then be obtained, along with a conditional estimate of proficiency under each. A conceptual example discusses response strategies for spatial rotation tasks, and a numerican example resolves a population of subjects into subpopulations of valid responders and random guessers.

Keywords

differential strategiesEM algorithmitem response theorylinear logistic test modelmixture models
Type
Original Paper
Information
Psychometrika , Volume 55 , Issue 2 , June 1990 , pp. 195 - 215
Copyright
Copyright © 1990 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The first author's work was supported by Contract No. N00014-85-K-0683, project designation NR 150-539, from the Cognitive Science Program, Cognitive and Neural Sciences Division, Office of Naval Research. We are grateful to Murray Aitkin, Isaac Bejar, Neil Dorans, Frederiksen, and Marklyn Wingersky for their comments and suggestions, and to Alison Gooding, Maxine Kingston, Donna Lembeck, Joling Liang, and Kentaro Yamamoto for their assistance with Example 2.

References

Birren, J. E., Cunningham, W. R., & Yamamoto, K. (1983). Psychology of adult development and aging. Annual Review of Psychology, 34, 543–575.CrossRefGoogle Scholar
Carter, P., Pazak, B., & Kail, R. (1983). Algorithms for processing spatial information. Journal of Experimental Child Psychology, 36, 284–304.CrossRefGoogle ScholarPubMed
Cooper, L. A., & Shepard, R. N. (1973). Chronometric studies of the rotation of mental images. In Chase, W. G. (Eds.), Visual information processing (pp. 76–176). Orlando, FL: Academic Press.Google Scholar
de Leeuw, J., & Verhelst, N. (1986). Maximum likelihood estimation in generalized Rasch models. Journal of Educational Statistics, 11, 183–196.CrossRefGoogle Scholar
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39, 1–38.CrossRefGoogle Scholar
Embretson, S. E. (1985). Multicomponenet latent trait models for test design. In Embretson, S. E. (Eds.), Testdesign: Developments in psychology and psychometrics (pp. 195–218). Orlando, EL: Academic Press.Google Scholar
Fischer, G. H. (1973). The linear logistic test model as instrument in educational research. Acta Psychologica, 36, 359–374.CrossRefGoogle Scholar
French, J. W. (1965). The relationship of problem-solving styles to the factor composition of tests. Educational and Psychological Measurement, 25, 9–28.CrossRefGoogle Scholar
Gagné, R. M. (1962). The acquisition of knowledge. Psychological Review, 69, 355–365.CrossRefGoogle Scholar
Gentner, D., & Gentner, D. R. (1983). Flowing waters or teeming crowds: Mental models of electricity. In Gentner, D., Stevens, A. L. (Eds.), Mental models (pp. 99–129). Hillsdale, NJ: Earlbaum.Google Scholar
Hulin, C. L., Drasgow, F., & Parsons, C. K. (1983). Item response theory: Application to psychological measurement, Homewood, IL: Dow Jones-Irwin.Google Scholar
Just, M. A., & Carpenter, P. A. (1985). Congnitive coordinate systems: Accounts of mental rotation and individual differences in spatial ability. Psychological Review, 92, 137–172.CrossRefGoogle Scholar
Kyllonen, P. C., & Lohman, D. F., Snow, R. E. (1984). Effects of aptitudes, strategy training, and task facets on spatial task performance. Journal of Educational Psychology, 76, 130–145.CrossRefGoogle Scholar
Lohman, D. F. (1979). Spatial ability: A review and reanalysis of the correlational literature, Stanford, CA: Stanford University, Department of Education, Aptitude Research Project.Google Scholar
Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society, 44, 226–233.CrossRefGoogle Scholar
Macrae, K. S. (n.d.). Strategies underlying psychometric test responses in young and middle-aged adults of varying educational background. Unpublished manuscript, La Trobe University, Australia.Google Scholar
Messick, S. (1984). The psychology of educational measurement. Journal of Educational Measurement, 21, 215–237.CrossRefGoogle Scholar
Mislevy, R. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51, 177–195.CrossRefGoogle Scholar
Mislevy, R. J., & Bock, R. D. (1983). BILOG: Item analysis and test scoring with binary logistic models, Mooresville, IN: Scientific Software.Google Scholar
Mislevy, R. J., & Verhelst, N. (1987). Modeling item responses when different subjects employ different solution strategies, Princeton, NJ: Educational Testing Service.Google Scholar
Paulson, J. (1985). Latent class representation of systematic patterns in test responses, Portland, OR: Protland State University.Google Scholar
Pellegrino, J. W., Mumaw, R. J., & Shute, V. J. (1985). Analysis of spatial aptitude and expertise. In Embretson, S. E. (Eds.), Test design: Developments in psychology and psychometrics (pp. 45–76). Orlando, FL: Academic Press.CrossRefGoogle Scholar
Piaget, J. (1960). The general problems of the psychological development of the child. In Tanner, J. M. & Inhelder, B. (Eds.), Discussions on child development: Vol. 4 (pp. 3–27). The fourth meeting of the World Health Organization Study Group on the Psychobiological Development of the Child, Geneva, 1956.Google Scholar
Ramsay, J. O. (1975). Soving implicit equations in psychometric data analysis. Psychometrika, 40, 361–372.CrossRefGoogle Scholar
Samejima, F. (1983). A latent trait model for differential strategies in cognitive processes, Knoxville, TN: University of Tennessee.Google Scholar
Scheiblechner, H. (1972). Das lernen und lösen komplexer denkaufgaben [The studying and solving of complex conceptual problems]. Zeitschrift für Experimentelle und Angewandte Psychologie, 19, 476–506.Google Scholar
Siegler, R. S. (1981). Developmental sequences within and between concepts. Monograph of the Society for Research in Child Development, 46 (2, Serial No. 189).CrossRefGoogle Scholar
Tapley, S. M., & Bryden, M. P. (1977). An investigation of sex differences in spatial ability: Mental rotation of three-dimensional objects. Canadian Journal of Psychology, 31, 122–130.CrossRefGoogle ScholarPubMed
Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.CrossRefGoogle Scholar
Titterington, D. M., Smith, A. F. M., & Makov, U. E. (1985). Statistical analysis of finite mixture distributions, Chichester: Wiley & Sons.Google Scholar
Tsutakawa, R. K., & Soltys, M. J. (1988). Approximation for Bayesian ability estimation. Journla of Educational Statistics, 13, 117–130.CrossRefGoogle Scholar
Wilson, M. R. (1984). A psychometric model of hierarchical development. Unpublished doctoral dissertation, University of Chicago.Google Scholar
Wison, M. R. (1989). Saltus: A psychometric model of discontinuity in cognitive development. Psychological Bullentin, 105, 276–289.CrossRefGoogle Scholar
Yamamoto, K. (1987). A hybrid model for item responses. Unpublished dissertation, University of Illinois.Google Scholar