Hostname: page-component-5f745c7db-nc56l Total loading time: 0 Render date: 2025-01-06T07:29:28.588Z Has data issue: true hasContentIssue false
  • English
  • Français

Marginal Maximum Likelihood Estimation for a Psychometric Model of Discontinuous Development

Published online by Cambridge University Press:  01 January 2025

Robert J. Mislevy  and
Mark Wilson
Robert J. Mislevy*
Affiliation:
Educational Testing Service
Mark Wilson
Affiliation:
Graduate School of Education, University of California, Berkeley
*
Requests for reprints should be sent to Robert J. Mislevy, Educational Testing Service, Princeton, NJ 08541.
Get access Rights & Permissions [Opens in a new window]

Abstract

Item response theory models posit latent variables to account for regularities in students' performances on test items. Wilson's “Saltus” model extends the ideas of IRT to development that occurs in stages, where expected changes can be discontinuous, show different patterns for different types of items, or even exhibit reversals in probabilities of success on certain tasks. Examples include Piagetian stages of psychological development and Siegler's rule-based learning. This paper derives marginal maximum likelihood (MML) estimation equations for the structural parameters of the Saltus model and suggests a computing approximation based on the EM algorithm. For individual examinees, empirical Bayes probabilities of learning-stage are given, along with proficiency parameter estimates conditional on stage membership. The MML solution is illustrated with simulated data and an example from the domain of mixed number subtraction.

Keywords

Cognitive diagnosisEM algorithmempirical Bayesitem response theorymarginal maximum likelihoodmixture modelsSaltus model
Type
Original Paper
Information
Psychometrika , Volume 61 , Issue 1 , March 1996 , pp. 41 - 71
Copyright
© 1996 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The authors' names appear in alphabetical order. We would like to thank Karen Draney for computer programming, Kikumi Tatsuoka for allowing us to use the mixed-number subtraction data, and Eric Bradlow, Chan Dayton, Kikumi Tatsuoka, and four anonymous referees for helpful suggestions. The first author's work was supported by Contract No. N00014-88-K-0304, R&T 4421552, from the Cognitive Sciences Program, Cognitive and Neural Sciences Division, Office of Naval Research, and by the Program Research Planning Council of Educational Testing Service. The second author's work was supported by a National Academy of Education Spencer Fellowship and by a Junior Faculty Research Grant from the Committee on Research, University of California at Berkeley. A copy of the Saltus computer program can be obtained from the second author.

References

Bock, R. D., Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.CrossRefGoogle Scholar
Bock, R. D., Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179–197.CrossRefGoogle Scholar
Cohen, L. (1979). Approximate expressions for parameter estimates in the Rasch model. British Journal of Mathematical and Statistical Psychology, 32, 113–120.CrossRefGoogle Scholar
Cronbach, L. J., Furby, L. (1970). How should we measure “change”—Or should we?. Psychological Bulletin, 74, 68–80.CrossRefGoogle Scholar
Dempster, A. P., Laird, N. M., Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1–38.CrossRefGoogle Scholar
Embretson, S. E. (1985). Multicomponent latent trait models for test design. In Embretson, S. E. (Eds.), Test design: Developments in psychology and psychometrics, Orlando: Academic Press.Google Scholar
Falmagne, J-C. (1989). A latent trait model via a stochastic learning theory for a knowledge space. Psychometrika, 54, 283–303.CrossRefGoogle Scholar
Gagné, R. M. (1968). Learning hierarchies. Educational Psychologist, 6, 1–9.CrossRefGoogle Scholar
Glaser, R. (1981). The future of testing: A research agenda for cognitive psychology and psychometrics. American Psychologist, 36, 923–936.CrossRefGoogle Scholar
Glaser, R., Lesgold, A., Lajoie, S. (1987). Toward a cognitive theory for the measurement of achievement. In Ronning, R., Glover, J., Conoley, J. C., Witt, J. (Eds.), The influence of cognitive psychology on testing and measurement: The Buros-Nebraska Symposium on measurement and testing (pp. 41–85). Hillsdale, NJ: Erlbaum.Google Scholar
Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika, 61, 215–231.CrossRefGoogle Scholar
Haertel, E. H. (1984). An application of latent class models to assessment data. Applied Psychological Measurement, 8, 333–346.CrossRefGoogle Scholar
Kelderman, H. (1989). Item bias detection using loglinear IRT. Psychometrika, 54, 681–697.CrossRefGoogle Scholar
Kelderman, H., Macready, G. B. (1990). The use of loglinear models for assessing differential item functioning across manifest and latent examinee groups. Journal of Educational Measurement, 27, 307–327.CrossRefGoogle Scholar
Klein, M. F., Birenbaum, M., Standiford, S. N., Tatsuoka, K. K. (1981). Logical error analysis and construction of tests to diagnose student “bugs” in addition and subtraction of fractions, Urbana, IL: University of Illinois, Computer-Based Education Research Laboratory.Google Scholar
Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society, Series B, 44, 226–233.CrossRefGoogle Scholar
Maris, E. (1992). Psychometric models for psychological processes and structures, Belgium: Katholieke Universiteit Leuven.Google Scholar
Masters, G. N. (1988). Item discrimination: When more is worse. Journal of Educational Measurement, 25, 15–29.CrossRefGoogle Scholar
McHugh, R. B. (1956). Efficient estimation and local identification in latent class analysis. Psychometrika, 21, 331–347.CrossRefGoogle Scholar
Meng, X-L, Rubin, D. B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika, 80, 267–278.CrossRefGoogle Scholar
Mislevy, R. J. (1984). Estimating latent distributions. Psychometrika, 49, 359–381.CrossRefGoogle Scholar
Mislevy, R. J. (1986). Bayes model estimation in item response models. Psychometrika, 51, 177–195.CrossRefGoogle Scholar
Mislevy, R. J., Bock, R. D. (1989). PC-BILOG 3: Item analysis and test scoring with binary logistic models, Mooresville, IN: Scientific Software.Google Scholar
Mislevy, R. J., Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195–215.CrossRefGoogle Scholar
Mislevy, R. J., Wingersky, M. S., Irvine, S. H., Dann, P. L. (1991). Resolving mixtures of strategies in spatial visualization tasks. British Journal of Mathematical and Statistical Psychology, 44, 265–288.CrossRefGoogle ScholarPubMed
Paulson, J. A. (1986). Latent class representation of systematic patterns in test responses, Portland, OR: Portland State University, Psychology Department.CrossRefGoogle Scholar
Ramsay, J. O. (1975). Solving implicit equations in psychometric data analysis. Psychometrika, 40, 361–372.CrossRefGoogle Scholar
Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests, Chicago: University of Chicago Press.Google Scholar
Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271–282.CrossRefGoogle Scholar
Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12, 1151–1172.CrossRefGoogle Scholar
Samejima, F. (1983). A latent trait model for differential strategies in cognitive strategies, Knoxville, TN: University of Tennessee.Google Scholar
Siegler, R. S. (1981). Developmental sequences within and between concepts. Monograph of the Society for Research in Child Development, 46.CrossRefGoogle Scholar
Snow, R. E., Lohman, D. F. (1989). Implications of cognitive psychology for educational measurement. In Linn, R. L. (Eds.), Educational measurement 3rd ed., (pp. 263–331). New York: American Council on Education/Macmillan.Google Scholar
Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.CrossRefGoogle Scholar
Tatsuoka, K. K. (1990). Toward an integration of item response theory and cognitive error diagnosis. In Frederiksen, N., Glaser, R., Lesgold, A., Shafto, M. G. (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 453–488). Hillsdale, NJ: Erlbaum.Google Scholar
Thompson, P. W. (1982). Were lions to speak, we wouldn't understand. Journal of Mathematical Behavior, 3, 147–165.Google Scholar
Wilson, M. (1984). A Psychometric model of hierarchical development. Unpublished doctoral dissertation, University of Chicago.Google Scholar
Wilson, M. (1989). Saltus: A psychometric model of discontinuity in cognitive development. Psychological Bulletin, 105(2), 276–289.CrossRefGoogle Scholar
Wilson, M. (1989). Empirical examination of a learning hierarchy using an item response theory model. Journal of Experimental Education, 57(4), 357–371.CrossRefGoogle Scholar
Wilson, M., Iventosch, L. (1988). Using the Partial Credit model to investigate responses to structured subtests. Applied Measurement in Education, 1(4), 319–334.CrossRefGoogle Scholar
Wright, B. D., Douglas, G. A. (1977). Conditional versus unconditional procedures for sample-free item analysis. Educational and Psychological Measurement, 37, 573–586.CrossRefGoogle Scholar
Yamamoto, K. (1987). A model that combines IRT and latent class models, Champaign-Urbana: University of Illinois.Google Scholar