Hostname: page-component-745bb68f8f-g4j75 Total loading time: 0 Render date: 2025-01-07T16:57:54.737Z Has data issue: false hasContentIssue false
  • English
  • Français

Exact Tests for the Rasch Model Via Sequential Importance Sampling

Published online by Cambridge University Press:  01 January 2025

Yuguo Chen  and
Dylan Small
Yuguo Chen
Affiliation:
Duke University
Dylan Small*
Affiliation:
University of Pennsylvania
*
Requests for reprints should be sent to Dylan Small, Department of Statistics, The Wharton School, University of Pennsylvania, 400 Jon M. Huntsman Hall, 3730 Locust Walk, Philadelphia, PA 19104, USA. E-mail: dsmall@wharton.upenn.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Rasch proposed an exact conditional inference approach to testing his model but never implemented it because it involves the calculation of a complicated probability. This paper furthers Rasch’s approach by (1) providing an efficient Monte Carlo methodology for accurately approximating the required probability and (2) illustrating the usefulness of Rasch’s approach for several important testing problems through simulation studies. Our Monte Carlo methodology is shown to compare favorably to other Monte Carlo methods proposed for this problem in two respects: it is considerably faster and it provides more reliable estimates of the Monte Carlo standard error.

Keywords

Rasch modelexact testsgoodness of fit testsMonte Carlo methodssequential importance sampling
Type
Original Paper
Information
Psychometrika , Volume 70 , Issue 1 , March 2005 , pp. 11 - 30
Copyright
Copyright © 2005 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This Research was supported in part by National Science Foundation grant DMS-0203762 and a University of Pennsylvania Research Foundation grant.

The authors are grateful to Don Burdick for helpful comments. In addition, the authors wish to thank the editor, the associate editor, and the referees for their helpful suggestions.

References

Andersen, E.B. (1973). Conditional inference for multiple-choice questionnaires. British Journal of Mathematical and Statistical Psychology, 26, 31–44.CrossRefGoogle Scholar
Andersen, E.B. (1980). Discrete Statistical Models with Social Science Applications. Amsterdam: North-Holland.Google Scholar
Andersen, E.B. (1995). What Georg Rasch would have thought about this book. In Fischer, G.H., & Molenaar, I.W. (Eds.), Rasch Models: Their Foundations, Recent Developments and Applications. New York: Springer-Verlag.Google Scholar
Békéssy, A., Békéssy, P., & Komlos, J. (1972). Asymptotic enumeration of regular matrices. Studia Scientiarum Mathematicarum Hungarica, 7, 343–353.Google Scholar
Bender, E.A. (1974). The asymptotic number of nonnegative integer matrices with given row and column sums. Discrete Mathematics, 10, 217–223.CrossRefGoogle Scholar
Besag, J., & Clifford, P. (1989). Generalized Monte-Carlo significance tests. Biometrika, 76, 633–642.CrossRefGoogle Scholar
Birch, M.W. (1964). The detection of partial association I: The 2×2 case. Journal of the Royal Statistical Society Series B, 26, 313–324.CrossRefGoogle Scholar
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In Lord, F.M., & Novick, M.R. (Eds.), Statistical Theories of Mental Test Scores. Reading: Addison-Wesley.Google Scholar
Bishop, Y.M.M., Fienberg, S.E., & Holland, P.W. (1975). Discrete Multivariate Analysis. Cambridge: MIT Press.Google Scholar
Chen, X.H., Dempster, A.P., & Liu, J.S. (1994). Weighted finite population sampling to maximize entropy. Biometrika, 81, 457–469.CrossRefGoogle Scholar
Chen, Y. (2001). Sequential importance sampling with resampling: theory and applications. PhD dissertation, Department of Statistics, Stanford University.Google Scholar
Chen, Y., Diaconis, P., Holmes, S., & Liu, J.S. (2005). Sequential Monte Carlo methods for statistical analysis of tables. Journal of the American Statistical Association, 100, 109–120.CrossRefGoogle Scholar
Coleman, J.S. (1957). Multidimensional scale analysis. American Journal of Sociology, 63, 253–263.CrossRefGoogle Scholar
Connor, E.F., & Simberloff, D. (1979). The assembly of species communities: chance or competition. Ecology, 60, 1132–1140.CrossRefGoogle Scholar
Duncan, O.D. (1984). Rasch measurement: further examples and discussion. In Turner, C.F., & Martih, E. (Eds.), Surveying Subjective Phenomena (Volume 2). New York: Russell Sage Foundation.Google Scholar
Efron, B., & Tibshirani, R.J. (1993). An Introduction to the Bootstrap. Boca Raton: Chapman & Hall.CrossRefGoogle Scholar
Fischer, G.H. (1993). Notes on the Mantel–Haenszel procedure and another chi-squared test for the assessment of DIF. Methodika, 7, 88–100.Google Scholar
Fischer, G.H. (1995). Some neglected problems in IRT. Psychometrika, 60, 449–487.CrossRefGoogle Scholar
Fischer, G.H., & Molenaar, I.W. (1995). Rasch Models: Foundations, Recent Developments, and Applications. New York: Springer-Verlag.CrossRefGoogle Scholar
Glas, C.A.W. (1988). The derivation of some tests for the Rasch model from the multinomial distribution. Psychometrika, 63, 525–546.CrossRefGoogle Scholar
Glas, C.A.W., & Verhelst, N.D. (1995). Testing the Rasch model. In Fischer, G.H., & Molenaar, I.W. (Eds.), Rasch Models: Their Foundations, Recent Developments and Applications. New York: Springer-Verlag.Google Scholar
Good, I.J., & Crook, J. (1977). The enumeration of arrays and a generalization related to contingency tables. Discrete Mathematics, 19, 23–65.CrossRefGoogle Scholar
Gustafsson, J. (1980). Testing and obtaining fit of data to the Rasch model. British Journal of Mathematical and Statistical Psychology, 33, 205–233.CrossRefGoogle Scholar
Haberman, S.J. (1974). The Analysis of Frequency Data. Chicago: The University of Chicago Press.Google Scholar
Holland, P.W., & Thayer, D.T. (1988). Differential item performance and the Mantel–Haenszel procedure. In Wainer, H., & Braun, H.I. (Eds.), Test Validity. Hillsdale: Erlbaum.Google Scholar
Kelderman, H. (1984). Loglinear Rasch model tests. Psychometrika, 49, 223–245.CrossRefGoogle Scholar
Kelderman, H. (1989). Item bias detection using loglinear IRT. Psychometrika, 54, 681–697.CrossRefGoogle Scholar
Kong, A., Liu, J.S., & Wong, W.H. (1994). Sequential imputations and Bayesian missing data problems. Journal of the American Statistical Association, 89, 278–288.CrossRefGoogle Scholar
Lancaster, H.O. (1961). Significance tests in discrete distributions. Journal of the American Statistical Association, 56, 223–234.CrossRefGoogle Scholar
Lehmann, E.L. (1986). Testing Statistical Hypotheses (2nd ed.). New York: Wiley.CrossRefGoogle Scholar
Liu, J.S. (2001). Monte Carlo Strategies in Scientific Computing. New York: Springer-Verlag.Google Scholar
Mantel, N., & Fleiss, J.L. (1980). Minimum expected cell size requirements for the Mantel–Haenszel one-degree-of-freedom chi square test and a related rapid procedure. American Journal of Epidemiology, 112, 129–134.CrossRefGoogle Scholar
Martin-Löf, P. (1973). Statistika Modeller. Anteckningar från seminarier läsåret 1969-1970 utarbetade av Rolf Sundberg. 2: a uppl.) (Statistical models. Notes from seminars 1969-1970 by Rolf Sundberg, 2nd ed). Institutet för Försä kringsmatematik och Matematisk Statistik vid Stockholms Universitet.Google Scholar
Mazor, K.M., Clauser, B.E., & Hambleton, R.K. (1992). The effect of sample size on the functioning of the Mantel–Haenszel statistic. Educational and Psychological Measurement, 52, 443–451.CrossRefGoogle Scholar
Mengersen, K.L., Robert, C.P., & Cuihenneuc-Joyaux, C. (1999). MCMC convergence diagnostics: A review. In Bernando, J.M., Berger, J.O., Dawid, A.P., & Smith, A.F.M. (Eds.), Bayesian Statistics (Volume 6). Oxford: Clarendon Press.Google Scholar
Molenaar, I.W. (1983). Some improved diagnostics for failure in the Rasch model. Psychometrika, 48, 49–72.CrossRefGoogle Scholar
Molenaar, I.W. (1995). Estimation of Item Parameters. In Fischer, G.H., Molenaar, I.W. (Eds.), Rasch Models: Their Foundations, Recent Developments and Applications. New York: Springer-Verlag.Google Scholar
Neyman, J., & Pearson, E.S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transcations of the Royal Society A, 231, 289–337.Google Scholar
Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.CrossRefGoogle Scholar
Pashall, C.G., & Miller, T.R. (1995). Exact versus asymptotic Mantel–Haenszel DIF statistics: a comparison of performance under small-sample conditions. Journal of Educational Measurement, 32, 302–316.CrossRefGoogle Scholar
Ponocny, I. (2001). Nonparametric goodness-of-fit tests for the Rasch model. Psychometrika, 66, 437–460.CrossRefGoogle Scholar
Rao, A.R., Jana, R., & Bandyopadhyay, S. (1996). A Markov chain Monte Carlo method for generating random (0,1) matrices with given marginals. Sankhya series A, 58, 225–242.Google Scholar
Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Danish Institute for Educational Research.Google Scholar
Rasch, G. (1966). An individualistic approach to item analysis. In Lazarsfeld, P.F., & Henry, N.W. (Eds.), Readings in Mathematical Social Science. Chicago: Science Research Associates.Google Scholar
Roberts, A., & Stone, L. (1990). Island-sharing by archipelago species. Oecologia, 83, 560–567.CrossRefGoogle ScholarPubMed
Shepard, L.A., Camilli, G., & Williams, D.M. (1985). Validity of approximation techniques for detecting item bias. Journal of Educational Measurement, 22, 77–105.CrossRefGoogle Scholar
Snijders, T.A.B. (1991). Enumeration and simulation methods for 0-1 matrices with given marginals. Psychometrika, 56 (3397–417.CrossRefGoogle Scholar
Stouffer, S.A., & Toby, J. (1951). Role conflict and personality. American Journal of Sociology, 56, 395–406.CrossRefGoogle Scholar
Tatsuoka, K.K., & Linn, R.L. (1983). Indices for detecting unusual patterns: links between two general approaches and potential applications. Applied Psychological Measurement, 7, 81–96.CrossRefGoogle Scholar
van den Wollenberg, A.L. (1982). Two new test statistics for the Rasch model. Psychometrika, 47, 123–139.CrossRefGoogle Scholar
Wright, B.D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14, 97–116.CrossRefGoogle Scholar
Wright, B.D., & Stone, M.H. (1979). Best Test Design. Chicago: Mesa Press.Google Scholar