Hostname: page-component-cd9895bd7-lnqnp Total loading time: 0 Render date: 2025-01-05T00:31:26.289Z Has data issue: false hasContentIssue false
  • English
  • Français

Bayes Factor Covariance Testing in Item Response Models

Published online by Cambridge University Press:  01 January 2025

Jean-Paul Fox ,
Joris Mulder  and
Sandip Sinharay
Jean-Paul Fox*
Affiliation:
University of Twente
Joris Mulder
Affiliation:
Tilburg University
Sandip Sinharay
Affiliation:
Educational Testing Service
*
Correspondence should be made to Jean-Paul Fox, Department of Research Methodology, Measurement and Data Analysis, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands. Email: j.p.fox@utwente.nl
Get access Rights & Permissions [Opens in a new window]

Abstract

Two marginal one-parameter item response theory models are introduced, by integrating out the latent variable or random item parameter. It is shown that both marginal response models are multivariate (probit) models with a compound symmetry covariance structure. Several common hypotheses concerning the underlying covariance structure are evaluated using (fractional) Bayes factor tests. The support for a unidimensional factor (i.e., assumption of local independence) and differential item functioning are evaluated by testing the covariance components. The posterior distribution of common covariance components is obtained in closed form by transforming latent responses with an orthogonal (Helmert) matrix. This posterior distribution is defined as a shifted-inverse-gamma, thereby introducing a default prior and a balanced prior distribution. Based on that, an MCMC algorithm is described to estimate all model parameters and to compute (fractional) Bayes factor tests. Simulation studies are used to show that the (fractional) Bayes factor tests have good properties for testing the underlying covariance structure of binary response data. The method is illustrated with two real data studies.

Keywords

Bayesian inferenceBayes factormarginal IRTlocal independencerandom item parameter
Type
Original Paper
Information
Psychometrika , Volume 82 , Issue 4 , December 2017 , pp. 979 - 1006
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Albert, J. H. & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association 88, 669–679CrossRefGoogle Scholar
Albert, J. H. & Chib, S. (1997). Bayesian test and model diagnostic in conditionally independent hierarchical models. Journal of the American Statistical Association 92, 916–925CrossRefGoogle Scholar
Azevedo, CLN Fox, J.-P. & Andrade, D. F. (2016). Bayesian longitudinal item response modeling with restricted covariance pattern structures. Statistics and Computing 26, 443–460CrossRefGoogle Scholar
Box, G. & Tiao, G. (1973). Bayesian inference in statistical analysis Reading, MA: Addison-WesleyGoogle Scholar
Cai, B. & Dunson, D. B. (2006). Bayesian covariance selection in generalized linear mixed models. Biometrics 62, 446–457CrossRefGoogle ScholarPubMed
Chib, S. & Greenberg, E. (1998). Analysis of multivariate probit models. Biometrika 85, 347–361CrossRefGoogle Scholar
Daniels, M. J. & Pourahmadi, M. (2002). Bayesian anaysis of covariance matrices and dynamic models for longitudinal data. Biometrika 89, 553–566CrossRefGoogle Scholar
De Boeck, P. (2008). Random item irt models. Psychometrika 73, 533–559CrossRefGoogle Scholar
De Jong, M. G. Steenkamp, JBEM & Fox, J.-P. (2007). Relaxing cross-national measurement invariance using a hierarchical IRT model. Journal of Consumer Research 34, 260–278CrossRefGoogle Scholar
Edwards, Y. D. & Allenby, G. M. (2003). Multivariate analysis of multiple response data. Journal of Marketing Research 40, 321–334CrossRefGoogle Scholar
Fox, J.-P. (2010). Bayesian item response modeling: Theory and applications New York, NY: SpringerCrossRefGoogle Scholar
Hoff, P. D. (2009). A first course in Bayesian statistical methods New York, NY: SpringerCrossRefGoogle Scholar
Hsiao, C. K. (1997). Approximate bayes factors when a mode occurs on the boundary. Journal of the American Statistical Association 92, 656–663CrossRefGoogle Scholar
Jeffreys, H. (1961). Theory of probability 3New York, NY: Oxford University PressGoogle Scholar
Kass, R. & Raftery, A. (1995). Bayes factors. Journal of the American Statistical Association 90, 773–795CrossRefGoogle Scholar
Kinney, S. K. & Dunson, D. B. (2008). Fixed and random effects selection in linear and logistic models. Biometrics 63, 690–698CrossRefGoogle Scholar
Klugkist, I. & Hoijtink, H. (2007). The bayes factor for inequality and about equality constrained models. Computational Statistics and Data Analysis 51, 6367–6379CrossRefGoogle Scholar
Lancaster, H. O. (1965). The helmert matrices. The American Mathematical Monthly 72, 4–12CrossRefGoogle Scholar
Mulder, J. (2014). Prior adjusted default Bayes factors for testing (in)equality constrained hypotheses. Computational Statistics and Data Analysis 71, 448–463CrossRefGoogle Scholar
Mulder, J. Hoijtink, H. & Klugkist, I. (2010). Equality and inequality constrained multivariate linear models: Objective model selection using constrained posterior priors. Journal of Statistical Planning and Inference 140, 887–906CrossRefGoogle Scholar
O’Hagan, A. (1995). Fractional bayes factors for model comparison. Journal of the Royal Statistical Society, Series B 57, 99–138CrossRefGoogle Scholar
Pauler, D. K. Wakefield, J. C. & Kass, R. E. (1999). Bayes factors and approximations for variance component models. Journal of the American Statistical Association 94, 1242–1253CrossRefGoogle Scholar
Perrakisa, K. Ntzoufrasa, I. & Tsionasb, E. (2014). On the use of marginal posteriors in marginal likelihood estimation via importance sampling. Computational Statistics and Data Analysis 77, 54–69CrossRefGoogle Scholar
Plummer, M. Best, N. Cowles, K. & Vines, K. (2006). Coda: Convergence diagnosis and output analysis for MCMC. R News 6 (1), 7–11Google Scholar
Saville, B. R. & Herring, A. H. (2009). Testing random effects in the linear mixed model using approximate bayes factors. Biometrics 65, 369–376CrossRefGoogle ScholarPubMed
Searle, S. R. (1971). Linear models 2London: WileyGoogle Scholar
Sinharay, S. (2013). A note on assessing the added value of subscores. Educational Measurement: Issues and Practice 32, 38–42CrossRefGoogle Scholar
Sinharay, S. Johnson, M. S. & Stern, H. S. (2006). Posterior predictive assessment of item response theory models. Applied Psychological Measurement 30, 298–321CrossRefGoogle Scholar
Sinharay, S. & Stern, H. S. (2002). On the sensitivity of bayes factors to the prior distributions. The American Statistician 56, 196–201CrossRefGoogle Scholar
Skrondal, A. & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models Boca Raton, FL: Chapman and HallCrossRefGoogle Scholar
Spiegelhalter, D. J. Best, N. G. Carlin, B. P. & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society, Series B 64, 583–639CrossRefGoogle Scholar
Stout, W. Habing, B. Douglas, J. Kim, H. R. Roussos, L. & Zhang, J. (1996). Conditional covariance-based nonparametric multidimensionality assessment. Applied Psychological Measurement 20 (4), 331–354CrossRefGoogle Scholar
van der Linden, W. J. Hambleton, R. K. van der Linden, W. & Hambleton, R. (1997). Item response theory: Brief history, common models, and extensions. Handbook of modern item response theory New York, NY: Springer 1–28CrossRefGoogle Scholar
Verhagen, A. J. & Fox, J.-P. (2013). Bayesian tests of measurement invariance. British Journal of Mathematical and Statistical Psychology 66, 383–401CrossRefGoogle ScholarPubMed
Verhagen, A. J. & Fox, J.-P. (2013). Longitudinal measurement in health-related surveys. A Bayesian joint growth model for multivariate ordinal responses. Statis- tics in Medicine 32, 2988–3005CrossRefGoogle ScholarPubMed
Wainer, H. Bradlow, E. T. & Wang, X. (2007). Testlet response theory and its applications New York, NY: Cambridge University PressCrossRefGoogle Scholar
Yao, L. (2010). Reporting valid and reliable overall scores and domain scores. Journal of Educational Measurement: Issues and Practice 47, 339–360CrossRefGoogle Scholar