Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-07T18:18:55.689Z Has data issue: false hasContentIssue false
  • English
  • Français

Detecting Intervention Effects Using a Multilevel Latent Transition Analysis with a Mixture IRT Model

Published online by Cambridge University Press:  01 January 2025

Sun-Joo Cho ,
Allan S. Cohen  and
Brian Bottge
Sun-Joo Cho*
Affiliation:
Vanderbilt University
Allan S. Cohen
Affiliation:
The University of Georgia
Brian Bottge
Affiliation:
University of Kentucky
*
Requests for reprints should be sent to Sun-Joo Cho, Department of Psychology and Human Development, Peabody #H213A, Peabody College of Vanderbilt University, 230 Appleton Place, Nashville, TN 37203, USA. E-mail: sj.cho@vanderbilt.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

A multilevel latent transition analysis (LTA) with a mixture IRT measurement model (MixIRTM) is described for investigating the effectiveness of an intervention. The addition of a MixIRTM to the multilevel LTA permits consideration of both potential heterogeneity in students’ response to instructional intervention as well as a methodology for assessing stage sequential change over time at both student and teacher levels. Results from an LTA–MixIRTM and multilevel LTA–MixIRTM were compared in the context of an educational intervention study. Both models were able to describe homogeneities in problem solving and transition patterns. However, ignoring a multilevel structure in LTA–MixIRTM led to different results in group membership assignment in empirical results. Results for the multilevel LTA–MixIRTM indicated that there were distinct individual differences in the different transition patterns. The students receiving the intervention treatment outscored their business as usual (i.e., control group) counterparts on the curriculum-based Fractions Computation test. In addition, 27.4 % of the students in the sample moved from the low ability student-level latent class to the high ability student-level latent class. Students were characterized differently depending on the teacher-level latent class.

Keywords

latent class modelmixture item response theory modelmultilevel latent transition analysis
Type
Original Paper
Information
Psychometrika , Volume 78 , Issue 3 , July 2013 , pp. 576 - 600
Copyright
Copyright © 2013 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Petrov, B.N., Caski, F. (Eds.), Second international symposium on information theory, Budapest: Academiai Kiado 267281Google Scholar
Andersen, E.B. (1985). Estimating latent correlations between repeated testings. Psychometrika, 50, 3–16CrossRefGoogle Scholar
Asparouhov, T., Muthén, B. (2008). Multilevel mixture models. In Hancock, G.R., Samuelsen, K.M. (Eds.), Advances in latent variable mixture models, Charlotte: Information Age Publishing 2751Google Scholar
Ball, T.L. (1990). Prospective elementary and secondary teachers understanding of division. Journal for Research in Mathematics Education, 21, 132–144CrossRefGoogle Scholar
Bottge, B.A. (1999). Effects of contextualized math instruction on problem solving of average and below-average achieving students. Journal of Special Education, 33, 81–92CrossRefGoogle Scholar
Bottge, B.A., Heinrichs, M., Mehta, Z., Hung, Y. (2002). Weighing the benefits of anchored math instruction for students with disabilities in general education classes. Journal of Special Education, 35, 186–200CrossRefGoogle Scholar
Bottge, B.A., Heinrichs, M., Mehta, Z.D., Rueda, E., Hung, Y., & Danneker, J. (2004). Teaching mathematical problem solving to middle school students in math, technology education, and special education classrooms. RMLE Online, 27. http://www.nmsa.org/portals/0/pdf/publications/RMLE/rmle_vol27_no1_article1.pdf. Retrieved July 20, 2006. Google Scholar
Bottge, B.A., Rueda, E., Serlin, R.C., Hung, Y., Kwon, J. (2007). Shrinking achievement differences with anchored math problems: challenges and possibilities. Journal of Special Education, 41, 31–49CrossRefGoogle Scholar
Bottge, B., Ma, X., Toland, M., Gassaway, L., & Butler, M. (2012). Effects of enhanced anchored instruction on middle school students with disabilities in math. Department of Special Education & Rehabilitation Counseling, University of Kentucky, Lexington, KY. Google Scholar
Bozdogan, H. (1987). Model selection and Akaike’s information criterion (AIC): the general theory and its analytical extensions. Psychometrika, 52, 345–370CrossRefGoogle Scholar
Cho, S.-J., Cohen, A.S. (2010). A multilevel mixture IRT model with an application to DIF. Journal of Educational and Behavioral Statistics, 35, 336–370CrossRefGoogle Scholar
Cho, S.-J., Cohen, A.S., Kim, S.-H., Bottge, B. (2010). Latent transition analysis with a mixture IRT measurement model. Applied Psychological Measurement, 34, 583–604CrossRefGoogle Scholar
Cho, S.-J., Rabe-Hesketh, S. (2011). Alternating imputation posterior estimation of models with crossed random effects. Computational Statistics & Data Analysis, 55, 12–25CrossRefGoogle Scholar
Cho, S.-J., Athay, M., Preacher, K. J. (2012). Measuring change for a multidimensional test using a generalized explanatory longitudinal item response model. British Journal of Mathematical and Statistical Psychology,Google ScholarPubMed
Cho, S.-J., Cohen, A. S., Kim, S.-H. (2012). Markov chain Monte Carlo estimation of a mixture item response theory model. Journal of Statistical Computation and Simulation,Google Scholar
Cognition Technology Group at Vanderbilt. (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19, 2–10CrossRefGoogle Scholar
Cognition Technology Group at Vanderbilt (1997). The Jasper project: lessons in curriculum, instruction, assessment, and professional development, Mahwah: ErlbaumGoogle Scholar
Collins, L.M., Wugalter, S.E. (1992). Latent class models for stage-sequential dynamic latent variables. Multivariate Behavioral Research, 27, 131–157CrossRefGoogle Scholar
CCSSI-M (Common Core State Standards Initiative—Mathematics) (2011). http://www.corestandards.org. Retrieved December 5, 2011. Google Scholar
Congdon, P. (2003). Applied Bayesian modeling, New York: WileyCrossRefGoogle Scholar
De Boeck, P., Wilson, M., Acton, G.S. (2005). A conceptual and psychometric framework for distinguishing categories and dimensions. Psychological Review, 112, 129–158CrossRefGoogle ScholarPubMed
De Boeck, P., Cho, S.-J., Wilson, M. (2011). Explanatory secondary dimension modeling of latent DIF. Applied Psychological Measurement, 35, 583–603CrossRefGoogle Scholar
Embretson, S.E., Reise, S.P. (2000). Item response theory for psychologists, Mahwah: Lawrence-ErlbaumGoogle Scholar
Frühwirth-Schnatter, S. (2006). Finite mixture and Markov switching models, New York: SpringerGoogle Scholar
Graham, J.W., Collins, L.M., Wugalter, S.E., Chung, N.K., Hansen, W.B. (1991). Modeling transition in latent stage-sequential processes: a substance use prevention example. Journal of Consulting and Clinical Psychology, 59, 48–57CrossRefGoogle ScholarPubMed
Heinen, T. (1996). Latent classes and discrete latent trait models, Thousand Oaks: Sage PublicationsGoogle Scholar
Henry, K.L., Muthén, B. (2010). Multilevel latent class analysis: an application of adolescent smoking typologies with individual and contextual predictors. Structural Equation Modeling. A Multidisciplinary Journal, 17, 193–215CrossRefGoogle ScholarPubMed
Hickey, D.T., Moore, A.L., Pellegrino, J.W. (2001). The motivational and academic consequences of elementary mathematics environments: do constructivist innovations and reforms make a difference?. American Educational Research Journal, 38, 611–652CrossRefGoogle Scholar
Lee van Horn, M., Fagan, A.A., Jaki, T., Brown, E.C., Hawkins, D., Arthur, M.W., Abbott, R.D., Catalano, R.F. (2008). Using multilevel mixtures to evaluate intervention effects in group randomized trials. Multivariate Behavioral Research, 43, 289–326CrossRefGoogle Scholar
Li, F., Cohen, A.S., Kim, S.-H., Cho, S.-J. (2009). Model selection methods for dichotomous mixture IRT models. Applied Psychological Measurement, 33, 353–373CrossRefGoogle Scholar
Maxwell, S.E., Tiberio, S. (2007). Multilevel models of change: fundamental concepts and relationships to mixed models and latent growth-curve models. In Ong, A., van Dulmen, M.H.M. (Eds.), Oxford handbook of methods in positive psychology, New York: Oxford University Press 439452Google Scholar
McDonald, R.P. (1967). Nonlinear factor analysis, Richmond: Psychometric Corporation Retrieved from http://www.psychometrika.org/journal/online/MN15.pdfGoogle ScholarPubMed
McLachlan, G., Peel, D. (2000). Finite mixture models, New York: WileyCrossRefGoogle Scholar
Muthén, B., Brown, C.H., Jo, B., Khoo, S.-T., Yang, C.C., Wang, C.-P., Kellam, S.G., Carlin, J.B., Liao, J. (2002). General growth mixture modeling for randomized preventive interventions. Biostatistics, 3, 459–475CrossRefGoogle ScholarPubMed
Muthén, B., Asparouhov, T. (2006). Item response mixture modeling: application to tobacco dependence criteria. Addictive Behaviors, 31, 1050–1066CrossRefGoogle ScholarPubMed
Muthén, L.K., Muthén, B.O. (2006–2010). Mplus [computer program], Los Angeles: Muthén & MuthénGoogle Scholar
Nylund, K.L., Asparouhov, T., Muthén, B.O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Structural Equation Modeling, 14, 535–569CrossRefGoogle Scholar
Paek, I., Cho, S.-J., & Cohen, A.S. (2010). A note on scaling linking in mixture IRT model applications. Paper presented at the National Council on Measurement in Education. Denver. Google Scholar
Raudenbush, S.W., Bryk, A.G. (2002). Hierarchical linear models: applications and data analysis methods, (2nd ed.). Thousand Oaks: SageGoogle Scholar
Rijmen, F., De Boeck, P. (2005). A relation between a between-item multidimensional IRT model and the mixture-Rasch model. Psychometrika, 70, 481–496CrossRefGoogle Scholar
Rijmen, F., De Boeck, P., van der Maas, H.L.J. (2005). An IRT model with a parameter-driven process for change. Psychometrika, 70, 651–669CrossRefGoogle Scholar
Rost, J. (1990). Rasch models in latent classes: an integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271–282CrossRefGoogle Scholar
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464CrossRefGoogle Scholar
Skrondal, A., Rabe-Hesketh, S. (2004). Generalized latent variable modeling: multilevel, longitudinal and structural equation models, Boca Raton: Chapman & HallCrossRefGoogle Scholar
Steinley, D., McDonald, R.P. (2007). Examining factor score distributions to determine the nature of latent spaces. Multivariate Behavioral Research, 42, 133–156CrossRefGoogle ScholarPubMed
Sugiura, N. (1978). Further analysis of the data by Akaike’s information criterion of model fitting. Suri-Kagaku (Mathematic Science), 153, 12–18 (in Japanese)Google Scholar
Vermunt, J.K. (2003). Multilevel latent class models. Sociological Methodology, 33, 213–239CrossRefGoogle Scholar
Vermunt, J.K. (2008). Latent class and finite mixture models for multilevel data sets. Statistical Methods in Medical Research, 17, 33–51CrossRefGoogle ScholarPubMed
Vermunt, J.K., Magidson, J. (2005). Technical guide for Latent GOLD 4.0: basic and advanced, Belmont: Statistical InnovationsGoogle Scholar
von Davier, M., Xu, X., Carstensen, C.H. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318–336CrossRefGoogle Scholar
von Davier, M., Yamamoto, K. (2004). Partially observed mixtures of IRT models: an extension of the generalized partial-credit model. Applied Psychological Measurement, 28, 389–406CrossRefGoogle Scholar
Yang, C. (2006). Evaluating latent class analyses in qualitative phenotype identification. Computational Statistics & Data Analysis, 50, 1090–1104CrossRefGoogle Scholar
Yu, H.-T. (2007). Multilevel latent Markov models for nested longitudinal discrete data. Unpublished doctoral dissertation. University of Illinois at Urbana-Champaign. Google Scholar