Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-07T17:47:52.174Z Has data issue: false hasContentIssue false
  • English
  • Français

A Mixture Cure-Rate Model for Responses and Response Times in Time-Limit Tests

Published online by Cambridge University Press:  01 January 2025

Yi-Hsuan Lee  and
Zhiliang Ying
Yi-Hsuan Lee*
Affiliation:
Educational Testing Service
Zhiliang Ying
Affiliation:
Columbia University Shanghai Center for Mathematical Sciences
*
Correspondence should be sent to Yi-Hsuan Lee, Educational Testing Service, Princeton, NJ, USA. E-mail: ylee@ets.org
Get access Rights & Permissions [Opens in a new window]

Abstract

Many large-scale standardized tests are intended to measure skills related to ability rather than the rate at which examinees can work. Time limits imposed on these tests make it difficult to distinguish between the effect of low proficiency and the effect of lack of time. This paper proposes a mixture cure-rate model approach to address this issue. Maximum likelihood estimation is proposed for parameter and variance estimation for three cases: when examinee parameters are to be estimated given precalibrated item parameters, when item parameters are to be calibrated given known examinee parameters, and when item parameters are to be estimated without assuming known examinee parameters. Large-sample properties are established for the cases under suitable regularity conditions. Simulation studies suggest that the proposed approach is appropriate for inferences concerning model parameters. In addition, not distinguishing between the effect of low proficiency and the effect of lack of time is shown to have considerable consequences for parameter estimation. A real data example is presented to demonstrate the new model. Choice of survival models for the latent power times is also discussed.

Keywords

response timetime limitpower testspeed testsurvival analysiscensoringcure model
Type
Original Paper
Information
Psychometrika , Volume 80 , Issue 3 , September 2015 , pp. 748 - 775
Copyright
Copyright © 2014 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Petrov, B.N., & Csaki, B.F. (Eds.), Second International Symposium on Information Theory (pp. 267–281). Budapest: Academiai Kiado.Google Scholar
Anderson, E.B. (1973). Conditional inference and models for measuring. Copenhagen: Mentalhygiejnisk Forskningsinstitut.Google Scholar
Bennett, S. (1983). Analysis of survival data by the proportional odds model. Statistics in Medicine, 2, 273–277.CrossRefGoogle ScholarPubMed
Berkson, J., & Gage, R.P. (1952). Survival curve for cancer patients following treatment. Journal of the American Statistical Association, 47(259), 501–515.CrossRefGoogle Scholar
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.CrossRefGoogle Scholar
Bolt, D.M., Cohen, A.S., & Wollack, J.A. (2002). Item parameter estimation under conditions of test speededness: Application of a mixture Rasch model with ordinal constraints. Journal of Educational Measurement, 39, 331–348.CrossRefGoogle Scholar
Bridgeman, B., McBride, A., & Monaghan, W., (2004). Testing and time limits. R&D Connections. Retrieved from http://www.ets.org/Media/Research/pdf/RD-Connections1.pdf.Google Scholar
Chang, Y-W, Tsai, R-C, & Hsu, N-J. (2014). A speeded item response model: Leave the harder till later. Psychometrika, 79, 255–274.CrossRefGoogle ScholarPubMed
Cox, D.R. (1961). Tests of separate families of hypotheses. In Proceedings of the Fourth Berkeley Symposium (Vol. 1, pp. 105–123). Berkeley: University of California Press.Google Scholar
Cox, D.R. (1962). Further results on tests of separate families of hypotheses. Journal of the Royal Statistical Society: Series B, 24, 406–424.CrossRefGoogle Scholar
Cox, D.R. (1972). Regression models and life-tables (with Discussion). Journal of the Royal Statistical Society: Series B, 34, 187–220.CrossRefGoogle Scholar
Farewell, V.T. (1982). The use of mixture models for the analysis of survival data with long-term survivors. Biometrics, 38, 1041–1046.CrossRefGoogle ScholarPubMed
Gulliksen, H. (1950). Theory of mental tests. New York: Wiley.CrossRefGoogle Scholar
Haberman, S.J. (2010). Model selection. In Peterson, P., Baker, E., & McGaw, B. (Eds.), International encyclopedia of education (pp. 289–295). Oxford: Elsevier.CrossRefGoogle Scholar
Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer.CrossRefGoogle Scholar
Ibrahim, J.G., Chen, M., & Sinha, D. (2001). Bayesian survival analysis. New York: Springer.CrossRefGoogle Scholar
Joe, H. (1997). Multivariate models and dependence concepts. London: Chapman & Hall.Google Scholar
Johnson, M.S. (2007). Marginal maximum likelihood estimation of item response models in R. Journal of Statistical Software, 20(10), 1–24.CrossRefGoogle Scholar
Kalbfleisch, J.D., & Prentice, R.L. (2002). The statistical analysis of failure time data (2nd ed.). New York: Wiley.CrossRefGoogle Scholar
Klein Entink, R.H., van der Linden, W.J., & Fox, J-P. (2009). A Box–Cox normal model for response times. British Journal of Mathematical and Statistical Psychology, 62, 621–640.CrossRefGoogle ScholarPubMed
Kuk, A.Y.C., & Chen, C-H. (1992). A mixture model combining logistic regression with proportional hazards regression. Biometrika, 79, 531–541.CrossRefGoogle Scholar
Lee, Y-H, & Chen, H. (2011). A review of recent response-time analyses in educational testing. Psychological Test and Assessment Modeling, 53(3), 359–379.Google Scholar
Lehmann, E.L., & Casella, G. (1998). Theory of point estimation (2nd ed.). New York: Springer.Google Scholar
Lu, W., & Ying, Z. (2004). On semiparametric transformation cure models. Biometrika, 91, 331–343.CrossRefGoogle Scholar
The MathWorks, Inc. (2007). Getting started with Matlab, Version 7.5. Natick, MA: The MathWorks, Inc.Google Scholar
Maris, G., & van der Maas, H. (2012). Speed-accuracy response models: Scoring rules based on response time and accuracy. Psychometrika, 77(4), 615–633.CrossRefGoogle Scholar
Neyman, J., & Scott, E. (1948). Consistent estimates based on partially consistent observations. Econometrica, 16(1), 1–32.CrossRefGoogle Scholar
Partchev, I., De Boeck, P., & Steyer, R. (2013). How much power and speed is measured in this test?. Assessment, 20(2), 242–252.CrossRefGoogle ScholarPubMed
Patz, R., & Junker, B.W. (1999). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 24, 342–366.CrossRefGoogle Scholar
Peng, Y., & Dear, K.B.G. (2000). A nonparametric mixture model for cure rate estimation. Biometrics, 56, 237–243.CrossRefGoogle ScholarPubMed
Peng, Y., Dear, K.B.G., & Denham, J.W. (1998). A generalized F mixture model for cure rate estimation. Statistics in Medicine, 17, 813–830.3.0.CO;2-#>CrossRefGoogle ScholarPubMed
Schnipke, D.L., & Scrams, D.J. (2002). Exploring issues of examinee behavior: Insights gained from response-time analyses. In Mills, C.N., Potenza, M., Fremer, J.J., & Ward, W. (Eds.), Computer-based testing: Building the foundation for future assessments (pp. 237–266). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Schwarz, G.E. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461–464.CrossRefGoogle Scholar
Scrams, D. J., & Schnipke, D. L., (1997, March). Making use of response times in standardized tests: Are accuracy and speed measuring the same thing? Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.Google Scholar
Sy, J.P., & Taylor, J.M.G. (2000). Estimation in a Cox proportional hazards cure model. Biometrics, 56, 227–236.CrossRefGoogle Scholar
Tatsuoka, K.K., & Tatsuoka, M.M. (1980). A model for incorporating response-time data in scoring achievement tests. In Weiss, D.J. (Ed.), Proceedings of the 1979 computerized adaptive testing conference (pp. 236–256). Minneapolis, MN: University of Minnesota, Department of Psychology, Psychometric Methods Program.Google Scholar
Thissen, D. (1983). Timed testing: An approach using item response theory. In Weiss, D.J. (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 179–203). New York: Academic Press.Google Scholar
van Breukelen, G.J.P. (2005). Psychometric modeling of response speed and accuracy with mixed and conditional regression. Psychometrika, 70(2), 359–376.CrossRefGoogle Scholar
van der Linden, W.J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31, 181–204.CrossRefGoogle Scholar
van der Linden, W.J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287–308.CrossRefGoogle Scholar
van der Linden, W.J. (2011). Setting time limits on tests. Applied Psychological Measurement, 35, 183–199.CrossRefGoogle Scholar
van der Linden, W. J., (2011b). Test design and speededness. Journal of Educational Measurement, 48, 44–60.CrossRefGoogle Scholar
Wang, C., Fan, Z., Chang, H.H., & Douglas, J.A. (2012). A semiparametric model for jointly analyzing response times and accuracy in computerized testing. Journal of Educational and Behavioral Statistics, 38, 381–417.CrossRefGoogle Scholar
Wang, T. (2006). A model for the joint distribution of item response and response time using one-parameter Weibull distribution (CASMA Research Report 20). Iowa City, IA: Center for Advanced Studies in Measurement and Assessment.Google Scholar
Yamaguchi, K. (1992). Accelerated failure-time regression models with a regression model of surviving fraction: An application to the analysis of “permanent employment" in Japan. Journal of the American Statistical Association, 87(418), 284–292.Google Scholar
Yamamoto, K.Y. (1989). HYBRID model of IRT and latent class models (ETS Research Report No. RR-89-41). Princeton, NJ: Educational Testing Service.Google Scholar
Yamamoto, K., & Everson, H. (1997). Modeling the effects of test length and test time on parameter estimation using the hybrid model. In Rost, J. (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 89–98). Munster: Waxmann.Google Scholar