Hostname: page-component-745bb68f8f-s22k5 Total loading time: 0 Render date: 2025-01-07T17:54:10.706Z Has data issue: false hasContentIssue false
  • English
  • Français

Modeling Conditional Dependence of Response Accuracy and Response Time with the Diffusion Item Response Theory Model

Published online by Cambridge University Press:  01 January 2025

Inhan Kang Open the ORCID record for Inhan Kang [Opens in a new window] ,
Paul De Boeck  and
Roger Ratcliff
Inhan Kang*
Affiliation:
The Ohio State University
Paul De Boeck
Affiliation:
The Ohio State University
Roger Ratcliff
Affiliation:
The Ohio State University
*
Correspondence should be made to Inhan Kang, The Ohio State University, 291 Psychology Building, 1835 Neil Avenue, Columbus, OH 43210, USA. Email: kang.985@osu.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, we propose a model-based method to study conditional dependence between response accuracy and response time (RT) with the diffusion IRT model (Tuerlinckx and De Boeck in Psychometrika 70(4):629–650, 2005, https://doi.org/10.1007/s11336-000-0810-3; van der Maas et al. in Psychol Rev 118(2):339–356, 2011, https://doi.org/10.1080/20445911.2011.454498). We extend the earlier diffusion IRT model by introducing variability across persons and items in cognitive capacity (drift rate in the evidence accumulation process) and variability in the starting point of the decision processes. We show that the extended model can explain the behavioral patterns of conditional dependency found in the previous studies in psychometrics. Variability in cognitive capacity can predict positive and negative conditional dependency and their interaction with the item difficulty. Variability in starting point can account for the early changes in the response accuracy as a function of RT given the person and item effects. By the combination of the two variability components, the extended model can produce the curvilinear conditional accuracy functions that have been observed in psychometric data. We also provide a simulation study to validate the parameter recovery of the proposed model and present two empirical applications to show how to implement the model to study conditional dependency underlying data response accuracy and RTs.

Keywords

diffusion IRT modelresponse timepsychological processconditional dependencyprocess modeling
Type
Theory and Methods
Information
Psychometrika , Volume 87 , Issue 2: Special Issue on Forecasting with Intensive Longitudinal Data , June 2022 , pp. 725 - 748
Copyright
Copyright © 2021 This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Birnbaum, A.(1969).Statistical theory for logistic mental test models with a prior distribution of ability.Journal of Mathematical Psychology,6(2),258–276.CrossRefGoogle Scholar
Blurton, S. P.,Kesselmeier, M., &Gondan, M.(2017).The first-passage time distribution for the diffusion model with variable drift.Journal of Mathematical Psychology,76 7–12.CrossRefGoogle Scholar
Bolsinova, M.,De Boeck, P., &Tijmstra, J.(2017).Modelling conditional dependence between response and accuracy.Psychometrika,82(4),1126–1148.CrossRefGoogle ScholarPubMed
Bolsinova, M., &Maris, G.(2016).A test for conditional independence between response time and accuracy.British Journal of Mathematical and Statistical Psychology,69(1),62–79.CrossRefGoogle ScholarPubMed
Bolsinova, M., &Molenaar, D.(2018).Modeling nonlinear conditional dependence between response time and accuracy.Frontiers in Psychology,9(1525),1–12.CrossRefGoogle ScholarPubMed
Bolsinova, M., &Molenaar, D.(2019).Nonlinear indicator-level moderation in latent variable models.Multivariate Behavioral Research,54(1),62–84.CrossRefGoogle ScholarPubMed
Bolsinova, M., &Tijmstra, J.(2018).Improving precision of ability estimation: Getting more from response times.British Journal of Mathematical and Statistical Psychology,71(1),13–38.CrossRefGoogle ScholarPubMed
Bolsinova, M.,Tijmstra, J., &Molenaar, D.(2017).Response moderation models for conditional dependence between response time and response accuracy.British Journal of Mathematical and Statistical Psychology,70 257–279.CrossRefGoogle ScholarPubMed
Bolsinova, M.,Tijmstra, J.,Molenaar, D., &De Boeck, P.(2017).Conditional dependence between response time and accuracy: An overview of its possible sources and directions for distinguishing between them.Frontiers in Psychology,8 202CrossRefGoogle ScholarPubMed
Borsboom, D.,Mellenbergh, G. J., &van Heerden, J.(2003).The theoretical status of latent variables.Psychological Review,110(2),203–219.CrossRefGoogle ScholarPubMed
Borsboom, D.,Mellenbergh, G. J., &van Heerden, J.(2004).The concept of validity.Psychological Review,111(4),1061–1071.CrossRefGoogle ScholarPubMed
Borst, G.,Kievit, R. A.,Thompson, W. L., &Kosslyn, S. M.(2011).Mental rotation is not easily cognitively penetrable.Journal of Cognitive Psychology,23(1),60–75.CrossRefGoogle Scholar
Brown, S., &Steyvers, M.(2005).The dynamics of experimentally induced criterion shifts.Journal of Experimental Psychology: Learning, Memory, and Cognition,31(4),587–599.Google ScholarPubMed
Chen, H.,De Boeck, P.,Grady, M.,Yang, C.-L., &Waldschmidt, D.(2018).Curvilinear dependency of response accuracy on response time in cognitive tests.Intelligence,69 16–23.CrossRefGoogle Scholar
Chen, H., De Boeck, P., Grady, M., Yang, C.-L., & Waldschmidt, D. (2018b). A bifactor approach to modeling dependencies between response time and accuracy. Paper presented at the annual meeting of the National Council on Measurement in Education (NCME).Google Scholar
Cooper, L. A., & Shepard, R. N. (1973). Chronometric studies of the rotation of mental images. In W. G. Chase (Ed.), Visual information processing (pp. 75–176). Academic Press. https://doi.org/10.1016/B978-0-12-170150-5.50009-3.CrossRefGoogle Scholar
Cox, D., &Miller, H. D.(1970 The theory of stochastic processes,London:Methuen.Google Scholar
De Boeck, P.,Chen, H., &Davison, M.(2017).Spontaneous and imposed speed of cognitive test responses.British Journal of Mathematical and Statistical Psychology,70(2),225–237.CrossRefGoogle ScholarPubMed
De Boeck, P., &Jeon, M.(2019).An overview of models for response times and processes in cognitive tests.Frontiers in Psychology,10 102.CrossRefGoogle ScholarPubMed
DiTrapani, J.,Jeon, M.,De Boeck, P., &Partchev, I.(2016).Attempting to differentiate fast and slow intelligence: Using generalized item response trees to examine the role of speed on intelligence tests.Intelligence,56 82–92.CrossRefGoogle Scholar
Embretson, S. (2021). Response Time relationships within examinees: Implications for item response time models. In Wiberg, M.Molenaar, D. González, J.Böckenholt, U. & Kim, J. S.(Eds.), Quantitative psychology. Springer Proceedings in Mathematics & Statistics (Vol. 353). Springer, Cham. https://doi.org/10.1007/978-3-030-74772-5_5CrossRefGoogle Scholar
Forstmann, B.,Ratcliff, R., & Wagenmakers, E.-J.(2016).Sequential sampling models in cognitive neuroscience: Advantages, applications, and extensions.Annual Review of Psychology,67(1),641–666.CrossRefGoogle ScholarPubMed
Gelman, A. (1996). Inference and monitoring convergence. In Gilks, W. R. Richardson, S.& Spiegelhalter, D. J.(Eds.), Markov Chain Monte Carlo in Practice (pp. 131–143). CRC Press.Google Scholar
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., & Vehtari, A. D. B. R., (2013). Bayesian data analysis (3rd ed.). CRC Press.Google Scholar
Goldhammer, F., Naumann, J., & Greiff, S. (2015). More is not always better: The relation between item response and item response time in raven’s matrices. Journal of Intelligence, 3(1)., 21–40. https://doi.org/10.3390/jintelligence3010021.CrossRefGoogle Scholar
Goldhammer, F.,Naumann, J.,Stelter, A.,Tóth, K.,Rólke, H., &Klieme, E.(2014).The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment.Journal of Educational Psychology,106(3),608–626.CrossRefGoogle Scholar
Kang, I., &Ratcliff, R.(2020).Modeling the interaction of numerosity and perceptual variables with the diffusion model.Cognitive Psychology,120 1–42.CrossRefGoogle ScholarPubMed
Kang, I.,Ratcliff, R., &Voskuilen, C.(2020).A note on decomposition of sources of variability in perceptual decision-making.Journal of Mathematical Psychology,98 102431.CrossRefGoogle ScholarPubMed
Laming, D. R. J. (1968). Information theory of choice-reaction times. Academic Press.Google Scholar
Luce, R. D. (1986). Response times: Their role in inferring elementary mental organization. Oxford University Publication.Google Scholar
Maris, G., &van der Maas, H. L. J.(2012).Speed-accuracy response models: Scoring rules based on response time and accuracy.Psychometrika,4 615–633.CrossRefGoogle Scholar
McKoon, G., &Ratcliff, R.(2016).Adults with poor reading skills: How lexical knowledge interacts with scores on standardized reading comprehension tests.Cognition,146 453–469.CrossRefGoogle ScholarPubMed
Meng, X.-B.,Tao, J., &Chang, H.-H.(2015).A conditional joint modeling approach for locally dependent item responses and response times.Journal of Educational Measurement,52(1),1–27.CrossRefGoogle Scholar
Molenaar, D.,Bolsinova, M.,Rozsa, S., &De Boeck, P.(2016 Response mixture modeling of intraindividual differences in responses and response times to the Hungarian wisc-iv block design test.Journal of Intelligence,CrossRefGoogle Scholar
Molenaar, D.,Tuerlinckx, F., &van der Maas, H. L. J.(2015).Fitting diffusion item response theory models for responses and response times using the r package diffirt.Journal of Statistical Software,66(4),1–34.CrossRefGoogle Scholar
Partchev, I., &De Boeck, P.(2012).Can fast and slow intelligence be differentiated?.Intelligence,40(1),23–32.CrossRefGoogle Scholar
Ranger, J., &Kuhn, J.-T.(2018).Estimating diffusion-based item response theory models: Exploring the robustness of three old and two new estimators.Journal of Educational and Behavioral Statistics,43(6),635–662.CrossRefGoogle Scholar
Ranger, J.,Kuhn, J.-T., &Szardenings, C.(2016).Limited information estimation of the diffusion-based item response theory model for responses and response times.British Journal of Mathematical and Statistical Psychology,69(2),122–138.CrossRefGoogle ScholarPubMed
Ranger, J.,Kuhn, J.-T., &Szardenings, C.(2017).Analysing model fit of psychometric process models: An overview, a new test and an application to the diffusion model.British Journal of Mathematical and Statistical Psychology,70(2),209–224.CrossRefGoogle Scholar
Ranger, J.,Kuhn, J.-T., &Szardenings, C.(2020).Minimum distance estimation of multidimensional diffusion-based item response theory models.Multivariate Behavioral Research,55(6),941–957.CrossRefGoogle ScholarPubMed
Ratcliff, R.(1978).A theory of memory retrieval.Psychological Review,85(2),59–108.CrossRefGoogle Scholar
Ratcliff, R.(2002).A diffusion model account of response time and accuracy in a brightness discrimination task: Fitting real data and failing to fit fake but plausible data.Psychological Science,9(2),278–291.Google Scholar
Ratcliff, R., &Childers, R.(2015).Individual differences and fitting methods for the two-choice diffusion model of decision making.Decision,2 237–279.CrossRefGoogle Scholar
Ratcliff, R.,Gomez, P., &McKoon, G.(2003).A diffusion model account of the lexical decision task.Psychological Review,111(1),159–182.CrossRefGoogle Scholar
Ratcliff, R., &McKoon, G.(2008).The diffusion decision model: Theory and data for two-choice decision tasks.Neural Computation,20(4),873–922.CrossRefGoogle ScholarPubMed
Ratcliff, R., &McKoon, G.(2018).Modeling numerosity representation with an integrated diffusion model.Psychological Review,125(2),183–217.CrossRefGoogle ScholarPubMed
Ratcliff, R., &Rouder, J. N.(1998).Modeling response times for two-choice decisions.Psychological Science,9(5),347–356.CrossRefGoogle Scholar
Ratcliff, R.,Smith, P. L.,Brown, S. D., &McKoon, G.(2016).Diffusion decision model: Current issues and history.Trends in Cognitive Sciences,20(4),260–281.CrossRefGoogle ScholarPubMed
Ratcliff, R.,Van Zandt, T., &McKoon, G.(1999).Connectionist and diffusion models of reaction time.Psychological Review,106(2),261–300.CrossRefGoogle ScholarPubMed
Ratcliff, R.,Voskuilen, C., &McKoon, G.(2018).Internal and external sources of variability in perceptual decision-making.Psychological Review,125(1),33–46.CrossRefGoogle ScholarPubMed
San Martín, E.,del Pino, G., &De Boeck, P.(2006).IRT models for ability-based guessing.Applied Psychological Measurement,30(3),183–203.CrossRefGoogle Scholar
Shepard, R. N., & Cooper, L. N. (1982). Mental images and their transformations. MIT Press.Google Scholar
Shepard, R. N., & Metzler, J. (1971). Mental rotation of three-dimensional objects. Science, 171(3972)., 701–703. https://doi.org/10.1126/science.171.3972.701CrossRefGoogle Scholar
Smith, P. L., &Vickers, D.(1988).The accumulator model of two-choice discrimination.Journal of Mathematical Psychology,32(2),135–168.CrossRefGoogle Scholar
Stone, M.(1960).Models for choice-reaction time.Psychometrika,25 251–260.CrossRefGoogle Scholar
Swensson, R. G.(1972).The elusive tradeoff: Speed vs accuracy in visual discrimination tasks.Perception & Psychophysics,12 16–32.CrossRefGoogle Scholar
Ter Braak, C. J. F.(2006).A Markov Chain Monte Carlo version of the genetic algorithm differential evolution: Easy Bayesian computing for real parameter spaces.Statistics and Computing,16 239–249.CrossRefGoogle Scholar
Tuerlinckx, F.(2004).The efficient computation of the cumulative distribution and probability density functions in the diffusion model.Behavior Research Methods, Instruments, and Computers,36(4),702–716.CrossRefGoogle ScholarPubMed
Tuerlinckx, F., &De Boeck, P.(2005).Two interpretations of the discrimination parameter.Psychometrika,70(4),629–650.CrossRefGoogle Scholar
Tuerlinckx, F., Molenaar, D., & van der Maas, H. L. J. (2016). Diffusion-based response time models. In W. J. van der Linden (Ed.), Handbook of item response theory (pp. 283–300). Chapman and Hall/CRC.Google Scholar
Turner, B. M.,Sederberg, P. B.,Brown, S. D., &Steyvers, M.(2013).A method for efficiently sampling from distributions with correlated dimensions.Psychological Methods,18(3),368–384.CrossRefGoogle ScholarPubMed
van der Linden, W. J.(2007).A hierarchical framework for modeling speed and accuracy on test items.Psychometrika,72(3),287–308.CrossRefGoogle Scholar
van der Linden, W. J., &Glas, C. A. W.(2010).Statistical tests of conditional independence between responses and/or response times on test items.Psychometrika,75 120–139.CrossRefGoogle Scholar
van der Maas, H. L. J.,Molenaar, D.,Maris, G.,Kievit, R. A., &Borsboom, D.(2011).Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences.Psychological Review,118(2),339–356.CrossRefGoogle ScholarPubMed
van Rijn, P. W., &Ali, U. S.(2017).A comparison of item response models for accuracy and speed of item responses with applications to adaptive testing.British Journal of Mathematical and Statistical Psychology,70(2),317–345.CrossRefGoogle ScholarPubMed
van Rijn, P. W., &Ali, U. S.(2018).A generalized speed-accuracy response model for dichotomous items.Psychometrika,83(1),109–131.CrossRefGoogle ScholarPubMed
Wang, C., &Xu, G.(2015).A mixture hierarchical model for response times and response accuracy.British Journal of Mathematical and Statistical Psychology,68(3),456–477.CrossRefGoogle ScholarPubMed
Wang, C.,Xu, G., &Shang, Z.(2018).A two-stage approach to differentiating normal and aberrant behavior in computer based testing.Psychometrika,83(1),223–254.CrossRefGoogle ScholarPubMed
Supplementary material: File

Kang et al. supplementary material

Kang et al. supplementary material
Download Kang et al. supplementary material(File)
File 1.2 MB