Diagnostic Classification Models for Testlets: Methods and Theory

Abstract

Diagnostic classification models (DCMs) have seen wide applications in educational and psychological measurement, especially in formative assessment. DCMs in the presence of testlets have been studied in recent literature. A key ingredient in the statistical modeling and analysis of testlet-based DCMs is the superposition of two latent structures, the attribute profile and the testlet effect. This paper extends the standard testlet DINA (T-DINA) model to accommodate the potential correlation between the two latent structures. Model identifiability is studied and a set of sufficient conditions are proposed. As a byproduct, the identifiability of the standard T-DINA is also established. The proposed model is applied to a dataset from the 2015 Programme for International Student Assessment. Comparisons are made with DINA and T-DINA, showing that there is substantial improvement in terms of the goodness of fit. Simulations are conducted to assess the performance of the new method under various settings.

In educational and psychological assessments, there is a surge of interest in adapting psychometric models to informed learning. Diagnostic classification models (DCMs; see Rupp et al. 2010), with the capability of measuring examinees’ mastery of fine-grained skills, have gained increasing attention and popularity. Over the past few decades, a wide range of DCMs has been developed, for example, the deterministic inputs, noisy “and” gate (DINA; Macready & Dayton, 1977; Junker & Sijtsma, 2001) model, the deterministic inputs, noisy “or” gate (DINO; Templin & Henson, 2006) model, the generalized DINA (G-DINA; de la Torre, 2011) model, and the log-linear CDM (LCDM; Henson et al., 2009).

One fundamental assumption in latent variable modeling is the local independence of item responses given the measured trait or profile. Practical assessments, however, may comprise testlets (e.g., Bradlow et al., 1999), that is, sets of items based on a common stimulus, for instance, a passage in a reading assessment. Correlation among item responses within a testlet often cannot be fully explained by the measured trait or profile, and such a testlet effect leads to the violation of the commonly assumed local independence. Under item response theory (IRT) models for continuous traits, studies have shown that ignoring the testlet effect leads to biased estimation of parameters and standard errors (e.g., Bradlow et al., 1999; Wainer et al., 2007; Sireci et al., 1991). Under DCMs, Hansen (2013) and Chen et al. (2018) discussed the consequences of ignoring testlet effects, including inaccurate estimation of parameters and misclassification of examinee attribute profiles, especially when testlet effects are large.

To account for the possible local dependence due to the testlet effect, a commonly adopted approach in compensatory IRT is to superimpose a testlet-specific random effect term on each testlet in addition to the general trait(s) to be measured by the whole test. The response distribution of an item within a testlet is typically assumed to depend on a linear combination of the general trait(s) and the corresponding testlet-specific random effect term. Examples of this include the bi-factor model (e.g., Gibbons & Hedeker, 1992; DeMars, 2006), the two-tier model (Cai, 2010), and the testlet-effect model Bradlow et al. (1999). The general and testlet-specific traits are usually assumed to be normally distributed, and three additional assumptions are required: (1) the primary and testlet-specific traits jointly work to capture the local dependency within a testlet; (2) the testlets are assumed to be mutually exclusive, meaning each item loads on at most one testlet-specific dimension; (3) the general and specific effects are independent or, equivalently under the normal distribution, uncorrelated. Assumption (1) is the fundamental and essential idea behind incorporating testlet effects. Assumption (2) reduces computational complexity in parameter estimation (e.g., Cai, 2010), and the independence between testlet effects can be relaxed to model the additional correlation between specific dimensions unexplained by the general trait (e.g., Jennrich & Bentler, 2012). Assumption (3), uncorrelated general and testlet effects, is essential, as Fang et al. (2021) proved that bi-factor-type models treating general and specific effects as dependent cannot be statistically identified.

For diagnostic assessments with testlets, similar testlet-effect DCMs have been proposed, where a testlet-specific random effect is incorporated into the item response function (Hansen, 2013; Hansen et al., 2016; Zhan et al., 2015, 2018; Ma et al., 2023): Assumption (1) above straightforwardly carries over to testlet-effect DCMs. Specifically, the proposed models leverage the fact that both general and specific DCMs can be parameterized as a logistic model (e.g., Henson et al., 2009; de la Torre, 2011), where the linear component contains attribute main effects and higher-order interactions. Subsequently, a testlet random effect is included as an additive term to the linear component to account for additional within-testlet dependency. Assumption (2), each item loading on at most one testlet effect, is also imposed in most models, with the exceptions of Zhan et al. (2015, 2018), where in general, an item is allowed to crossload on multiple testlet effects. To impose Assumption (3) in testlet-effect DCMs, existing models still imposed the structural independence assumption, by assuming that the testlet-specific traits each stem from an independent normal distribution, either uncorrelated with a higher-order continuous trait (de la Torre and Douglas, 2004) underlying the discrete attributes (Hansen, 2013; Hansen et al., 2016; Zhan et al., 2015, 2018), or independent from the attribute pattern stemming from a multinomial distribution over permissible latent classes (Ma et al., 2023).

In the current paper, however, we show that imposing Assumption (3) under testlet-effect DCMs can be restrictive, resulting in a model that does not adequately account for the within-testlet local dependency. The reasoning is as follows: In confirmatory IRT and DCMs, focal traits are typically expert-defined and cannot be rotated without altering substantive interpretations. A testlet effect, which enters the measurement model as an additive term, defines a person-specific effect that explains the dependency in responses within the same testlet that has not been accounted for by the existing terms in the measurement model. For compensatory IRT models, it is natural to restrict the testlet effect to be independent of the focal traits. This is because, when the compensatory item response model is linear and the relationship between testlet-specific and focal traits is linear, any correlation between the testlet-specific and focal traits can be removed by projecting the testlet-specific factor to the vector space spanned by the focal factor(s) and, subsequently, treating the residual of the projection as the testlet-specific effect. However, for DCMs, where the focal traits are discrete and the measurement model involves higher-order interactions of latent attributes, such projection cannot be performed. Under testlet-effect DCMs, one thus faces a dilemma where the orthogonality between testlet-specific and focal effects is key to identifiability but cannot be naturally granted. To this end, we propose an extension to the testlet-effect DINA model, to account for the additional local dependency by adding an interaction term involving testlet-specific and focal effects.

The second issue addressed by the current work is testlet-effect DCM identifiability. Model identifiability is fundamental to consistent parameter estimation and valid statistical inference, and thus it is not only crucial to latent variable modeling but also has direct implications in test designs. It is well known that, without suitable assumptions, neither DCMs nor bi-factor models are identifiable. Identifiability conditions for various DCMs and restricted latent class models without testlet effects have been studied extensively (see e.g. Allman et al., 2009; Chen et al., 2015; Xu & Zhang, 2016; Xu, 2017; Fang et al., 2019; Gu & Xu, 2019). For bi-factor models, which are commonly used to account for testlet effects in continuous latent trait models, Fang et al. (2021) provided relatively complete characterizations of their identifiability conditions. On the other hand, establishing the identifiability conditions for testlet-effect DCMs is much more challenging as it involves both continuous testlet effects and discrete attribute profiles. To this end, the current work provides theoretical results on the identifiability conditions for testlet-effect DINA and DINO models under the probit link.

It is worth noting that, aside from testlet-effect DCMs, there exist other approaches for handling the local dependency induced by testlets in diagnostic assessments. This includes treating the items in a testlet as one single polytomous item (Ma and de la Torre, 2016), graphical DCM that additionally models item pairwise dependence (Kang et al., 2017), and nonparametric diagnostic classification that employs a testlet-effect-weighted distance measure (Sha, 2016). In addition, under testlet-effect DCMs, more general DCMs than the DINA and the DINO have been adopted (e.g., Hansen, 2013; Ma et al., 2023). However, the current paper will focus on accounting for the additional dependency and identifiability theory of the testlet-effect DINA and DINO models. While the DINA and DINO models are restrictive on the permissible attribute interactions, we aim to provide a thorough analysis for this simple case, in the hope that these initial results contribute to further discussions on more general models.

The rest of the paper is organized as follows. Section 1 introduces the extended testlet DINA model and discusses its connection to the independent testlet DINA model. Section 2 presents the main theoretical results for model identifiability. Applications to the 2015 Program for International Student Assessment (PISA) collaborative problem solving (CPS) assessment data are presented in Sect. 3. Section 4 is devoted to simulation studies. A discussion of the main results, limitations, and future work is provided in Sect. 5.

1. Testlet DINA with Dependency

In this section, we introduce our extension to the testlet DINA model, the interaction testlet DINA (IT-DINA) model. The IT-DINA model incorporates the potential dependency between testlet effects and latent attribute profiles via an interaction term. To explain the rationale behind incorporating the additional latent interaction, we begin with introducing the testlet-effect DINA measurement model as seen in Hansen (2013), Hansen et al. (2016), Zhan et al. (2015), Zhan et al. (2018), and Ma et al. (2023).

Consider a diagnostic test with J items designed to measure K attributes. The examinee’s attribute profile is represented by a K-dimensional binary vector $\alpha =({\alpha }_1,\cdots ,{\alpha }_K)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }=(\alpha _1,\cdots ,\alpha _K)$$\end{document} , where ${\alpha }_k=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha _k = 1$$\end{document} or 0, indicating the mastery or nonmastery of the kth attribute. Under the DINA model (Tatsuoka, 1983; Junker and Sijtsma, 2001), the relationship between item responses and attributes is specified by an incidence matrix $Q={\left(q_{\mathrm{jk}}\right)}_{J\times K}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q=( q_{jk}) _{J\times K}$$\end{document} , where $q_{\mathrm{jk}}=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$q_{jk}=1$$\end{document} if the kth attribute is required for answering the jth item correctly, and $q_{\mathrm{jk}}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$q_{jk}=0$$\end{document} otherwise. The ideal response to item j, ${\eta }_j=I(\alpha ⪰q_j)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _{j} = \mathcal {I}(\varvec{\alpha }\succeq \textbf{q}_j)$$\end{document} , is 1 when the examinee masters all attributes required by item j and is 0 otherwise. Under the DINA model, the probability of correct response can be parameterized as a generalized linear model,

(1) $\begin{array}{c}P(Y_j=1\mid \alpha )=h({\delta }_{0j}+{\delta }_{1j}{\eta }_j),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(Y_{j} = 1\mid \varvec{\alpha }) = h(\delta _{0j} + \delta _{1j}\eta _{j}), \end{aligned}$$\end{document}

where $h(·)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$h(\cdot )$$\end{document} is a link function, such as the probit or the logit link.

In the presence of testlets, suppose that the J-item test comprises of S testlets. Let $J_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J_s$$\end{document} denote the number of items in the sth testlet, with ${\sum }_{s=1}^S{J}_s=J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sum _{s=1}^{S}J_s=J$$\end{document} , and $I_s=\{j\mid \text{item}j\text{belongs to testlet}s\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_{s}=\{j\mid \text { item } j \text { belongs to testlet } s\}$$\end{document} denote the set of items in testlet s. Within a testlet, dependency among items often cannot be fully explained by the measured attributes ( $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }$$\end{document} ). As in testlet-effect IRT models (e.g. Bradlow et al., 1999), testlet-effect DCMs assume that items within a testlet additionally measure the same specific factor known as the testlet effect. For an item j belonging to testlet s,

(2) $\begin{array}{c}P(Y_j=1\mid \alpha ,{\zeta }_s)=h({\delta }_{0j}+{\delta }_{1j}{\eta }_j+{\delta }_{\mathrm{sj}}{\zeta }_s).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(Y_{j} = 1\mid \varvec{\alpha }, \zeta _s) = h(\delta _{0j} + \delta _{1j}\eta _{j} + \delta _{sj}\zeta _s). \end{aligned}$$\end{document}

Here, ${\zeta }_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\zeta _s$$\end{document} is the examinee’s testlet-specific trait for testlet s. Inside the linear component, the item parameters, ${\delta }_{0j},{\delta }_{1j}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta _{0j}, \delta _{1j}$$\end{document} , and ${\delta }_{\mathrm{sj}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta _{sj}$$\end{document} , represent the intercept, the slope for mastering all requisite attributes, and the slope for the testlet-specific trait, respectively.

At the population level, similar to Ma et al. (2023), we assume that a randomly chosen examinee’s attribute profile follows a multinomial distribution with proportion parameter vector $\pi ={\left({\pi }_{\alpha }\right)}_{\alpha \in {\{0,1\}}^K}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }=(\pi _{\varvec{\alpha }})_{\varvec{\alpha }\in \{0,1\}^K}$$\end{document} , where ${\sum }_{\alpha \in {\{0,1\}}^K}{\pi }_{\alpha }=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sum _{\varvec{\alpha }\in \{0,1\}^K}\pi _{\varvec{\alpha }}=1$$\end{document} . In general, ${\zeta }_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\zeta _s$$\end{document} , which accounts for the additional dependence in correct response probability for all $j\in I_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j \in \mathcal I_s$$\end{document} yet to be explained by the ideal responses ( ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} s), can still depend on $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }$$\end{document} . To capture this dependency, we decompose the last term in Eq. (2), ${\delta }_{\mathrm{sj}}{\zeta }_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta _{sj}\zeta _{s}$$\end{document} , which controls the expected change in item j’s linear component due to the testlet effect, as follows:

(3) $\begin{array}{c}{\delta }_{\mathrm{sj}}{\zeta }_s=f_{s1j}\left(\alpha \right)+f_{s2j}\left(\alpha \right){\xi }_{s2}+{\beta }_{\mathrm{js}}{\xi }_{s1}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \delta _{sj}\zeta _s = f_{s1j}(\varvec{\alpha }) + f_{s2j}(\varvec{\alpha })\xi _{s2} + \beta _{js}\xi _{s1}. \end{aligned}$$\end{document}

Here, ${\xi }_{s1},{\xi }_{s2}\stackrel{i.i.d.}{\sim }N(0,1)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\xi _{s1}, \xi _{s2} \overset{i.i.d.}{\sim }\ N(0,1)$$\end{document} are two examinee-specific traits on testlet s, and $f_{s1j}(·),f_{s2j}(·)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s1j}(\cdot ), f_{s2j}(\cdot )$$\end{document} are functions of $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }$$\end{document} . Specifically, $f_{s1j}\left(\alpha \right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s1j}(\varvec{\alpha })$$\end{document} must be orthogonal to ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} and describes how $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }$$\end{document} further influences the mean of the testlet effect term ( ${\delta }_{\mathrm{sj}}{\zeta }_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta _{sj}\zeta _s$$\end{document} ), and $f_{s2j}\left(\alpha \right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s2j}(\varvec{\alpha })$$\end{document} describes how $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }$$\end{document} influences the variance of the testlet effect term. The remaining component, ${\beta }_{\mathrm{js}}{\xi }_{s1}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}\xi _{s1}$$\end{document} , is the unique testlet effect unrelated to $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }$$\end{document} . With this parameterization for the distribution of the testlet effect, (2) becomes

(4) $\begin{array}{c}P(Y_j=1\mid \alpha ,{\zeta }_s)=h({\delta }_{0j}+{\delta }_{1j}{\eta }_j+f_{s1j}\left(\alpha \right)+f_{s2j}\left(\alpha \right){\xi }_{s2}+{\beta }_{\mathrm{js}}{\xi }_{s1}).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(Y_{j} = 1\mid \varvec{\alpha }, \zeta _s) = h(\delta _{0j} + \delta _{1j}\eta _{j} + f_{s1j}(\varvec{\alpha }) + f_{s2j}(\varvec{\alpha })\xi _{s2} + \beta _{js}\xi _{s1}). \end{aligned}$$\end{document}

This general formulation is not directly suitable for practical use. We discuss a few special cases.

• • T-DINA: Conditioning on ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} , if the distribution of the testlet-specific term ( ${\delta }_{\mathrm{sj}}{\zeta }_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta _{sj}\zeta _s$$\end{document} ) is independent of $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }$$\end{document} , $f_{s1j}\left(\alpha \right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s1j}(\varvec{\alpha })$$\end{document} and $f_{s2j}\left(\alpha \right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s2j}(\varvec{\alpha })$$\end{document} reduce to 0, and Eq. (4) becomes

  (5) $\begin{array}{c}P(Y_j=1\mid \alpha ,{\zeta }_s)=h({\delta }_{0j}+{\delta }_{1j}{\eta }_j+{\beta }_{\mathrm{js}}{\xi }_{s1}),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(Y_{j} = 1\mid \varvec{\alpha }, \zeta _s) = h(\delta _{0j} + \delta _{1j}\eta _{j} + \beta _{js}\xi _{s1}), \end{aligned}$$\end{document}
  which is the T-DINA model (e.g., Zhan et al., 2015).

• • General testlet-effect DCM: Conditioning on ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} , if the mean of the testlet effect term depends on the requisite skills of j but the variance is the same across attribute classes, then $f_{s2j}\left(\alpha \right)=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s2j}(\varvec{\alpha }) = 0$$\end{document} . Equation (4) reduces to

  (6) $\begin{array}{c}P(Y_j=1\mid \alpha ,{\zeta }_s)=h({\delta }_{0j}+{\delta }_{1j}{\eta }_j+f_{s1j}\left(\alpha \right)+{\beta }_{\mathrm{js}}{\xi }_{s1}),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(Y_{j} = 1\mid \varvec{\alpha }, \zeta _s) = h(\delta _{0j} + \delta _{1j}\eta _{j} + f_{s1j}(\varvec{\alpha }) + \beta _{js}\xi _{s1}), \end{aligned}$$\end{document}
  where $f_{s1j}\left(\alpha \right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s1j}(\varvec{\alpha })$$\end{document} can be written as the main effect and higher-order interactions of select requisite skills. This model is equivalent to the testlet-effect general DCM (Hansen, 2013; Ma et al., 2023). In this case, the additional dependency in responses within a testlet is due to the DINA ideal response, ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} , not being able to fully describe the attribute-response relationship, and adopting a more general DCM resolves this issue.

• • IT-DINA: Conditioning on ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} , if the mean of the testlet effect term does not further depend on $\alpha$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }$$\end{document} , but the variance differs depending on the ideal response ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} , then $f_{s1j}\left(\alpha \right)=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s1j}(\varvec{\alpha }) = 0$$\end{document} , and $f_{s2j}\left(\alpha \right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{s2j}(\varvec{\alpha })$$\end{document} can be parameterized as ${\gamma }_{\mathrm{js}}(2{\eta }_j-1)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}(2\eta _j - 1)$$\end{document} . In this case, Eq. (4) reduces to

  (7) $\begin{array}{c}P(Y_j=1\mid \alpha ,{\zeta }_s)=h({\delta }_{0j}+{\delta }_{1j}{\eta }_j+{\beta }_{\mathrm{js}}{\xi }_{s1}+{\gamma }_{\mathrm{js}}(2{\eta }_j^{\alpha }-1){\xi }_{s2}).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(Y_{j}=1\mid \varvec{\alpha }, \zeta _{s})=h(\delta _{0j}+\delta _{1j}\eta _{j}+\beta _{js}\xi _{s1}+\gamma _{js}(2\eta _{j}^{\varvec{\alpha }}-1)\xi _{s2}). \end{aligned}$$\end{document}
  Adopting the probit link for $h(·)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$h(\cdot )$$\end{document} , we arrive at the proposed IT-DINA model,
  (8) $\begin{array}{c}P(Y_j=1\mid \alpha ,{\xi }_s)=\Phi ({\delta }_{0j}+{\delta }_{1j}{\eta }_j+{\beta }_{\mathrm{js}}{\xi }_{s1}+{\gamma }_{\mathrm{js}}(2{\eta }_j-1){\xi }_{s2}),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(Y_{j}=1\mid \varvec{\alpha }, \varvec{\xi }_{s})=\Phi (\delta _{0j}+\delta _{1j}\eta _{j}+\beta _{js}\xi _{s1}+\gamma _{js}(2\eta _{j}-1)\xi _{s2}), \end{aligned}$$\end{document}
  where $\Phi (·)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Phi (\cdot )$$\end{document} is the standard normal cumulative distribution function, and ${\xi }_s={({\xi }_{s1},{\xi }_{s2})}^{\prime }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\xi }_{s} = (\xi _{s1}, \xi _{s2})^\prime $$\end{document} .

Equation 8 gives our proposed IT-DINA model, which will be the focal scenario considered in the current paper. Let ${\delta }_j=({\delta }_{0j},{\delta }_{1j})$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta }_j=(\delta _{0j},\delta _{1j})$$\end{document} , $\delta ={\left({\delta }_j\right)}_{1\le j\le J}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta }=(\varvec{\delta }_j)_{1\le j \le J}$$\end{document} , ${\beta }_s={\left({\beta }_{\mathrm{js}}\right)}_{j\in I_s}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\beta }_s=(\beta _{js})_{j\in \mathcal {I}_s}$$\end{document} , $\beta ={\left({\beta }_s\right)}_{1\le s\le S}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\beta }=(\varvec{\beta }_s)_{1\le s\le S}$$\end{document} , ${\gamma }_s={\left({\gamma }_{\mathrm{js}}\right)}_{j\in I_s}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\gamma }_s=(\gamma _{js})_{j\in \mathcal {I}_s}$$\end{document} , and $\gamma ={\left({\gamma }_s\right)}_{1\le s\le S}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\gamma }=(\varvec{\gamma }_s)_{1\le s\le S}$$\end{document} . The joint marginal distribution of $Y=(Y_1,Y_2,\cdots ,Y_J)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{Y}=(Y_1,Y_2,\cdots ,Y_J)$$\end{document} can be expressed as

(9) $\begin{array}{c}P(Y=y\mid \delta ,\beta ,\gamma ,\pi )=\sum _{\alpha }{\pi }_{\alpha }\prod _{s=1}^S\int \prod _{j\in I_s}P(Y_j=y_j\mid \alpha ,{\xi }_s,{\delta }_j,{\beta }_{\mathrm{js}},{\gamma }_{\mathrm{js}})p_{\xi }\left({\xi }_s\right)d{\xi }_s,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(\varvec{Y}=\varvec{y}\mid \varvec{\delta },\varvec{\beta },\varvec{\gamma },\varvec{\pi })=\sum _{\varvec{\alpha }}\pi _{\varvec{\alpha }}\prod _{s=1}^{S}\int \prod _{j\in \mathcal {I}_{s}}P(Y_{j}=y_j\mid \varvec{\alpha },\varvec{\xi }_{s},\varvec{\delta }_j,\beta _{js},\gamma _{js})p_{\varvec{\xi }}(\varvec{\xi }_{s})d\varvec{\xi }_{s}, \end{aligned}$$\end{document}

where $p_{\xi }\left({\xi }_s\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{\varvec{\xi }}(\varvec{\xi }_{s})$$\end{document} is the standard bivariate normal density.

Below, we present a few remarks on the IT-DINA model’s interpretation, its relationship to existing models, and an alternative parameterization based on model-implied first and second moments given ideal responses.

Remark 1

(Differential testlet effect for masterers and non-masterers). The IT-DINA model simplifies the attribute-response relationship to involve only the ideal response, ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} , but affords nice interpretations. Without explicitly modeling the structural relationship between item j’s ideal response and the testlet effect ${\zeta }_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\zeta _s$$\end{document} , the interaction term $(2{\eta }_j-1){\xi }_{s2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(2\eta _{j}-1)\xi _{s2}$$\end{document} controls how the testlet-specific effect that enters the measurement model will differ, depending on whether the examinee has mastered all of item j’s requisite skills. We coin this the “differential testlet effect”: When an examinee has all requisite skills, ${\eta }_j=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j =1$$\end{document} , $(2{\eta }_j-1){\xi }_{s2}={\xi }_{s2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(2\eta _{j}-1)\xi _{s2} = \xi _{s2}$$\end{document} , and the combined testlet effect becomes ${\beta }_{\mathrm{js}}{\xi }_{s1}+{\gamma }_{\mathrm{js}}{\xi }_{s2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}\xi _{s1}+\gamma _{js}\xi _{s2}$$\end{document} ; when an examinee misses any requisite skill for item j, ${\eta }_j=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j = 0$$\end{document} , $(2{\eta }_j-1){\xi }_{s2}=-{\xi }_{s_2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(2\eta _{j}-1)\xi _{s2} = -\xi _{s_2}$$\end{document} , and the combined testlet effect becomes ${\beta }_{\mathrm{js}}{\xi }_{s1}-{\gamma }_{\mathrm{js}}{\xi }_{s2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}\xi _{s1}-\gamma _{js}\xi _{s2}$$\end{document} . In other words, the testlet-specific trait that enters the measurement model will differ by $2{\gamma }_{\mathrm{js}}{\xi }_{s2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2\gamma _{js}\xi _{s2}$$\end{document} , depending on the mastery status of the required skills. With the variance of both ${\xi }_{s1},{\xi }_{s2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\xi _{s1}, \xi _{s2}$$\end{document} fixed to 1, the coefficient ${\gamma }_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}$$\end{document} will capture the extent to which the combined testlet effect on item j is dependent on the possession of requisite skills. As a hypothetical example, consider a testlet on an elementary probability assessment, where the items involve the permutations and combinations based on a deck of cards. It is natural to imagine cases where one unit of increase in card game familiarity (the testlet effect) will influence the odds of correct response on an item differently, depending on whether the examinee has the relevant probability knowledge for solving the item.

Remark 2

(Relationship with T-DINA and DINA) When item parameters ${\gamma }_{\mathrm{js}}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}=0$$\end{document} for all j, the measurement model in Eq. (8) reduces to the T-DINA model (e.g., Hansen, 2013; Hansen et al., 2016; Zhan et al., 2015). It further reduces to the DINA model (reparameterized in probit form), when ${\beta }_{\mathrm{js}}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js} = 0$$\end{document} for all j. Hypothesis testing on the coefficients for the testlet main effect ( $\beta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta $$\end{document} s) and interaction ( $\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma $$\end{document} s) offers a natural way for model comparison, via empirically testing the presence of testlet effect and testlet-attribute dependence on an item.

Remark 3

(Differences with the multidimensional T-DINA model) Zhan et al. (2015, 2018) proposed a multidimensional T-DINA model, where each item can load on more than one testlet-specific dimension. While the proposed IT-DINA model also involves each item loading on two testlet-specific dimensions, the two models are inherently different. The multidimensional T-DINA model essentially allows testlet-specific dimensions to go beyond a simple loading structure. The IT-DINA model, other the other hand, still assumes that each item in testlet s only loads on the two-dimensional testlet effects specific to testlet s. The two dimensions here are introduced for the purpose of capturing the differential testlet effects associated with different ideal responses.

Remark 4

( $\eta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta $$\end{document} -implied first moment: DINA slipping and guessing) Under the DINA model, the uncertainty at the item level is captured by the slipping parameter $s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{s}$$\end{document} and guessing parameter $g$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{g}$$\end{document} : for item j, $g_j:=P(Y_j=1\mid {\eta }_j^{\alpha }=0)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g_j:=P(Y_j=1\mid \eta _j^{\varvec{\alpha }}=0)$$\end{document} and $c_j:=1-s_j=P(Y_j=1\mid {\eta }_j^{\alpha }=1)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$c_j:=1-s_j=P(Y_j=1\mid \eta _j^{\varvec{\alpha }}=1)$$\end{document} . The new IT-DINA model can be reparameterized to obtain $c$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{c}$$\end{document} and $g$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{g}$$\end{document} . For item j in testlet s, given the testlet effect ${\xi }_s,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\xi }_{s},$$\end{document} we have

$\begin{array}{cc}P(Y_j& =1\mid {\eta }_j^{\alpha }=1,{\xi }_s,{\delta }_j,{\beta }_{\mathrm{js}},{\gamma }_{\mathrm{js}})=\Phi ({\delta }_{0j}+{\delta }_{1j}+{\beta }_{\mathrm{js}}{\xi }_{s1}+{\gamma }_{\mathrm{js}}{\xi }_{s2}),\\ P(Y_j& =1\mid {\eta }_j^{\alpha }=0,{\xi }_s,{\delta }_j,{\beta }_{\mathrm{js}},{\gamma }_{\mathrm{js}})=\Phi ({\delta }_{0j}+{\beta }_{\mathrm{js}}{\xi }_{s1}-{\gamma }_{\mathrm{js}}{\xi }_{s2}).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P(Y_{j}&=1\mid \eta _{j}^{\varvec{\alpha }}=1,\varvec{\xi }_{s}, \varvec{\delta }_j,\beta _{js},\gamma _{js}) = \Phi (\delta _{0j} + \delta _{1j}+\beta _{js}\xi _{s1}+\gamma _{js}\xi _{s2}),\\ P(Y_{j}&=1\mid \eta _{j}^{\varvec{\alpha }}=0,\varvec{\xi }_{s}, \varvec{\delta }_j,\beta _{js},\gamma _{js}) =\Phi (\delta _{0j}+\beta _{js}\xi _{s1}-\gamma _{js}\xi _{s2}). \end{aligned}$$\end{document}

With the probit link, by integrating out ${\xi }_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\xi }_s$$\end{document} , we get

(10) $\begin{array}{cc}{c}_{\mathrm{js}}& =E_{{\xi }_s}\left(P,(,Y_j,=,1,\mid ,{\eta }_j^{\alpha },=,1,,,{\xi }_s,,,{\delta }_j,,,{\beta }_{\mathrm{js}},,,{\gamma }_{\mathrm{js}},)\right)=\Phi \left(\frac{{\delta }_{0j}+{\delta }_{1j}}{\sqrt{1+{\beta }_{\mathrm{js}}^2+{\gamma }_{\mathrm{js}}^2}}\right),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} c_{js}&=\mathbb {E}_{\varvec{\xi }_s}\left[ P(Y_j=1\mid \eta _{j}^{\varvec{\alpha }}=1,\varvec{\xi }_s,\varvec{\delta }_j,\beta _{js},\gamma _{js})\right] =\Phi \left( \frac{\delta _{0j}+\delta _{1j}}{\sqrt{1+\beta _{js}^2+\gamma _{js}^2}}\right) , \end{aligned}$$\end{document}

(11) $\begin{array}{cc}{g}_{\mathrm{js}}& =E_{{\xi }_s}\left(P,(,Y_j,=,1,\mid ,{\eta }_j^{\alpha },=,0,,,{\xi }_s,,,{\delta }_j,,,{\beta }_{\mathrm{js}},,,{\gamma }_{\mathrm{js}},)\right)=\Phi \left(\frac{{\delta }_{0j}}{\sqrt{1+{\beta }_{\mathrm{js}}^2+{\gamma }_{\mathrm{js}}^2}}\right).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} g_{js}&=\mathbb {E}_{\varvec{\xi }_s}\left[ P(Y_j=1\mid \eta _{j}^{\varvec{\alpha }}=0,\varvec{\xi }_s,\varvec{\delta }_j,\beta _{js},\gamma _{js})\right] =\Phi \left( \frac{\delta _{0j}}{\sqrt{1+\beta _{js}^2+\gamma _{js}^2}}\right) . \end{aligned}$$\end{document}

For the case of ${\gamma }_{\mathrm{js}}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}=0$$\end{document} , $c_{\mathrm{js}}=\Phi \left(({\delta }_{0j}+{\delta }_{1j}),/,\sqrt{1+{\beta }_{\mathrm{js}}^2}\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ c_{js} =\Phi \left( (\delta _{0j}+\delta _{1j}) /\sqrt{1+\beta _{js}^2}\right) $$\end{document} and $g_{\mathrm{js}}=\Phi \left({\delta }_{0j},/,\sqrt{1+{\beta }_{\mathrm{js}}^2}\right),$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ g_{js} =\Phi \left( \delta _{0j}/\sqrt{1+\beta _{js}^2}\right) , $$\end{document} which, when ${\beta }_{\mathrm{js}}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}=0$$\end{document} , further reduce to $c_j=\Phi ({\delta }_{0j}+{\delta }_{1j})$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ c_j =\Phi (\delta _{0j}+\delta _{1j})$$\end{document} and $g_j=\Phi \left({\delta }_{0j}\right).$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g_j =\Phi (\delta _{0j}). $$\end{document}

Remark 5

( $\eta -$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta -$$\end{document} implied second moments) Following Fang et al. (2021), we can get the joint marginal probability of correct responses for a pair of items $j_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j_1$$\end{document} and $j_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j_2$$\end{document} in testlet s,

(12) $\begin{array}{cc}& P(Y_{j_1}=1,Y_{j_2}=1\mid {\delta }_s,{\beta }_s,{\gamma }_s,\pi )\\ & =\sum _{\alpha }{\pi }_{\alpha }{\Phi }_2\left({\Phi }^{-1},\left(c_{j_1}^{{\eta }_{j_1}^{\alpha }},g_{j_1}^{\left(1,-,{\eta }_{j_1}^{\alpha }\right)}\right),,,{\Phi }^{-1},\left(c_{j_2}^{{\eta }_{j_2}^{\alpha }},g_{j_2}^{\left(1,-,{\eta }_{j_2}^{\alpha }\right)}\right),;,{\rho }_{j_1{j}_{2}s}^{\alpha }\right),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned}{} & {} P(Y_{j_1}=1,Y_{j_2}=1\mid \varvec{\delta }_s,\varvec{\beta }_s,\varvec{\gamma }_s,\varvec{\pi }) \nonumber \\{} & {} \quad =\sum _{\varvec{\alpha }}\pi _{\varvec{\alpha }}\Phi _2 \left( \Phi ^{-1}\left( c_{j_1}^{\eta _{j_1}^{\varvec{\alpha }}} g_{j_1}^{\left( 1-\eta _{j_1}^{\varvec{\alpha }}\right) }\right) ,\Phi ^{-1}\left( c_{j_2}^{\eta _{j_2}^{\varvec{\alpha }}} g_{j_2}^{\left( 1-\eta _{j_2}^{\varvec{\alpha }}\right) }\right) ; \rho _{j_1j_2 s}^{\varvec{\alpha }}\right) , \end{aligned}$$\end{document}

where

$\begin{array}{c}{\rho }_{j_1{j}_{2}s}^{\alpha }=\frac{{\beta }_{j_{1}s}{\beta }_{j_{2}s}+{\gamma }_{j_{1}s}{\gamma }_{j_{2}s}(2{\eta }_{j_1}^{\alpha }-1)(2{\eta }_{j_2}^{\alpha }-1)}{\sqrt{(1+{\beta }_{j_{1}s}^2+{\gamma }_{j_{1}s}^2)(1+{\beta }_{j_{2}s}^2+{\gamma }_{j_{2}s}^2)}},\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \rho _{j_1j_2s}^{\varvec{\alpha }}= \frac{\beta _{j_1s}\beta _{j_2s}+\gamma _{j_1s}\gamma _{j_2s}(2\eta _{j_1}^{\varvec{\alpha }}-1)(2\eta _{j_2}^{\varvec{\alpha }}-1)}{\sqrt{(1+\beta _{j_{1}s}^2+\gamma _{j_{1}s}^2)(1+\beta _{j_{2}s}^2+\gamma _{j_{2}s}^2)}}, \end{aligned}$$\end{document}

${\delta }_s=\delta [I_s,:]$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta }_{s}=\varvec{\delta }[\mathcal {I}_s,:]$$\end{document} denotes the submatrix of matrix $\delta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta }$$\end{document} consisting only of rows in the set $I_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_s$$\end{document} , and ${\Phi }_2(a,b;\rho ):=P(X_1\le a,X_2\le b)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Phi _2(a,b; \rho ):=P(X_1\le a, X_2\le b)$$\end{document} where $X_1,X_2\sim N(0,1)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_1, X_2\sim N(0,1)$$\end{document} , and $\text{corr}(X_1,X_2)=\rho$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{corr}(X_1,X_2)=\rho $$\end{document} . Typically, a, b, and $\rho$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho $$\end{document} are referred to as the thresholds and the tetrachoric correlation.

Because ${\eta }_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _j$$\end{document} is either 0 or 1, ${\rho }_{j_1{j}_{2}s}^{\alpha }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho _{j_1j_2s}^{\varvec{\alpha }}$$\end{document} can only take two values,

$\begin{array}{c}{\rho }_{j_1{j}_2}^1=\frac{{\beta }_{j_{1}s}{\beta }_{j_{2}s}+{\gamma }_{j_{1}s}{\gamma }_{j_{2}s}}{\sqrt{(1+{\beta }_{j_{1}s}^2+{\gamma }_{j_{1}s}^2)(1+{\beta }_{j_{2}s}^2+{\gamma }_{j_{2}s}^2)}},\text{and}{\rho }_{j_1{j}_2}^2=\frac{{\beta }_{j_{1}s}{\beta }_{j_{2}s}-{\gamma }_{j_{1}s}{\gamma }_{j_{2}s}}{\sqrt{(1+{\beta }_{j_{1}s}^2+{\gamma }_{j_{1}s}^2)(1+{\beta }_{j_{2}s}^2+{\gamma }_{j_{2}s}^2)}}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \rho _{{j_1j_2}}^{1}=\frac{\beta _{j_1s}\beta _{j_2s}+\gamma _{j_1s}\gamma _{j_2s}}{\sqrt{(1+\beta _{j_{1}s}^2+\gamma _{j_{1}s}^2)(1+\beta _{j_{2}s}^2+\gamma _{j_{2}s}^2)}}, \text { and }\rho _{{j_1j_2}}^{2}=\frac{\beta _{j_1s}\beta _{j_2s}-\gamma _{j_1s}\gamma _{j_2s}}{\sqrt{(1+\beta _{j_{1}s}^2+\gamma _{j_{1}s}^2)(1+\beta _{j_{2}s}^2+\gamma _{j_{2}s}^2)}}. \end{aligned}$$\end{document}

For a more concise presentation, we let ${\rho }_s^1={\left({\rho }_{j_1{j}_{2}s}^1\right)}_{j_1<j_2,j_1,j_2\in I_s}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\rho }_s^1=(\rho _{j_1 j_2 s}^1)_{j_1<j_2, j_1,j_2\in \mathcal {I}_s}$$\end{document} and ${\rho }_s^2={\left({\rho }_{j_1{j}_{2}s}^2\right)}_{j_1<j_2,j_1,j_2\in I_s}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\rho }_s^2=(\rho _{j_1 j_2 s}^2)_{j_1<j_2, j_1,j_2\in \mathcal {I}_s}$$\end{document} , and further let $\rho ={({\rho }_s^1,{\rho }_s^2)}_{s=1,\dots ,S}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\rho }=(\varvec{\rho }_s^1,\varvec{\rho }_s^2)_{s=1,\ldots ,S}$$\end{document} denote the collection of all the tetrachoric correlation parameter matrices.

2. Identifiability

In this section, we develop the theoretical results for the identifiability of the proposed IT-DINA model. Let $P:=\{(\delta ,\beta ,\gamma ,\pi )\mid {\delta }_{1j}>0,{\pi }_{\alpha }>0,{\sum }_{\alpha }{\pi }_{\alpha }=1\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {P}:=\{(\varvec{\delta },\varvec{\beta },\varvec{\gamma },\varvec{\pi })\mid \delta _{1j}>0, \pi _{\varvec{\alpha }}>0, \sum _{\varvec{\alpha }}\pi _{\varvec{\alpha }} = 1\}$$\end{document} denote the parameter space. We also introduce its reparameterized version $P^{\left(r\right)}=\{(c,g,\rho ,\pi )\mid c_{\mathrm{js}}>g_{\mathrm{js}},{\pi }_{\alpha }>0,{\sum }_{\alpha }{\pi }_{\alpha }=1\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {P}^{(r)}=\{(\varvec{ c}, \varvec{ g}, \varvec{\rho },\varvec{\pi })\mid c_{js}>g_{js}, \pi _{\varvec{\alpha }}>0, \sum _{\varvec{\alpha }}\pi _{\varvec{\alpha }} = 1\}$$\end{document} ( ${\delta }_{1j}>0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta _{1j}>0$$\end{document} implies $c_{\mathrm{js}}>g_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$c_{js}>g_{js}$$\end{document} ).

2.1. Definition of Identifiability

We introduce two definitions of identifiability, strict identifiability (Definition 1) and ( $\delta ,\beta ,\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta },\varvec{\beta }, \varvec{\gamma }$$\end{document} )-partial identifiability (Definition 2).

Definition 1

(strict identifiability) We say the IT-DINA is identifiable at $(\delta ,\beta ,\gamma ,\pi )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{ \delta }, \varvec{ \beta }, \varvec{\gamma },\varvec{\pi })$$\end{document} if the following holds: If there exists another set of parameters $(\overline{\delta },\overline{\beta },\overline{\gamma },\overline{\pi })\in P$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{ \bar{\delta }}, \varvec{ \bar{\beta }}, \varvec{\bar{\gamma }},\varvec{\bar{\pi }}) \in \mathcal {P}$$\end{document} that defines the same probability distribution in Eq. (9), it must hold that $\left(\right|\delta |,|\beta |,|\gamma |,|\pi \left|\right)=\left(\right|\overline{\delta }|,|\overline{\beta }|,|\overline{\gamma }|,|\overline{\pi }\left|\right).$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(|\varvec{ \delta }|, |\varvec{ \beta }|, |\varvec{\gamma }|,|\varvec{\pi }|)=(|\varvec{ \bar{\delta }}|, |\varvec{ \bar{\beta }}|, |\varvec{\bar{\gamma }}|,|\varvec{\bar{\pi }}|).$$\end{document}

Definition 2

(( $\delta ,\beta ,\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta },\varvec{\beta },\varvec{\gamma }$$\end{document} )-partial identifiability) For the IT-DINA, the model parameters $(\delta ,\beta ,\gamma ,\pi )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{ \delta }, \varvec{ \beta }, \varvec{\gamma },\varvec{\pi })$$\end{document} are said to be ( $\delta ,\beta ,\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta },\varvec{\beta }, \varvec{\gamma }$$\end{document} )-partially identifiable if ( $c,g,\rho ,\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{c}, \varvec{g}, \varvec{\rho }, \varvec{\pi }$$\end{document} ) are identifiable.

Definition 2 is weaker than Definition 1, i.e., the strict identifiability of ( $\delta ,\beta ,\gamma ,\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta },\varvec{\beta }, \varvec{\gamma }, \varvec{\pi }$$\end{document} ) implies ( $\delta ,\beta ,\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta },\varvec{\beta }, \varvec{\gamma }$$\end{document} )-partial identifiability. Nevertheless, ( $\delta ,\beta ,\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta },\varvec{\beta }, \varvec{\gamma }$$\end{document} )-partial identifiability can uniquely determine parameters $(c,g,\rho )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{ c}, \varvec{ g}, \varvec{\rho })$$\end{document} but can fail to guarantee the strict identifiability. Examples in the appendix provide an illustration of models whose $(\delta ,\beta ,\gamma )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{ \delta }, \varvec{ \beta }, \varvec{\gamma })$$\end{document} is non-identifiable, while $(c,g,\rho )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{ c}, \varvec{ g}, \varvec{\rho })$$\end{document} can be identified.

2.2. Main Results

For reader convenience, we now first summarize our main theoretical results.

• • Theorem 1 provides checkable sufficient conditions for the $(\delta ,\beta ,\gamma )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{\delta },\varvec{\beta }, \varvec{\gamma })$$\end{document} -partial identifiability.

• • Theorem 2 gives the conditions which guarantee strict identifiability.

• • Proposition 1, essentially a byproduct of Theorem 2, provides sufficient conditions for the identifiability of the independent testlet DINA model.

We require the following two conditions.

1. C1 After row permutations, there exist $S_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S_0$$\end{document} testlets whose $Q_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document} -matrix can be partitioned into a $K\times K$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K\times K$$\end{document} identity matrix $I_K$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I_K$$\end{document} and a matrix $Q^{\prime }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q'$$\end{document} , where $Q^{\prime }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q'$$\end{document} is a $(J_s-K)\times K$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(J_s-K)\times K$$\end{document} matrix, for $s=1,2,\cdots ,S_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s=1,2,\dots ,S_0$$\end{document} . In other words, the $Q_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document} submatrix is complete (Chiu et al., 2009) in $S_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S_0$$\end{document} testlets.

2. C2 The number $S_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S_0$$\end{document} in C1 is no smaller than $max\{K,3\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\max \{K,3\}$$\end{document} .

Theorem 1

Let $Q_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document} denote the $J_s\times K$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J_s \times K$$\end{document} submatrix of Q that corresponds to items in testlet s. Under the model specified in Eq. (8), Conditions C1 and C2 are sufficient for $(\delta ,\beta ,\gamma )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{\delta },\varvec{\beta }, \varvec{\gamma })$$\end{document} -partial identifiability of $(\delta ,\beta ,\gamma ,\pi )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{\delta },\varvec{\beta }, \varvec{\gamma }, \varvec{\pi })$$\end{document} .

In addition to the first two conditions, we further impose the following requirement.

1. C3 For each testlet $s=1,2,\cdots ,S$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s=1,2,\cdots ,S$$\end{document} , there exist at least three items with nonzero $\beta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta $$\end{document} parameters, and at least three items with nonzero $\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma $$\end{document} parameters. The two sets of three items are allowed to differ.

By meeting all three conditions above, the following Theorem 2 provides sufficient conditions for strict identifiability.

Theorem 2

Under the model specified in Eq. (8), if conditions C1, C2 and C3 are satisfied, then the parameters $(\delta ,\beta ,\gamma ,\pi )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{ \delta }, \varvec{ \beta }, \varvec{\gamma },\varvec{\pi })$$\end{document} are identifiable.

The T-DINA model is a special case of the proposed model, where ${\gamma }_{\mathrm{js}}\equiv 0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js} \equiv 0$$\end{document} for all s and all j. By modification of Condition C3, we can achieve the strict identifiability of the T-DINA model. This result provides the first-ever sufficient conditions for the identifiability of T-DINA in the literature.

1. C4 For testlet s, $s=1,2,\cdots ,S$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s=1,2,\cdots ,S$$\end{document} , there exist at least three items such that their $\beta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta $$\end{document} parameters are nonzero.

Proposition 1

For the standard T-DINA model specified in Eq. (8) with ${\gamma }_{\mathrm{js}}\equiv 0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js} \equiv 0$$\end{document} , Conditions C1, C2, and C4 are sufficient for the identifiability of $(\delta ,\beta ,\gamma ,\pi )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{ \delta }, \varvec{ \beta }, \varvec{\gamma },\varvec{\pi })$$\end{document} .

Remark 6

(Extension to DINO model) The result presented in Theorems 1 can be directly applied to the DINO-based model by the mathematical duality between the DINA and DINO models (Proposition 1 in Chen et al., 2015; Köhn & Chiu, 2016). Additionally, Theorem 2 and Proposition 1 can also be extended directly to DINO-based models as the $(\delta ,\beta ,\gamma )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{\delta },\varvec{\beta }, \varvec{\gamma })$$\end{document} -partial identifiability of $(\delta ,\beta ,\gamma ,\pi )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{\delta },\varvec{\beta }, \varvec{\gamma }, \varvec{\pi })$$\end{document} .

2.3. Technical Discussions

There are some technical issues that need further clarification. In the remarks below, we provide discussions on some of the issues.

Remark 7

(Identifiability up to sign changes) By the definition of parameters $(\delta ,\beta ,\gamma ,\pi )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{\delta }, \varvec{\beta },\varvec{\gamma }, \varvec{\pi })$$\end{document} , the identifiability holds up to sign changes to the slope coefficients within a testlet, particularly, ${\beta }_s,{\gamma }_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\beta }_s, \varvec{\gamma }_s$$\end{document} . For example, we consider ${\beta }_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{j s}$$\end{document} identifiable if ${\beta }_{\mathrm{js}}/{\overline{\beta }}_{\mathrm{js}}=\pm 1,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${ \beta _{j s}}/{\bar{\beta }_{j s}} = \pm 1,$$\end{document} for item j.

Remark 8

(DINA and T-DINA identifiability) Theorem 1 implies the strict identifiability of the DINA model. The Q-matrix in Theorem 1 satisfies the conditions in Gu and Xu (2020); Gu (2020); Gu and Xu (2019, 2022) for identifying the slipping, guessing, and latent membership parameters. However, the proposed conditions are stricter because Theorem 1 aims at identifying a more general IT-DINA model. Further, for a T-DINA model, a complete characterization of identifiability is not yet available to our knowledge. Theorem 1 can provide the partial identifiability of a T-DINA model and Proposition 1 implies its strict identifiability. However, under a T-DINA model, because of the additional restriction ${\rho }_{j_1{j}_{2}s}^1={\rho }_{j_1{j}_{2}s}^2={\beta }_{j_{1}s}{\beta }_{j_{2}s}/\sqrt{(1+{\beta }_{j_{1}s}^2)(1+{\beta }_{j_{2}s}^2)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho _{{j_1j_2 s}}^{1}=\rho _{{j_1j_2 s}}^{2}=\beta _{j_1s}\beta _{j_2s}/\sqrt{(1+\beta _{j_{1}s}^2)(1+\beta _{j_{2}s}^2)}$$\end{document} , there might exist more relaxed identifiability conditions.

Remark 9

Condition C1 states that the Q-matrix, after swapping of testlets, can be partitioned into matrices $Q_1,\cdots ,Q_{S_0}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_{1}, \dots , Q_{S_0}$$\end{document} and $Q^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q^{*}$$\end{document} where $Q^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q^{*}$$\end{document} is a $(J-{\sum }_{s=1}^{S_0}{J}_s)\times K$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(J-\sum _{s=1}^{S_0}J_s)\times K$$\end{document} matrix. And for $s=1,2,\cdots ,S_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s=1,2,\cdots ,S_0$$\end{document} , $Q_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document} contains an identity matrix $I_K$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I_K$$\end{document} .

Remark 10

For a given testlet s, Condition C3 states, after swapping indices of items, $\beta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta $$\end{document} and $\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma $$\end{document} parameters can be expressed as follows,

$\begin{array}{c}({\beta }_{j_{1}s},{\beta }_{j_{2}s},{\beta }_{j_{3}s},{\beta }_{j_{4}s},\cdots ),({\gamma }_{l_{1}s},{\gamma }_{l_{2}s},{\gamma }_{l_{3}s},{\gamma }_{l_{4}s},\cdots )\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} (\beta _{j_1 s},\beta _{j_2 s},\beta _{j_3 s},\beta _{j_4 s},\dots ), (\gamma _{l_1 s},\gamma _{l_2 s},\gamma _{l_3 s},\gamma _{l_4 s},\dots ) \end{aligned}$$\end{document}

where ${\beta }_{j_{1}s},{\beta }_{j_{2}s},{\beta }_{j_{3}s}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{j_1 s},\beta _{j_2 s},\beta _{j_3 s}$$\end{document} and ${\gamma }_{l_{1}s},{\gamma }_{l_{2}s},{\gamma }_{l_{3}s}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{l_1 s},\gamma _{l_2 s},\gamma _{l_3 s}$$\end{document} are all nonzero. Note that $\{j_1,j_2,j_3\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{j_1,j_2,j_3\}$$\end{document} and $\{l_1,l_2,l_3\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{l_1,l_2,l_3\}$$\end{document} are subsets of $I_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_s$$\end{document} , which can either intersect or be disjoint.

Remark 11

Since $J_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J_s$$\end{document} can be as small as one, the result here covers the situation in which some of the testlets may contain a single item.

Remark 12

The probit model assumption here is indeed crucial in developing our theory. It allows us to compute the tetrachoric correlation so that the explicit formula of the joint probability of item responses, i.e. (12), can be obtained. Whether other types of link functions can provide such mathematical convenience is worth future study.

Remark 13

Theorems 1–2 guarantee the identifiability of model parameters when the Q-matrix remains fixed. Since Q-matrix is usually unknown in real applications, a common practice is to estimate the Q-matrix in advance from the data. Therefore, it is important and challenging to identify the conditions for the identifiability of Q-matrix. One possible solution is to follow the techniques in Gu and Xu (2021) where they developed necessary and sufficient conditions of identifiability of Q-matrix for DINA model. We leave this to future work.

2.4. Examples

To help readers to gain intuitions of the identifiability conditions, we end this section with two illustrative examples.

Example 1

Consider the IT-DINA model with $K=2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=2$$\end{document} attributes and $S=3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S=3$$\end{document} testlets, with the Q-matrix given by Table 1, and $I_1=\{1,2,3\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_1=\{1,2,3\}$$\end{document} , $I_2=\{4,5,6\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_2=\{4,5,6\}$$\end{document} , and $I_3=\{7,8,9\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_3=\{7,8,9\}$$\end{document} . Assume ${\beta }_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}$$\end{document} and ${\gamma }_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}$$\end{document} are nonzero, for $j=1,\cdots ,9$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j=1,\dots ,9$$\end{document} , $s=1,2,3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s=1,2,3$$\end{document} .

For each testlet, there exists an identity matrix in the corresponding $Q_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document} -matrix. Additionally, there are three items in each testlet. Thus Conditions C1, C2 and C3 are satisfied. By Theorem 2, we know that the model is identifiable.

Table 1 Q-matrix in Example 1.

Example 2

Next we consider a model with $K=4$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=4$$\end{document} and $S=4$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S=4$$\end{document} , with the Q-matrix given by Table 2, and $I_1=\{1,2,3,4,5,6\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_1=\{1,2,3,4,5,6\}$$\end{document} , $I_2=\{7,8,9,10,11\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_2=\{7,8,9,10,11\}$$\end{document} , $I_3=\{12,13,14,15,16\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_3=\{12,13,14,15,16\}$$\end{document} , and $I_4=\{17,18,19,20,21\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {I}_4=\{17,18,19,20,21\}$$\end{document} . Note that items 22–24 are single items. Assume ${\beta }_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}$$\end{document} and ${\gamma }_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}$$\end{document} are nonzero, for $j=1,\cdots ,21$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j=1,\dots ,21$$\end{document} , $s=1,\cdots ,4$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s=1,\dots ,4$$\end{document} .

It can be seen that for each testlet, there exists an identity matrix in $Q_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document} . There also exist at least three items in each testlet, and $S_0\ge min\{K,3\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S_0 \ge \min \{K,3\}$$\end{document} . By Theorem 2, we know that this model is also identifiable.

Table 2 Q-matrix in Example 2.

3. Applications

In this section, we apply the proposed IT-DINA model (model (8)), the T-DINA model (model (5)), and the DINA model (model (1)) to a dataset from the 2015 PISA. We conduct hypothesis testing for the existence of possible dependency between testlet effects. We also provide parameter estimation for the attribute profiles.

3.1. Data

We fitted the models to the data from the 2015 computer-based PISA collaborative problem-solving (CPS) test. Four of five testlets in main survey cluster 1 were selected. Within the framework of the 2015 PISA survey, CPS skills were represented by a cross matrix of three collaboration competencies and four core problem-solving competencies (OECD, 2016). Yavuz and Atar (2020) presented three core competencies that could provide the best model fit. Therefore, we take the assignment of the three competencies as attributes in this paper. $K=3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=3$$\end{document} attributes (i.e., CPS competencies) were measured, specifically, ${\alpha }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha _1$$\end{document} : establishing and maintaining shared understanding; ${\alpha }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha _2$$\end{document} : taking appropriate action to solve the problem; ${\alpha }_3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha _3$$\end{document} : establishing and maintaining team organization. This resulted in a total of $J=28$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J=28$$\end{document} items grouped in $S=4$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S=4$$\end{document} testlets. The corresponding Q-matrix and structure of testlets are given in Table 3. It could be seen that testlets 2, 3, and 4 contained complete $Q_s$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document} matrices. This empirical example hence met the identifiability conditions for Theorem 1. To evaluate the appropriateness of retrofitting a DCM to this data set, following the approach of de La Torre and Karelitz (2009), we have calculated item diagnosticity and found that almost all the items exhibited typical/high diagnosticity. Therefore, we fitted DCMs on PISA CPS data in an attempt to extract diagnostic information.

Table 3 The Q-matrix in PISA data and the testlet structure. $\ast$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$*$$\end{document} indicates which testlet an item belongs to.

For the current study, test takers were chosen from all OECD countries and regions where the English version of the exam was administered. For simplicity, all polytomous responses were recoded to dichotomous scores based on whether the highest score was received. Examinees with missingness were removed, resulting in a total of $N=8880$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=8880$$\end{document} examinees.

3.2. Methods and Results

To assess the goodness of fit, the likelihood ratio test (LRT), the Akaike information criterion (AIC), and the Bayesian information criterion (BIC) were used. For the LRT, we use the usual ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} approximation for the log-likelihood ratio statistic. The AIC and BIC are given by

$\begin{array}{cc}\text{AIC}=& -2log\left(\hat{L}\right)+2p,\\ \text{BIC}=& -2log\left(\hat{L}\right)+plog\left(N\right),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \textrm{AIC}= & {} -2\log (\hat{L})+2p, \\ \textrm{BIC}= & {} -2\log (\hat{L})+p\log {(N)}, \end{aligned}$$\end{document}

where p denotes the number of model parameters, and N the sample size.

Table 4 presents the results of the LRT. It shows that, for DINA versus T-DINA, the DINA should be rejected in favor of T-DINA as the corresponding p-value was much less than 0.05. Furthermore, for T-DINA versus the proposed IT-DINA, T-DINA should be rejected. Using AIC and BIC, we also conclude that IT-DINA provided the best fit as shown by Table 5.

Table 4 Results on the LRT on the PISA data. $H_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$H_0$$\end{document} is the null hypothesis, $H_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$H_1$$\end{document} is the alternative hypothesis, ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^{2}$$\end{document} denotes the values of the LRT statistic and df denotes the degrees of freedom of the LRT.

Table 5 Results on goodness of fit for PISA data.

Tables 6 and 7 provide the parameter estimates under the IT-DINA and T-DINA, respectively. The interaction on some items was much higher, such as ${\gamma }_{25,4}=0.768$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{25,4}=0.768$$\end{document} , while the corresponding ${\beta }_{25,4}=-0.078$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{25,4}=-0.078$$\end{document} . However, under the T-DINA model, ${\beta }_{25,4}=0.534$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{25,4}=0.534$$\end{document} which was higher than that under the proposed model. For classes ${\eta }_{25}^{\alpha }=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _{25}^{\varvec{\alpha }}=1$$\end{document} and ${\eta }_{25}^{\alpha }=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta _{25}^{\varvec{\alpha }}=0$$\end{document} , the impact of testlet for item 25 is mainly reflected through ${\xi }_{4,2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\xi _{4,2}$$\end{document} not ${\xi }_{4,1}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\xi _{4,1}$$\end{document} .

Table 6 Parameter estimates under the proposed IT-DINA model for PISA data. For $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pi $$\end{document} s, the order is ${\pi }_{(0,0,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(0,0,0)}$$\end{document} , ${\pi }_{(1,0,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(1,0,0)}$$\end{document} , ${\pi }_{(0,1,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(0,1,0)}$$\end{document} , ${\pi }_{(0,0,1)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(0,0,1)}$$\end{document} , ${\pi }_{(1,1,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(1,1,0)}$$\end{document} , ${\pi }_{(1,0,1)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(1,0,1)}$$\end{document} , ${\pi }_{(0,1,1)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(0,1,1)}$$\end{document} , ${\pi }_{(1,1,1)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(1,1,1)}$$\end{document} .

Table 7 Parameter estimates under the T-DINA model for PISA data. For $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pi $$\end{document} s, the order is ${\pi }_{(0,0,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(0,0,0)}$$\end{document} , ${\pi }_{(1,0,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(1,0,0)}$$\end{document} , ${\pi }_{(0,1,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(0,1,0)}$$\end{document} , ${\pi }_{(0,0,1)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(0,0,1)}$$\end{document} , ${\pi }_{(1,1,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(1,1,0)}$$\end{document} , ${\pi }_{(1,0,1)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(1,0,1)}$$\end{document} , ${\pi }_{(0,1,1)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(0,1,1)}$$\end{document} , ${\pi }_{(1,1,1)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }_{(1,1,1)}$$\end{document} .

Tables 8 and 9 display the frequencies of latent classes across various models estimated using expected a posteriori (EAP). Both the proposed IT-DINA and T-DINA models modified the classification of examinees when compared to DINA model. For instance, 65 and 84 students with class (0, 0, 0) in the DINA model were instead classified into class (1, 1, 1). One explanation is that the DINA model could potentially classify examinees who possess all the required attributes but are low on the context-based stimuli as non-masterers. However, the IT-DINA and the T-DINA model incorporated additional testlet effects, resulting in adjustments for these students. A similar trend was observed in other classes.

Furthermore, upon comparing Tables 8 and 9, differences were found in the attribute classification under the IT-DINA and the T-DINA. This discrepancy arose because the T-DINA model and the IT-DINA model imply different levels of the testlet effects for the participants, which could in turn shift the attribute classifications.

Table 8 Tabulation of attribute classification based on DINA and IT-DINA models on PISA data.

Table 9 Tabulation of attribute classification based on DINA and T-DINA models on PISA data.

4. Simulation Study

This section provides a simulation study that evaluates parameter recovery under the proposed testlet DINA model and the consequences of ignoring the testlet and the interaction effects.

4.1. Design

The Q-matrix and testlet structures are specified by Tables 3 and 10 for $K=3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=3$$\end{document} and 5, respectively. In both cases, the identifiability conditions in Theorem 1 are met. Sample sizes of $N=500$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=500$$\end{document} , 1000, 2000 and 4000 were considered. The data were generated according to the following specifications. Five parameter settings were considered. Cases 1–3 considered different scenarios where the true data generating process was the IT-DINA, and cases 4 and 5 consider scenarios where the DINA and the T-DINA were the true data-generating models, respectively.

• • Case 1: ${\gamma }_{\mathrm{js}}=0.5$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}=0.5$$\end{document} , ${\beta }_{\mathrm{js}}=0.5$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}=0.5$$\end{document} , $K=3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=3$$\end{document} , and attribute profiles were generated from a discrete uniform distribution, i.e., $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} was set to $\frac{1}{8}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\frac{1}{8}$$\end{document} for all $\alpha \in {\{0,1\}}^3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }\in \{0,1\}^3$$\end{document} .

• • Case 2: ${\gamma }_{\mathrm{js}}=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}=1$$\end{document} , ${\beta }_{\mathrm{js}}=0.5$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}=0.5$$\end{document} , $K=3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=3$$\end{document} , and $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} was set to $\frac{1}{8}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\frac{1}{8}$$\end{document} for all $\alpha \in {\{0,1\}}^3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }\in \{0,1\}^3$$\end{document} .

• • Case 3: Each testlet contains two ${\gamma }_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}$$\end{document} and two ${\beta }_{\mathrm{js}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}$$\end{document} with negative values, and $|{\gamma }_{\mathrm{js}}|=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$|\gamma _{js}|=1$$\end{document} , $|{\beta }_{\mathrm{js}}|=0.5$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$|\beta _{js}|=0.5$$\end{document} , $K=5$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=5$$\end{document} , and $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} was set as an non-uniform distribution following Culpepper (2015) in which the probability of belonging to a class with three or more skills is twice as likely as belonging to a class with zero, one, or two skills. That is, ${\pi }_c=0.021$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pi _c=0.021$$\end{document} for all ${\alpha }_c$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }_c$$\end{document} with two or fewer skills and ${\pi }_c=0.042$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pi _c=0.042$$\end{document} for ${\alpha }_c$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }_c$$\end{document} with three or more skills.

• • Case 4: ${\gamma }_{\mathrm{js}}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}=0$$\end{document} , ${\beta }_{\mathrm{js}}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}=0$$\end{document} , $K=3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=3$$\end{document} , and $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} was set to $\frac{1}{8}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\frac{1}{8}$$\end{document} for all $\alpha \in {\{0,1\}}^3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }\in \{0,1\}^3$$\end{document} .

• • Case 5: ${\gamma }_{\mathrm{js}}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _{js}=0$$\end{document} , ${\beta }_{\mathrm{js}}=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _{js}=1$$\end{document} , $K=3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=3$$\end{document} , and $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} was set to $\frac{1}{8}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\frac{1}{8}$$\end{document} for all $\alpha \in {\{0,1\}}^3$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\alpha }\in \{0,1\}^3$$\end{document} .

• • For more details on other parameters, please refer to Tables 1–3 in Appendix.

Table 10 The Q-matrix for $K=5$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$K=5$$\end{document} in the simulation study and the testlet structure. $\ast$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$*$$\end{document} indicates which testlet an item belongs to.

Each condition was replicated for $T=500$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T = 500$$\end{document} times. To evaluate performance of the proposed model, we first consider recovery of the original parameters $(\delta ,\beta ,\gamma )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{\delta }, \varvec{\beta }, \varvec{\gamma })$$\end{document} . To compare the performances of DINA, T-DINA and IT-DINA, recovery of transformed parameters $(s,g)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{s}, \varvec{g})$$\end{document} was evaluated. Recovery was evaluated in terms of average bias (BIAS), root-mean-squared error (RMSE), standard error (SE) and BIAS ratio (BR) (Kish et al., 1962; Meng, 1993; Coelho and Pereira, 2011) for each parameter, across all the replications. For instance, for the slipping parameter $s_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_j$$\end{document} , denote its estimate from the tth replication by ${\hat{s}}_j^{\left(t\right)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{s}_{j}^{(t)}$$\end{document} , then

$\begin{array}{cc}\text{BIAS}\left({\hat{s}}_j\right)=& \frac{1}{T}\sum _{t=1}^T({\hat{s}}_j^{\left(t\right)}-s_j),\\ \text{RMSE}\left({\hat{s}}_j\right)=& \sqrt{\frac{1}{T}\sum _{t=1}^T{({\hat{s}}_j^{\left(t\right)}-s_j)}^2},\\ \text{SE}\left({\hat{s}}_j\right)=& \sqrt{\frac{1}{T}\sum _{t=1}^T{\left({\hat{s}}_j^{\left(t\right)},-,\frac{1}{T},\sum _{t=1}^T,{\hat{s}}_j^{\left(t\right)}\right)}^2},\\ \text{BR}\left({\hat{s}}_j\right)=& \frac{\text{BIAS}\left({\hat{s}}_j\right)}{\text{SE}\left({\hat{s}}_j\right)}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \text{ BIAS }(\hat{s}_{j})= & {} \frac{1}{T}\sum _{t = 1}^{T}(\hat{s}_{j}^{(t)}- s_{j}),\\ \text{ RMSE }(\hat{s}_{j})= & {} \sqrt{\frac{1}{T}\sum _{t= 1}^{T}(\hat{s}_{j}^{(t)} - s_{j})^2},\\ \text{ SE }(\hat{s}_{j})= & {} \sqrt{\frac{1}{T}\sum _{t = 1}^{T}\left( \hat{s}_{j}^{(t)} - \frac{1}{T}\sum _{t = 1}^{T}\hat{s}_{j}^{(t)} \right) ^2},\\ \text{ BR }(\hat{s}_{j})= & {} \frac{\text{ BIAS }(\hat{s}_{j})}{\text{ SE }(\hat{s}_{j})}. \end{aligned}$$\end{document}

Table 11 The average BIAS and RMSE of item parameters under the proposed testlet DINA model.

Figure 1 BIAS, RMSE, and SE of slipping parameters under Cases 1 and 2 when $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . X-axis is the Q-matrix, where a black square denotes the requirement of the attribute for an item.

Figure 2 BIAS, RMSE, and SE of guessing parameters under Cases 1 and 2 when $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . X-axis is the Q-matrix, where a black square denotes the requirement of the attribute for an item

Figure 3 BIAS, RMSE, and SE of slipping and guessing parameters under Case 3 when $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . X-axis is the Q-matrix, where a black square denotes the requirement of the attribute for an item.

Figure 4 BIAS, RMSE and SE of slipping and guessing parameters under Case 4 considering $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . X-axis is the Q-matrix, where a black square denotes the presence of the attribute.

Figure 5 BIAS, RMSE and SE of slipping and guessing parameters under Case 5 considering $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . X-axis is the Q-matrix, where a black square denotes the presence of the attribute.

4.2. Results

First, we analyzed the BIC-based model selection outcomes, calculating the percentage of instances the true data-generating model was selected using BIC. Across all cases, Case 1 (IT-DINA with weaker interaction effect) was the only condition where the data-generating model was not favored by BIC-based selection across all sample sizes: In Case 1, with sample sizes of $N=500$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N = 500$$\end{document} and 1000, the IT-DINA (true model) was selected in 11% and 98.6% of the replications, and the remaining 89% and 1.4% of instances favored the simpler T-DINA model. Under larger sample sizes of 2000 or 4000, BIC selected the IT-DINA in 100% of replications.

For Cases 2 and 3, IT-DINA (the true model) was consistently chosen in 100% of replications, irrespective of the sample sizes considered. In Case 4, the DINA model (true model) emerged as the preferred choice across 100% of replications across different sample sizes. In Case 5, the BIC consistently favored the T-DINA model (true model), with a 100% selection rate across all conditions. In sum, BIC-based model selection could usually select the correct data-generating model but tended to favor simpler models when both the true dependency ( $\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma $$\end{document} ) and the sample size were smaller.

Under IT-DINA, the BIAS, RMSE, SE and BR of item parameters $\delta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\delta }$$\end{document} were averaged across all the items, and those of item parameters $(\beta ,\gamma )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\varvec{\beta }, \varvec{\gamma })$$\end{document} were averaged across items in corresponding testlets. The results are presented in Table 11 for Cases 1 and 2. When the sample size increased, BIAS decreased to 0. Furthermore, RMSEs, SEs and BRs under the proposed model decreased as sample size increased, indicating consistency of the estimators. Similar trends for parameter recovery were also found in Cases 3–5.

Figure 1 presents the recovery of recalculated slipping parameters when $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . Parameter estimates under both the T-DINA and DINA models showed a larger bias compared with the proposed model. Further, under DINA and T-DINA models, the BIAS of slipping parameters under Case 2 was larger than that under Case 1. Therefore, the values of $\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma $$\end{document} would influence the consequences of ignoring interactions.

Figure 2 presents the recovery of recalculated guessing parameters when $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . BIAS under the three models showed the same trend as in Fig. 1. Fitting T-DINA and DINA models to data generated under the proposed model produced biased estimates, and the BIAS was more obvious under a larger $\gamma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma $$\end{document} . Further, when comparing Figs. 1 and 2, BIAS of slipping parameters under T-DINA and DINA models was more obvious than that of guessing parameters, while the proposed model yielded unbiased estimates for both slipping and guessing.

Figure 3 illustrates the recovery of recalculated slipping and guessing parameters under Case 3 when $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . A similar trend is observed as in Cases 1 and 2, wherein the T-DINA and DINA models exhibit a greater bias compared to the proposed model. Additionally, Fig. 3 indicates that the variability in slipping and guessing parameters is larger for items requiring three skills compared to those requiring only one skill.

Figures 4 and 5 present the recovery of recalculated slipping and guessing parameters under Cases 4 and 5 when $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} . Under Case 4, parameter estimates under the IT-DINA, T-DINA, and DINA models were nearly identical, suggesting that the more complex IT-DINA and T-DINA models could accurately recover the parameters when the true model is the DINA. Under Case 5, when the T-DINA was the data-generating model, the recovery of parameters under the IT-DINA and T-DINA model was highly similar, both better than the recovery of parameters under the DINA model.

To take a closer look at the parameter estimates, Table 12 showed recovery of $s_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_j$$\end{document} and $g_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g_j$$\end{document} of all the items. Under Cases 1–3, as the sample size increased, RMSE and SE of parameters $s_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_j$$\end{document} and $g_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g_j$$\end{document} decreased for the proposed model. However, the RMSE under the T-DINA model remained large for $s_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_j$$\end{document} and $g_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g_j$$\end{document} , especially under Case 2, indicating the interaction between the testlet effect and latent profiles cannot be ignored. Further, RMSEs of parameters $s_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_j$$\end{document} and $g_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g_j$$\end{document} under the DINA model were larger than those under the T-DINA model, indicating the need to model the testlet effect when it is present. Overfitting a more complex model (e.g., the IT-DINA) to data generated under simpler models did not compromise parameter recovery. Specifically, under Case 4 (DINA), results for fitting the IT-DINA, T-DINA, and DINA models were nearly identical across different sample sizes. Further, under Case 5 (T-DINA), the IT-DINA and T-DINA models performed similarly and better than the DINA model across different sample sizes.

Table 13 shows the recovery of $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} under Case 1. As the sample size increased, BIAS reduced to near 0 under the proposed model. Under the T-DINA and DINA models, estimates of $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} were biased, especially for ${\pi }_{(0,0,0)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pi _{(0,0,0)}$$\end{document} , and the values of RMSE and BR were remarkably larger.

Table 14 shows the recovery of parameters $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} under Case 2. In line with Case 1, absolute values of bias under IT-DINA were near 0 when $N=4000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=4000$$\end{document} , whereas estimators under the T-DINA and DINA models were biased. Further, with stronger true interactions, the T-DINA and DINA model parameter estimates under Case 2 were much less accurate than those under Case 1.

5. Discussion

This paper aimed to provide an in-depth inspection of the model formulation and identifiability of testlet-effect DCMs. In particular, we discussed the need for modeling potential dependencies between attributes and testlet effects. When the attribute-response relationship could be described by the DINA and DINO models, an approach to account for this dependency via an interaction term is proposed. The proposed IT-DINA model generalizes the testlet DINA model, with potential dependency modeled through $\gamma (2\eta -1){\xi }_2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma (2\eta - 1)\xi _2$$\end{document} , an interaction term between the attribute profile and the testlet effect. This specific form of interaction facilitated model comparisons (i.e., testing for the independence between ideal response and testlet effect). Model selection based on BIC demonstrated good performance in selecting the data-generating model under most simulation conditions. In turn, simulation and empirical results from the PISA 2015 CPS assessment showed the utility of adopting the more general IT-DINA model in capturing the potential additional dependency between ideal response and testlet effect.

Another contribution of the current work is the identifiability conditions for testlet-effect DINA and DINO models. The formulation of the dependency as an interaction effect facilitated the derivation of the identifiability results for IT-DINA. In addition, the probit link allowed derivation of an explicit formula of conditional probability $P\left(Y\right|{\eta }^{\alpha })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P(Y|\eta ^{\varvec{\alpha }})$$\end{document} after integrating out the normally distributed testlet effect $\xi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\xi }$$\end{document} . This allowed us to establish easy-to-check sufficient conditions for the identifiability of the IT-DINA model, as well as the T-DINA model as a special case.

While the proposed IT-DINA model extends the T-DINA to account for potential structural dependency between the testlet effects and attribute patterns, one should note that the IT-DINA could still remain an oversimplification: The DINA and DINO models are very special cases of DCMs where the attribute and ideal response relationship is fully conjunctive or disjunctive. When the T-DINA model does not fit the observed data well, the presence of interaction effects (i.e., dependency) is one potential explanation, but it could also be due to a misspecified model, for instance, when the DINA model or the Q-matrix does not fully account for the actual attribute-response relationship (see the discussion of Eq. (6)). In empirical analyses, adopting the DINA/DINO models requires researchers to thoroughly evaluate the appropriateness of this assumption, for instance, by performing model comparison under general DCMs (e.g., de la Torre, 2011).

Table 12 The recovery of transformed slipping and guessing parameters under the DINA, T-DINA and IT-DINA model.

Table 13 BIAS and RMSE of $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} under Case 1.

Table 14 BIAS and RMSE of $\pi$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\pi }$$\end{document} under Case 2.

As the DINA/DINO are special cases to general DCMs, there are clear limitations to the scope of the current work. In particular, the proposed IT-DINA model only considers the situation where the attribute pattern enters the measurement model and influences testlet effect distribution via the conjunctive ideal response. The addition of a single interaction term between the ideal response and a testlet effect likely will not be adequate under a more complex DCM. How to parameterize the potential dependency between testlet effects and attribute patterns under general DCMs remains an open question. Further, the current identifiability results for the IT-DINA model and the T-DINA model are only applicable to cases when the DINA and DINO DCMs are appropriate. Although more general testlet-effect DCMs have been proposed (Hansen, 2013; Ma et al., 2023), identifiability conditions for these general models are not yet available. We hope the current discussion of the simple case provides initial results for addressing these open questions related to testlet-effect DCMs.

Aside from extension to more general DCMs, there are several directions for future work. Firstly, testlet-effect DINA and its identifiability conditions may be extended to model polytomous responses. In the literature, there exist several different parameterizations for polytomous DCMs, e.g., Fang et al. (2019), Culpepper (2019), and Ma (2019). We expect the incorporation of testlet effect and potential latent dependency to differ depending on the adopted parameterization. Secondly, the current identifiability results assumed the Q-matrix to be known. Future research may consider extension of the results to situations of unknown Q. Thirdly, in the current paper, the dependence is modeled through a specific interaction term. Alternative formulation of dependency may be of interest. Lastly, with the development of computer-based test, we have access to not only the response data but also additional data such as the response times. How to incorporate these more complicated data may require more sophisticated modeling techniques.