Item replenishing is essential for item bank maintenance in cognitive diagnostic computerized adaptive testing (CD-CAT). In regular CAT, online calibration is commonly used to calibrate the new items continuously. However, until now no reference has publicly become available about online calibration for CD-CAT. Thus, this study investigates the possibility to extend some current strategies used in CAT to CD-CAT. Three representative online calibration methods were investigated: Method A (Stocking in Scale drift in on-line calibration. Research Rep. 88-28, 1988), marginal maximum likelihood estimate with one EM cycle (OEM) (Wainer & Mislevy In H. Wainer (ed.) Computerized adaptive testing: A primer, pp. 65–102, 1990) and marginal maximum likelihood estimate with multiple EM cycles (MEM) (Ban, Hanson, Wang, Yi, & Harris in J. Educ. Meas. 38:191–212, 2001). The objective of the current paper is to generalize these methods to the CD-CAT context under certain theoretical justifications, and the new methods are denoted as CD-Method A, CD-OEM and CD-MEM, respectively. Simulation studies are conducted to compare the performance of the three methods in terms of item-parameter recovery, and the results show that all three methods are able to recover item parameters accurately and CD-Method A performs best when the items have smaller slipping and guessing parameters. This research is a starting point of introducing online calibration in CD-CAT, and further studies are proposed for investigations such as different sample sizes, cognitive diagnostic models, and attribute-hierarchical structures.

We consider the identification of a semiparametric multidimensional fixed effects item response model. Item response models are typically estimated under parametric assumptions about the shape of the item characteristic curves (ICCs), and existing results suggest difficulties in recovering the distribution of individual characteristics under nonparametric assumptions. We show that if the shape of the ICCs are unrestricted, but the shape is common across individuals and items, the individual characteristics are identified. If the shape of the ICCs are allowed to differ over items, the individual characteristics are identified in the multidimensional linear compensatory case but only identified up to a monotonic transformation in the unidimensional case. Our results suggest the development of two new semiparametric estimators for the item response model.

Latent class regression models relate covariates and latent constructs such as psychiatric disorders. Though full maximum likelihood estimation is available, estimation is often in three steps: (i) a latent class model is fitted without covariates; (ii) latent class scores are predicted; and (iii) the scores are regressed on covariates. We propose a new method for predicting class scores that, in contrast to posterior probability-based methods, yields consistent estimators of the parameters in the third step. Additionally, in simulation studies the new methodology exhibited only a minor loss of efficiency. Finally, the new and the posterior probability-based methods are compared in an analysis of mobility/exercise.

A variety of joint space multidimensional scaling (MDS) methods have been utilized for the spatial analysis of two- or three-way dominance data involving subjects’ preferences, choices, considerations, intentions, etc. so as to provide a parsimonious spatial depiction of the underlying relevant dimensions, attributes, stimuli, and/or subjects’ utility structures in the same joint space representation. We demonstrate that care must be taken with respect to a key assumption in existent joint space MDS models such that all estimated dimensions are utilized by each and every subject in the sample, as this assumption can lead to serious distortions with respect to the derived joint spaces. We develop a new Bayesian dimension selection methodology for the multidimensional unfolding model which accommodates heterogeneity with respect to such dimensional utilization at the individual subject level for the analysis of two or three-way dominance data. A consumer psychology application regarding the preference for Over-the-Counter (OTC) analgesics is provided. We conclude by discussing the practical implications of the results, as well as directions for future research.

In an addendum to his seminal 1969 article Jöreskog stated two sets of conditions for rotational identification of the oblique factor solution under utilization of fixed zero elements in the factor loadings matrix (Jöreskog in Advances in factor analysis and structural equation models, pp. 40–43, 1979). These condition sets, formulated under factor correlation and factor covariance metrics, respectively, were claimed to be equivalent and to lead to global rotational uniqueness of the factor solution. It is shown here that the conditions for the oblique factor correlation structure need to be amended for global rotational uniqueness, and, hence, that the condition sets are not equivalent in terms of unicity of the solution.

Multiple regression is frequently used across the various social sciences to analyze cross-sectional data. However, it can often times be challenging to justify the assumption of common regression coefficients across all respondents. This manuscript presents a heterogeneous Bayesian regression model that enables the estimation of individual-level-regression coefficients in cross-sectional data involving a single observation per response unit. A Gibbs sampling algorithm is developed to implement the proposed Bayesian methodology. A Monte Carlo simulation study is constructed to assess the performance of the proposed methodology across a number of experimental factors. We then apply the proposed method to analyze data collected from a consumer psychology study that examines the differential importance of price and quality in determining perceived value evaluations.

The quadratically weighted kappa is the most commonly used weighted kappa statistic for summarizing interrater agreement on an ordinal scale. The paper presents several properties of the quadratically weighted kappa that are paradoxical. For agreement tables with an odd number of categories n it is shown that if one of the raters uses the same base rates for categories 1 and n, categories 2 and n−1, and so on, then the value of quadratically weighted kappa does not depend on the value of the center cell of the agreement table. Since the center cell reflects the exact agreement of the two raters on the middle category, this result questions the applicability of the quadratically weighted kappa to agreement studies. If one wants to report a single index of agreement for an ordinal scale, it is recommended that the linearly weighted kappa instead of the quadratically weighted kappa is used.

Randomization tests are often recommended when parametric assumptions may be violated because they require no distributional or random sampling assumptions in order to be valid. In addition to being exact, a randomization test may also be more powerful than its parametric counterpart. This was demonstrated in a simulation study which examined the conditional power of three nondirectional tests: the randomization t test, the Wilcoxon–Mann–Whitney (WMW) test, and the parametric t test. When the treatment effect was skewed, with degree of skewness correlated with the size of the effect, the randomization t test was systematically more powerful than the parametric t test. The relative power of the WMW test under the skewed treatment effect condition depended on the sample size ratio.

Nested logit models have been presented as an alternative to multinomial logistic models for multiple-choice test items (Suh and Bolt in Psychometrika 75:454–473, 2010) and possess a mathematical structure that naturally lends itself to evaluating the incremental information provided by attending to distractor selection in scoring. One potential concern in attending to distractors is the possibility that distractor selection reflects a different trait/ability than that underlying the correct response. This paper illustrates a multidimensional extension of a nested logit item response model that can be used to evaluate such distinctions and also defines a new framework for incorporating collateral information from distractor selection when differences exist. The approach is demonstrated in application to questions faced by a university testing center over whether to incorporate distractor selection into the scoring of its multiple-choice tests. Several empirical examples are presented.

Recent research reflects a growing awareness of the value of using structural equation models to analyze repeated measures data. However, such data, particularly in the presence of covariates, often lead to models that either fit the data poorly, are exceedingly general and hard to interpret, or are specified in a manner that is highly data dependent. This article introduces methods for developing parsimonious models for such data. The underlying technology uses reduced-rank representations of the variances, covariances and means of observed and latent variables. The value of this approach, which may be implemented using standard structural equation modeling software, is illustrated in an application study aimed at understanding heterogeneous consumer preferences. In this application, the parsimonious representations characterize systematic relationships among consumer demographics, attitudes and preferences that would otherwise be undetected. The result is a model that is parsimonious, illuminating, and fits the data well, while keeping data dependence to a minimum.

In a number of psychological studies, answers to reasoning vignettes have been shown to result from both intuitive and deliberate response processes. This paper utilizes a psychometric model to separate these two response tendencies. An experimental application shows that the proposed model facilitates the analysis of dual-process item responses and the assessment of individual-difference factors, as well as conditions that favor one response tendency over another one.

Many psychological processes are characterized by recurrent shifts between distinct regimes or states. Examples that are considered in this paper are the switches between different states associated with premenstrual syndrome, hourly fluctuations in affect during a major depressive episode, and shifts between a “hot hand” and a “cold hand” in a top athlete. We model these processes with the regime switching state-space model proposed by Kim (J. Econom. 60:1–22, 1994), which results in both maximum likelihood estimates for the model parameters and estimates of the latent variables and the discrete states of the process. However, the current algorithm cannot handle missing data, which limits its applicability to psychological data. Moreover, the performance of standard errors for the purpose of making inferences about the parameter estimates is yet unknown. In this paper we modify Kim’s algorithm so it can handle missing data and we perform a simulation study to investigate its performance in (relatively) short time series in cases of different kinds of missing data and in case of complete data. Finally, we apply the regime switching state-space model to the three empirical data sets described above.