Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-08T21:18:37.660Z Has data issue: false hasContentIssue false
  • English
  • Français

Using the Conditional Grade-Of-Membership Model to Assess Judgment Accuracy

Published online by Cambridge University Press:  01 January 2025

Bruce Cooil  and
Sajeev Varki
Bruce Cooil*
Affiliation:
Owen Graduate School of Management, Vanderbilt University
Sajeev Varki
Affiliation:
College of Business Administration, University of Rhode Island
*
Requests for reprints should be sent to Bruce Cooil, OGSM, Vanderbilt University, 401 21st Avenue South, Nashville, TN 37203. E-Mail: bruce.cooil@owen.vanderbilt.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Consider the case where J instruments are used to classify each of I objects relative to K nominal categories. The conditional grade-of-membership (GoM) model provides a method of estimating the classification probabilities of each instrument (or “judge”) when the objects being classified consist of both pure types that lie exclusively in one of K nominal categories, and mixtures that lie in more than one category. Classification probabilities are identifiable whenever the sample of GoM vectors includes pure types from each category. When additional, relatively mild, assumptions are made about judgment accuracy, the identifiable correct classification probabilities are the greatest lower bounds among all solutions that might correspond to the observed multinomial process, even when the unobserved GoM vectors do not include pure types from each category. Estimation using the conditional GoM model is illustrated on a simulated data set. Further simulations show that the estimates of the classification probabilities are relatively accurate, even when the sample contains only a small percentage of approximately pure objects.

Keywords

nominal classificationincidental parametersextreme profilesmixtures
Type
Article
Information
Psychometrika , Volume 68 , Issue 3 , September 2003 , pp. 453 - 471
Copyright
Copyright © 2003 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The authors thank Max A. Woodbury, Kenneth G. Manton and H. Dennis Tolley for their help and four anonymous Psychometrika reviewers (including an associate editor) for their beneficial expository and technical suggestions. This work was supported by the Dean's Fund for Summer Research, Owen Graduate School of Management, Vanderbilt University.

References

Aptech Systems (1995). Constrained maximum likelihood. Kent, WA: Author.Google Scholar
Batchelder, W.H., Romney, A.K. (1986). The statistical analysis of a general Condorcet model for dichotomous choice situations. In Grofman, B., Owen, G. (Eds.), Information pooling and group decision making (pp. 103–112). Greenwich, CN: JAI Press.Google Scholar
Batchelder, W.H., Romney, A.K. (1988). Test theory without an answer key. Psychometrika, 53, 193–224.CrossRefGoogle Scholar
Batchelder, W.H., Romney, A.K. (1989). New results in test theory without an answer key. In Roskam, Edward E. (Eds.), Mathematical psychology in progress (pp. 229–248). Berlin, Heidelberg, New York: Springer-Verlag.CrossRefGoogle Scholar
Berkman, L., Singer, B., Manton, K.G. (1989). Black/white differences in health status and mortality among the elderly. Demography, 26, 661–678.CrossRefGoogle ScholarPubMed
Blazer, D., Woodbury, M.A., Hughes, D., George, L.K., Manton, K.G., Bachar, J.R., Fowler, N. (1989). A statistical analysis of the classification of depression in a mixed community and clinical sample. Journal of Affective Disorders, 16, 11–20.CrossRefGoogle Scholar
Chavez, J.M., Buriel, R. (1988). Mother-child interactions involving a child with epilepsy: A comparison of immigrant and native-born Mexican Americans. Journal of Pediatric Psychology, 13, 349–361.CrossRefGoogle ScholarPubMed
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.CrossRefGoogle Scholar
Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–220.CrossRefGoogle ScholarPubMed
Cooil, B., Rust, R.T. (1995). General estimators for the reliability of qualitative data. Psychometrika, 60, 199–220.CrossRefGoogle Scholar
Dillon, W.R., Mulani, N. (1984). A probabilistic latent class model for assessing inter-judge reliability. Multivariate Behavioral Research, 19, 438–458.CrossRefGoogle ScholarPubMed
Holland, J.L. (1985). Making vocational choices: A theory of vocational personalities and work environments. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Kiefer, J., Wolfowitz, J. (1956). Consistency of the Maximum likelihood estimator in the presence of infinitely many incidental parameters. Annals of Mathematical Statistics, 27, 887–906.CrossRefGoogle Scholar
Klauer, C.K., Batchelder, W.H. (1996). Structural analysis of subjective categorical data. Psychometrika, 61, 199–240.CrossRefGoogle Scholar
Manton, K.G., Woodbury, M.A., Tolley, H.D. (1994). Statistical applications using fuzzy sets. New York, NY: John Wiley & Sons.Google Scholar
Perreault, W.D. Jr., Leigh, L.E. (1989). Reliability of nominal data based on qualitative judgments. Journal of Marketing Research, 26, 135–148.CrossRefGoogle Scholar
Richards, F.S.G. (1961). A method of maximum-likelihood estimation. Journal of the Royal Statistical Society, Series B, 23, 469–476.CrossRefGoogle Scholar
Tolley, H.D., Manton, K.G. (1992). Large sample properties of estimates of a discrete grade of membership model. Annals of the Institute of Statistical Mathematics, 44, 85–95.CrossRefGoogle Scholar
Tsai, C.Y., Denton, J.J. (1993). Reliability assessment of a classroom observation system. Journal of Classroom Interaction, 28, 23–32.Google Scholar
Varki, S. (1996). New strategies and methodologies in customer satisfaction. Nashville, TN: Vanderbilt University.Google Scholar
Varki, S., Cooil, B., Rust, R.T. (2000). Modeling fuzzy data in qualitative marketing research. Journal of Marketing Research, 37, 480–489.CrossRefGoogle Scholar
Vertrees, J., Manton, K.G. (1986). A multivariate approach for classifying hospitals and computing blended payment rates. Medical Care, 24, 283–300.CrossRefGoogle ScholarPubMed
Woodbury, M.A., Clive, J. (1974). Clinical pure types as a fuzzy partition. Journal of Cybernetics, 4, 111–121.CrossRefGoogle Scholar
Woodbury, M.A., Manton, K.G. (1982). A new procedure for analysis of medical classification. Methods of Information in Medicine, 21, 210–220.Google ScholarPubMed
Woodbury, M.A., Manton, K.G., Tolley, H.D. (1994). A general model for statistical analysis using fuzzy sets: Sufficient conditions for identifiability and statistical properties. Information Sciences, 1, 149–180.Google Scholar
Yale, L., Gilly, M.C. (1988). Trends in advertising research: A look at the content of marketing-oriented journals from 1967 to 1985. Journal of Advertising, 17, 12–22.CrossRefGoogle Scholar