Hostname: page-component-5f745c7db-j9pcf Total loading time: 0 Render date: 2025-01-06T07:45:50.571Z Has data issue: true hasContentIssue false
  • English
  • Français

Ramifications of a Population Model for κ as a Coefficient of Reliability

Published online by Cambridge University Press:  01 January 2025

Helena Chmura Kraemer
Helena Chmura Kraemer*
Affiliation:
Stanford University
*
Requests for reprints should be sent to the author, Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA 94305.
Get access Rights & Permissions [Opens in a new window]

Abstract

Coefficient κ is generally defined in terms of procedures of computation rather than in terms of a population. Here a population definition is proposed. On this basis, the interpretation of κ as a measure of diagnostic reliability in characterizing an individual, and the effect of reliability, as measured by κ, on estimation bias, precision, and test power are examined. Factors influencing the magnitude of κ are identified. Strategies to improve reliability are proposed, including that of combining multiple unreliable diagnoses.

Keywords

kappareliabilitySpearman-Brown formula
Type
Original Paper
Information
Psychometrika , Volume 44 , Issue 4 , December 1979 , pp. 461 - 472
Copyright
Copyright © 1979 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This investigation was supported in part by the National Institute of Mental Health Specialized Research Center Grant # MH-30854.

References

Brown, W. Some experimental results in the correlation of mental abilities. British J. of Psychology, 1910, 3, 296–322.Google Scholar
Cochran, W. G. Errors of measurement in statistics. Techmometrics, 1968, 10, 637–666.CrossRefGoogle Scholar
Cohen, J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 1960, 20, 37–46.CrossRefGoogle Scholar
Cronbach, L. J., Gleser, G. C., Nanda, H. & Rajaratnam, N. The dependability of behavioral measurements, 1972, New York: John Wiley & Sons, Inc..Google Scholar
Everitt, B. S. Moments of the statistics kappa and weighted kappa. British Journal of Mathematical and Statistical Psychology, 1968, 21, 97–103.CrossRefGoogle Scholar
Fleiss, J. L. Statistical methods for rates and proportions, 1973, New York: John Wiley & Sons.Google Scholar
Fleiss, J. L. Measuring agreement between two judges on the presence or absence of a trait. Biometrics, 1975, 31, 651–659.CrossRefGoogle ScholarPubMed
Fleiss, J. L., Cohen, J. & Everitt, B. S. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin, 1969, 72, 323–327.CrossRefGoogle Scholar
Galen, R. S. & Gambino, S. R. Beyond normality: The predictive value and efficiency of medical diagnosis, 1975, New York: John Wiley & Sons.Google Scholar
Helzer, J. E., Robins, L. N., Tarbleson, M., Woodruff, R. A., Reich, T. & Wish, E. D. Reliability of psychiatric diagnosis: I. A methodological review. Archives of General Psychiatry, 1977, 34, 129–133.CrossRefGoogle Scholar
Helzer, J. E., Clayton, P. J., Pambakian, R., Reich, T., Woodruff, R. A. & Reveley, M. A. Reliability of diagnostic classification. Archives of General Psychiatry, 1977, 34, 136–141.CrossRefGoogle ScholarPubMed
Hubert, L. Kappa revisited. Psychological Bulletin, 1977, 84, 289–297.CrossRefGoogle Scholar
Kirk, D. B. On the numerical approximation of the bivariate normal (tetrachoric) correlation coefficient. Psychometrika, 1973, 38, 259–267.CrossRefGoogle Scholar
Koran, L. M. The reliability of clinical methods, data and judgments. N.E. Journal of Medicine, 1975, 293, 695–701.CrossRefGoogle ScholarPubMed
Kraemer, H. C. On estimation and hypothesis testing problems for correlation coefficients. Psychometrika, 1975, 40, 473–485.CrossRefGoogle Scholar
Landis, J. R. & Koch, G. G. The measurement of observer agreement for categorical data. Biometrics, 1977, 33, 159–174.CrossRefGoogle ScholarPubMed
Light, R. J. Measures of agreement for qualitative data: Some generalizations and alternatives. Psychological Bulletin, 1971, 76, 365–377.CrossRefGoogle Scholar
Scheffé, H. The analysis of variance, 1959, New York: John Wiley & Sons, Inc..Google Scholar
Spearman, C. Correlation calculated from faulty data. British Journal of Psychology, 1910, 3, 271–295.Google Scholar
Sptizer, R. L. & Fleiss, J. L. A re-analysis of the reliability of psychiatric diagnosis. British Journal of Psychiatry, 1974, 125, 341–347.CrossRefGoogle Scholar
Walker, M. H. & Lev, J. Statistical inference, 1953, New York: Henry Holt & Company.CrossRefGoogle Scholar