Hostname: page-component-6bf8c574d5-vmclg Total loading time: 0 Render date: 2025-02-23T13:45:38.315Z Has data issue: false hasContentIssue false
  • English
  • Français

Estimating the Reliability of Interview Data

Published online by Cambridge University Press:  01 January 2025

Joseph L. Fleiss
Joseph L. Fleiss*
Affiliation:
Biometrics Research, New York State Department of Mental Hygiene, and Columbia University
Get access Rights & Permissions [Opens in a new window]

Abstract

A model for a score based on an interview is presented which identifies the effect due to the subject, to the manner in which the interviewer tends to conduct his interviews, to the criteria he tends to use in scoring subjects' responses, to the compromises he tends to adopt between the demands of interviewing and those of scoring, and to chance errors. A suggested experimental design calls for each of K investigators to interview a different sample of N subjects, but for all investigators to score each subject. The drawing of inferences when interest is only in the K participants in the reliability study is considered, and a numerical example is given.

Type
Original Paper
Information
Psychometrika , Volume 35 , Issue 2 , June 1970 , pp. 143 - 162
Copyright
Copyright © 1970 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

This work was supported in part by grant DE R01 00793 from the National Institute of Dental Research, and in part by grants MH 08534 and MH 09191 from the National Institute of Mental Health, and forms part of the author's Ph.D. dissertation at Columbia University. The guidance provided by Professor T. W. Anderson is gratefully acknowledged.

References

Cochran, W. G. Testing a linear relation among variances. Biometrics, 1951, 7, 17–32.CrossRefGoogle ScholarPubMed
Ebel, R. L. Estimation of the reliability of ratings. Psychometrika, 1951, 16, 407–424.CrossRefGoogle Scholar
Fleiss, J. L. Determination of the reliability of ratings by means of incomplete block designs. American Psychologist, 1963, 18, 420–420.Google Scholar
Fleiss, J. L., Spitzer, R. L. & Burdock, E. I. Estimating accuracy of judgment using recorded interviews. Archives of general Psychiatry, 1965, 12, 562–567.CrossRefGoogle ScholarPubMed
Hyman, H. H., Cobb, W. J., Feldman, J. J., Hart, C. W., & Stember, C. A. Interviewing in social research, 1954, Chicago: Univ. Chicago Pr..Google Scholar
Lehmann, E. L. & Scheffé, H. Completeness, similar regions and unbiased estimation—Part I. Sankhyā, 1950, 10, 305–340.Google Scholar
Lev, J. & Kinder, E. F. New analysis of variance formulas for treating data from mutually paired subjects. Psychometrika, 1957, 22, 1–15.CrossRefGoogle Scholar
Lorr, M., Klett, C. J., McNair, D. M., & Lasky, J. J. Inpatient Multidimensional Psychiatric Scale: Manual, 1962, Palo Alto, Calif.: Consulting Psychologist Pr..Google Scholar
Satterthwaite, F. E. The synthesis of variance. Psychometrika, 1941, 6, 309–316.CrossRefGoogle Scholar
Spitzer, R. L., Fleiss, J. L., Endicott, J. & Cohen, J. Mental Status Schedule: Properties of factor-analytically derived scales. Archives of general Psychiatry, 1967, 16, 479–493.CrossRefGoogle ScholarPubMed
Winer, B. J. Statistical principles in experimental design, 1962, New York: McGraw-Hill.CrossRefGoogle Scholar