Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2025-01-06T00:52:33.498Z Has data issue: false hasContentIssue false

Meta-analysis of Diagnostic Accuracy and ROC Curves with Covariate Adjusted Semiparametric Mixtures

Published online by Cambridge University Press:  01 January 2025

Philipp Doebler*
Affiliation:
Westfälische Wilhelms-Universität
Heinz Holling
Affiliation:
Westfälische Wilhelms-Universität
*
Correspondence should be made to Philipp Doebler, Institut für Psychologie, Westfälische Wilhelms-Universität, Fliednerstr. 21, 48149 Münster, Germany. Email: doebler@uni-muenster.de; philipp.doebler@googlemail.com; holling@uni-muenster.de

Abstract

Many screening tests dichotomize a measurement to classify subjects. Typically a cut-off value is chosen in a way that allows identification of an acceptable number of cases relative to a reference procedure, but does not produce too many false positives at the same time. Thus for the same sample many pairs of sensitivities and false positive rates result as the cut-off is varied. The curve of these points is called the receiver operating characteristic (ROC) curve. One goal of diagnostic meta-analysis is to integrate ROC curves and arrive at a summary ROC (SROC) curve. Holling, Böhning, and Böhning (Psychometrika 77:106–126, 2012a) demonstrated that finite semiparametric mixtures can describe the heterogeneity in a sample of Lehmann ROC curves well; this approach leads to clusters of SROC curves of a particular shape. We extend this work with the help of the \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$t_{\alpha }$$\end{document} transformation, a flexible family of transformations for proportions. A collection of SROC curves is constructed that approximately contains the Lehmann family but in addition allows the modeling of shapes beyond the Lehmann ROC curves. We introduce two rationales for determining the shape from the data. Using the fact that each curve corresponds to a natural univariate measure of diagnostic accuracy, we show how covariate adjusted mixtures lead to a meta-regression on SROC curves. Three worked examples illustrate the method.

Type
Original Paper
Copyright
Copyright © 2015 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aertgeerts, B., Buntinx, F., Ansoms, S., & Fevery, J. (2001). Screening properties of questionnaires and laboratory tests for the detection of alcohol abuse or dependence in a general practice population. The British Journal of General Practice, 51, 206217.Google ScholarPubMed
Aitkin, M. (1999). A general maximum likelihood analysis of variance components in generalized linear models. Biometrics, 55, 117128.CrossRefGoogle ScholarPubMed
Aitkin, M. (1999). Meta-analysis by random effect modelling in generalized linear models. Statistics in Medicine, 18, 23432351.3.0.CO;2-3>CrossRefGoogle ScholarPubMed
Aranda-Ordaz, F. (1981). On two families of transformations to additivity for binary response data. Biometrika, 68, 357363.CrossRefGoogle Scholar
Arends, L., Hamza, T., Van Houwelingen, J., Heijenbrok-Kal, M., Hunink, M., & Stijnen, T. (2008). Bivariate random effects meta-analysis of ROC curves. Medical Decision Making, 28, 621638.CrossRefGoogle ScholarPubMed
Böhning, D. (2000). Computer-assisted analysis of mixtures and applications: Meta-analysis, disease mapping and others. Boca Raton, FL: Chapman & Hall/CRC.CrossRefGoogle Scholar
Böhning, D., Dietz, E., & Schlattmann, P. (1998). Recent developments in computer-assisted analysis of mixtures. Biometrics, 54, 525536.CrossRefGoogle ScholarPubMed
Böhning, D., Böhning, W., & Holling, H. (2008). Revisiting Youden’s index as a useful measure of the misclassification error in meta-analysis of diagnostic studies. Statistical Methods in Medical Research, 17, 543554.CrossRefGoogle ScholarPubMed
Brent, R.P. (1973). Algorithms for minimization without derivatives. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Bush, K., Kivlahan, D., McDonell, M., Fihn, S., & Bradley, K. (1998). The AUDIT alcohol consumption questions (AUDIT-C): An effective brief screening test for problem drinking. Archives of Internal Medicine, 158, 17891795.CrossRefGoogle ScholarPubMed
Chu, H., & Cole, S.R. (2006). Bivariate meta-analysis of sensitivity and specificity with sparse data: A generalized linear mixed model approach. Journal of Clinical Epidemiology, 59(12), 13311332.CrossRefGoogle Scholar
Deeks, J. (2001). Systematic reviews of evaluations of diagnostic and screening tests. British Medical Journal, 323, 157162.CrossRefGoogle ScholarPubMed
DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177188.CrossRefGoogle ScholarPubMed
Doebler, P., Holling, H., & Böhning, D. (2012). A mixed model approach to meta-analysis of diagnostic studies with binary test outcome. Psychological Methods, 17, 418436.CrossRefGoogle ScholarPubMed
Efron, B. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82(397), 171185.CrossRefGoogle Scholar
Folstein, M.F., Folstein, S.E., & McHugh, P.R. (1975). Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189198.CrossRefGoogle Scholar
Glas, A., & Lijmer, J., Prins, M., Bonsel, G., & Bossuyt, P. (2003). The diagnostic odds ratio: A single indicator of test performance. Journal of Clinical Epidemiology, 56, 11291135.CrossRefGoogle ScholarPubMed
Gordon, A., Maisto, S., McNeil, M., Kraemer, K., Conigliaro, R., Kelley, M., & Conigliaro, J. (2001). Three questions can detect hazardous drinkers. Journal of Family Practice, 50, 313320.Google ScholarPubMed
Guerrero, V., & Johnson, R. (1982). Use of the Box–Cox transformation with binary response models. Biometrika, 69, 309314.CrossRefGoogle Scholar
Hamza, T., Reitsma, J., & Stijnen, T. (2008). Meta-analysis of diagnostic studies: A comparison of random intercept, normal–normal, and binomial–normal bivariate summary ROC approaches. Medical Decision Making, 28, 639649.CrossRefGoogle ScholarPubMed
Harbord, R., Deeks, J., Egger, M., Whiting, P., & Sterne, J. (2007). A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics, 8, 239251.CrossRefGoogle ScholarPubMed
Holling, H., Böhning, W., & Böhning, D. (2012). Likelihood-based clustering of meta-analytic SROC curves. Psychometrika, 77, 106126.CrossRefGoogle Scholar
Holling, H., Böhning, W., & Böhning, D. (2012). Meta-analysis of diagnostic studies based upon SROC-curves: A mixed model approach using the Lehmann family. Statistical Modelling, 12, 347375.CrossRefGoogle Scholar
Keribin, C. (2000). Consistent estimation of the order of mixture models. Sankhyā: The Indian Journal of Statistics, Series A, 62, 4966.Google Scholar
Koenker, R., & Yoon, J. (2009). Parametric links for binary choice models: A Fisherian–Bayesian colloquy. Journal of Econometrics, 152, 120130.CrossRefGoogle Scholar
Kriston, L., Hölzel, L., Weiser, A., Berner, M., & Härter, M. (2008). Meta-analysis: Are 3 questions enough to detect unhealthy alcohol use?. Annals of Internal Medicine, 149, 879888.CrossRefGoogle ScholarPubMed
Le, C. (2006). A solution for the most basic optimization problem associated with an ROC curve. Statistical Methods in Medical Research, 15, 571584.CrossRefGoogle ScholarPubMed
Lindsay, B. (1983). The geometry of mixture likelihoods: A general theory. The Annals of Statistics, 11, 8694.CrossRefGoogle Scholar
Lindsay, B. (1995). Mixture models: Theory, geometry and applications. NSF-CBMS regional conference series in probability and statistics.CrossRefGoogle Scholar
Littenberg, B., & Moses, L. (1993). Estimating diagnostic accuracy from multiple conflicting reports: A new meta-analytic method. Medical Decision Making, 13, 313321.CrossRefGoogle ScholarPubMed
Macaskill, P., Gatsonis, C., Deeks, J., Harbord, R., & Takwoingi, Y. (2010). Chapter 10: Analysing and presenting results. In J. Deeks, P. Bossuyt & C. Gatsonis (Eds.), Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0. The Cochrane Collaboration. Retrieved from: http://srdta.cochrane.org/. Accessed 25 Oct 2014.Google Scholar
Ma, X., Nie, L., Cole, S. R., & Chu, H. (2013). Statistical methods for multivariate meta-analysis of diagnostic tests: An overview and tutorial. Statistical Methods in Medical Research. doi:10.1177/0962280213492588.CrossRefGoogle Scholar
McCullagh, P., & Nelder, J. (1989). Generalized linear models. Boca Raton, FL: Chapman & Hall/CRC.CrossRefGoogle Scholar
Mitchell, A. (2009). A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. Journal of Psychiatric Research, 43, 411431.CrossRefGoogle ScholarPubMed
Moses, L., Shapiro, D., & Littenberg, B. (1993). Combining independent studies of a diagnostic test into a summary ROC curve: Data-analytic approaches and some additional considerations. Statistics in Medicine, 12, 12931316.CrossRefGoogle ScholarPubMed
Patrick, D., Cheadle, A., Thompson, D., Diehr, P., Koepsell, T., & Kinne, S. (1994). The validity of self-reported smoking: A review and meta-analysis. American Journal of Public Health, 84, 10861093.CrossRefGoogle ScholarPubMed
Pepe, M. (2000). Receiver operating characteristic methodology. Journal of the American Statistical Association, 95, 308311.CrossRefGoogle Scholar
Pepe, M. (2004). The statistical evaluation of medical tests for classification and prediction. Oxford, UK: Oxford University Press.Google Scholar
Piepho, H. (2003). The folded exponential transformation for proportions. Journal of the Royal Statistical Society: Series D, 52, 575589.Google Scholar
Prentice, R. (1976). A generalization of the probit and logit methods for dose response curves. Biometrics, 32(4), 761–768.CrossRefGoogle Scholar
Reitsma, J., Glas, A., Rutjes, A., Scholten, R., Bossuyt, P., & Zwinderman, A. (2005). Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. Journal of Clinical Epidemiology, 58, 982990.CrossRefGoogle ScholarPubMed
Rücker, G., & Schumacher, M. (2010). Summary ROC curve based on a weighted Youden index for selecting an optimal cutpoint in meta-analysis of diagnostic accuracy. Statistics in Medicine, 29, 30693078.CrossRefGoogle ScholarPubMed
Rumpf, H., Hapke, U., Meyer, C., & John, U. (2002). Screening for alcohol use disorders and at-risk drinking in the general population: Psychometric performance of three questionnaires. Alcohol and Alcoholism, 37, 261268.CrossRefGoogle ScholarPubMed
Rutter, C., & Gatsonis, C. (2001). A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Statistics in Medicine, 20, 28652884.CrossRefGoogle ScholarPubMed
Schlattmann, P. (2009). Medical applications of finite mixture models. New York, NY: Springer.Google Scholar
Schlattmann, P. & Hoehne, J. (2013). CAMAN: Finite mixture models and meta-analysis tools—based on C.A.MAN. R package version 0.67.Google Scholar
Sweeting, M., Sutton, A., & Lambert, P. (2004). What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Statistics in Medicine, 23, 13511375.CrossRefGoogle ScholarPubMed
Walter, S. (2002). Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data. Statistics in Medicine, 21(9), 12371256.CrossRefGoogle ScholarPubMed
Wedel, M. (2002). Concomitant variables in finite mixture models. Statistica Neerlandica, 56, 362375.CrossRefGoogle Scholar