Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-10T15:18:36.625Z Has data issue: false hasContentIssue false
  • English
  • Français

Applying generalizability theory to examine assessments of subjective cognitive complaints: whose reports should we rely on – participant versus informant?

Published online by Cambridge University Press:  13 April 2021

Q. C. Truong ,
C. Choo ,
K. Numbers ,
A. G. Merkin ,
H. Brodaty Open the ORCID record for H. Brodaty [Opens in a new window] ,
N. A. Kochan ,
P. S. Sachdev ,
V. L. Feigin  and
O. N. Medvedev Open the ORCID record for O. N. Medvedev [Opens in a new window]
Q. C. Truong
Affiliation:
School of Psychology, University of Waikato, Hamilton, New Zealand
C. Choo
Affiliation:
School of Psychology, University of Waikato, Hamilton, New Zealand
K. Numbers
Affiliation:
Centre for Healthy Brain Ageing, School of Psychiatry, Faculty of Medicine, University of New South Wales, Sydney, Australia
A. G. Merkin
Affiliation:
Auckland University of Technology, Auckland, New Zealand Centre for Precise Psychiatry and Neurosciences, Germany/Switzerland
H. Brodaty
Affiliation:
Centre for Healthy Brain Ageing, School of Psychiatry, Faculty of Medicine, University of New South Wales, Sydney, Australia
N. A. Kochan
Affiliation:
Centre for Healthy Brain Ageing, School of Psychiatry, Faculty of Medicine, University of New South Wales, Sydney, Australia
P. S. Sachdev
Affiliation:
Centre for Healthy Brain Ageing, School of Psychiatry, Faculty of Medicine, University of New South Wales, Sydney, Australia
V. L. Feigin
Affiliation:
Auckland University of Technology, Auckland, New Zealand
O. N. Medvedev*
Affiliation:
School of Psychology, University of Waikato, Hamilton, New Zealand
*
Correspondence should be addressed to: Oleg N. Medvedev, School of Psychology, University of Waikato, Private Bag 3105, Hamilton 3240, New Zealand. Phone: + 64 7 837 9212. Email: oleg.medvedev@waikato.ac.nz
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives:

This study aimed to apply the generalizability theory (G-theory) to investigate dynamic and enduring patterns of subjective cognitive complaints (SCC), and reliability of two widely used SCC assessment tools.

Design:

G-theory was applied to assessment scales using longitudinal measurement design with five assessments spanning 10 years of follow-up.

Setting:

Community-dwelling older adults aged 70–90 years and their informants, living in Sydney, Australia, participated in the longitudinal Sydney Memory and Ageing Study.

Participants:

The sample included 232 participants aged 70 years and older, and 232 associated informants. Participants were predominantly White Europeans (97.8%). The sample of informants included 76 males (32.8%), 153 females (65.9%), and their age ranged from 27 to 86 years, with a mean age of 61.3 years (SD = 14.38).

Measurements:

The Memory Complaint Questionnaire (MAC-Q) and the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE).

Results:

The IQCODE demonstrated strong reliability in measuring enduring patterns of SCC with G = 0.86. Marginally acceptable reliability of the 6-item MAC-Q (G = 0.77–0.80) was optimized by removing one item resulting in G = 0.80–0.81. Most items of both assessments were measuring enduring SCC with exception of one dynamic MAC-Q item. The IQCODE significantly predicted global cognition scores and risk of dementia incident across all occasions, while MAC-Q scores were only significant predictors on some occasions.

Conclusions:

While both informants’ (IQCODE) and self-reported (MAC-Q) SCC scores were generalizable across sample population and occasions, self-reported (MAC-Q) scores may be less accurate in predicting cognitive ability and diagnosis of each individual.

Keywords

agingcognitive assessmentdementiamental capacitygeneralizability theorylongitudinal design
Type
Original Research Article
Information
International Psychogeriatrics , Volume 34 , Special Issue 12: Issue Theme: Subjective Cognitive Complaints: What Do They Predict? , December 2022 , pp. 1023 - 1033
Copyright
© International Psychogeriatric Association 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Allen, M. J. and Yen, W. M. (2001). Introduction to measurement theory. Long Grove, IL: Waveland Press.Google Scholar
American Psychology Association (1994). Diagnostic and Statistical Manual of Mental Disorders. Washington, DC: American Psychiatric Association.Google Scholar
Arterberry, B. J., Martens, M. P., Cadigan, J. M. and Rohrer, D. (2014). Application of generalizability theory to the big five inventory. Personality and Individual Differences, 69, 98–103.CrossRefGoogle Scholar
Bloch, R. and Norman, G. (2012). Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68. Medical Teacher, 34(11), 960–992.CrossRefGoogle ScholarPubMed
Brennan, R. L. (2010). Generalizability theory and classical test theory. Applied Measurement in Education, 24(1), 1–21.CrossRefGoogle Scholar
Brodaty, H. et al. (2002). The GPCOG: a new screening test for dementia designed for general practice. Journal of the American Geriatrics Society, 50(3), 530–534.CrossRefGoogle ScholarPubMed
Buckley, R. et al. (2013). Factors affecting subjective memory complaints in the AIBL aging study: biomarkers, memory, affect, and age. International Psychogeriatrics, 25(8), 1307–1315.CrossRefGoogle ScholarPubMed
Burmester, B., Leathem, J. and Merrick, P. (2016). Subjective cognitive complaints and objective cognitive function in aging: a systematic review and meta-analysis of recent cross-sectional findings. Neuropsychology Review, 26(4), 376–393.CrossRefGoogle ScholarPubMed
Cardinet, J., Johnson, S. and Pini, G. (2011). Applying Generalizability Theory Using EduG. New York: Taylor & Francis.CrossRefGoogle Scholar
Caselli, R. J. et al. (2014). Subjective cognitive decline: self and informant comparisons. Alzheimer’s & Dementia, 10(1), 93–98.CrossRefGoogle ScholarPubMed
Chin, Y., Choo, C. C. and Doshi, K. (2019). A case of subjective cognitive complaints in older adults: anxiety, stress, and aging in an elderly client. In: Clinical Psychology Casebook Across the Lifespan (pp. 71–77). Singapore: Springer Nature.CrossRefGoogle Scholar
Choo, C. C., Harris, K. M., Chew, P. K. and Ho, R. C. (2017). Does ethnicity matter in risk and protective factors for suicide attempts and suicide lethality? PLoS One, 12(4), e0175752.CrossRefGoogle ScholarPubMed
Cronbach, L. J., Rajaratnam, N. and Gleser, G. C. (1963). Theory of generalizability: a liberalization of reliability theory. British Journal of Statistical Psychology, 16(2), 137–163.CrossRefGoogle Scholar
Crook, T. H., Feher, E. P. and Larrabee, G. J. J. I. P. (1992). Assessment of memory complaint in age-associated memory impairment: the MAC-Q. International Psychogeriatrics, 4(2), 165–176.CrossRefGoogle ScholarPubMed
Harrison, J. K., Fearon, P., Noel-Storr, A. H., McShane, R., Stott, D. J. and Quinn, T. J. (2015). Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) for the diagnosis of dementia within a secondary care setting. Cochrane Database of Systematic Reviews (3).CrossRefGoogle ScholarPubMed
Harrison, J. K., Stott, D. J., McShane, R., Noel-Storr, A. H., Swann-Price, R. S. and Quinn, T. J. (2016). Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) for the early diagnosis of dementia across a variety of healthcare settings. Cochrane Database of Systematic Reviews, (11), 1–51.Google ScholarPubMed
Hildreth, K. L. and Church, S. J. M. C. (2015). Evaluation and management of the elderly patient presenting with cognitive complaints. Medical Clinics of North America, 99(2), 311–335.CrossRefGoogle ScholarPubMed
Huisman, M. (2000). Imputation of missing item responses: some simple techniques. Quality and Quantity, 34(4), 331–351.CrossRefGoogle Scholar
Jansen, A. P. D. et al. (2008). Self-reports on the IQCODE in older adults: a psychometric evaluation. Journal of Geriatric Psychiatry and Neurology, 21(2):83–92.CrossRefGoogle ScholarPubMed
Jonker, C., Geerlings, M. I. and Schmand, B. (2000). Are memory complaints predictive for dementia? A review of clinical and population-based studies. International Journal of Geriatric Psychiatry, 15(11), 983–991.3.0.CO;2-5>CrossRefGoogle ScholarPubMed
Jorm, A. (1994). A short form of the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE): development and cross-validation. Psychological Medicine, 24(1), 145–153.CrossRefGoogle Scholar
Jorm, A., Scott, R., Cullen, J. and MacKinnon, A. J. P. M. (1991). Performance of the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) as a screening test for dementia. Psychological Medicine, 21(3), 785–790.CrossRefGoogle ScholarPubMed
Jorm, A. F. (2004). The Informant Questionnaire on cognitive decline in the elderly (IQCODE): a review. International Psychogeriatrics, 16(3), 275.CrossRefGoogle ScholarPubMed
Langa, K. M. and Levine, D. A. (2014). The diagnosis and management of mild cognitive impairment: a clinical review. The Journal of the American Medical Association, 312(23), 2551–2561.CrossRefGoogle ScholarPubMed
Lenehan, M. E., Klekociuk, S. Z. and Summers, M. J. (2012). Absence of a relationship between subjective memory complaint and objective memory impairment in mild cognitive impairment (MCI): is it time to abandon subjective memory complaint as an MCI diagnostic criterion? International Psychogeriatrics, 24(9), 1505–1514.CrossRefGoogle ScholarPubMed
Medvedev, O. N. et al. (2020). A novel way to quantify schizophrenia symptoms in clinical trials. European Journal of Clinical Investigation, 51(3), e13398.Google ScholarPubMed
Medvedev, O. N., Krägeloh, C. U., Narayanan, A. and Siegert, R. J. (2017). Measuring mindfulness: applying generalizability theory to distinguish between state and trait. Mindfulness, 8(4), 1036–1046.CrossRefGoogle Scholar
Mitchell, A. J., Beaumont, H., Ferguson, D., Yadegarfar, M. and Stubbs, B. (2014). Risk of dementia and mild cognitive impairment in older people with subjective memory complaints: meta-analysis. Acta Psychiatrica Scandinavica, 130(6), 439–451.CrossRefGoogle ScholarPubMed
Numbers, K., Crawford, J. D., Kochan, N. A., Draper, B., Sachdev, P. S. and Brodaty, H. (2020). Participant and informant memory-specific cognitive complaints predict future decline and incident dementia: findings from the Sydney Memory and Ageing Study. PLoS One, 15(5). e0232961.CrossRefGoogle ScholarPubMed
Park, M. H. (2017). Informant questionnaire on cognitive decline in the elderly (IQCODE) for classifying cognitive dysfunction as cognitively normal, mild cognitive impairment, and dementia. International Psychogeriatrics, 29(9), 1461–1467.CrossRefGoogle ScholarPubMed
Paterson, J. et al. (2018). Distinguishing transient versus enduring aspects of depression in New Zealand Pacific Island children using Generalizability Theory. Journal of Affective Disorders, 227, 698–704.CrossRefGoogle ScholarPubMed
Perroco, T. R. et al. (2008). Short IQCODE as a screening tool for MCI and dementia: preliminary results. Dementia & Neuropsychologia, 2(4), 300–304.CrossRefGoogle ScholarPubMed
Petersen, R. C. (2016). Mild cognitive impairment. CONTINUUM: lifelong Learning in Neurology, 22(2 Dementia), 404.Google ScholarPubMed
Phung, T. K. T. et al. (2015). Performance of the 16-item informant questionnaire on cognitive decline for the elderly (IQCODE) in an Arabic-speaking older population. Dementia and Geriatric Cognitive Disorders, 40(5–6), 276–289.CrossRefGoogle Scholar
Ponds, R. W. and Jolles, J. (1996). Memory complaints in elderly people: the role of memory abilities, metamemory, depression, and personality. Educational Gerontology: An International Quarterly, 22(4), 341–357.CrossRefGoogle Scholar
Reid, L. M. and MacLullich, A. M. J. (2006) Subjective memory complaints and cognitive impairment in older people. Dementia and Geriatric Cognitive Disorders, 22(5–6), 471–485.CrossRefGoogle ScholarPubMed
Reid, M., Parkinson, L., Gibson, R., Schofield, P., D’Este, C., Attia, J., ... and Byles, J. (2012). Memory complaint questionnaire performed poorly as screening tool: validation against psychometric tests and affective measures. Journal of Clinical Epidemiology, 65(2), 199–205.CrossRefGoogle ScholarPubMed
Sachdev, P. S. et al. (2010). The Sydney Memory and Ageing Study (MAS): methodology and baseline medical and neuropsychiatric characteristics of an elderly epidemiological non-demented cohort of Australians aged 70-90 years. International Psychogeriatrics, 22(8), 1248.CrossRefGoogle ScholarPubMed
Shavelson, R. J. and Webb, N. M. (1991). Generalizability Theory: A Primer (Vol. 1). London, UK: Sage.Google Scholar
Shavelson, R. J., Webb, N. M. and Rowley, G. L. (1989). Generalizability theory. American Psychologist, 44(6), 922.CrossRefGoogle Scholar
Slavin, M. J. et al. (2015). Predicting cognitive, functional, and diagnostic change over 4 years using baseline subjective cognitive complaints in the Sydney Memory and Ageing Study. The American Journal of Geriatric Psychiatry, 23(9), 906–914.CrossRefGoogle ScholarPubMed
Swiss Society for Research in Education Working Group. (2006). EDUG User Guide. Euchatel, Switzerland: IRDP.Google Scholar
Tang, W. K. et al. (2004). The scoring scheme of the informant questionnaire on cognitive decline in the elderly needs revision: results of Rasch analysis. Dementia and Geriatric Cognitive Disorders, 18(3–4), 250–256.CrossRefGoogle ScholarPubMed
Truong, Q. C., Krageloh, C. U., Siegert, R. J., Landon, J. and Medvedev, O. N. (2020). Applying generalizability theory to differentiate between trait and state in the Five Facet Mindfulness Questionnaire (FFMQ). Mindfulness, 11(4), 953–963. doi: 10.1007/s12671-020-01324-7 CrossRefGoogle Scholar
Vispoel, W. P., Morris, C. A. and Kilinc, M. (2018). Applications of generalizability theory and their relations to classical test theory and structural equation modeling. Psychological Methods, 23(1), 1.CrossRefGoogle ScholarPubMed
Winblad, B. et al. (2004). Mild cognitive impairment–beyond controversies, towards a consensus: report of the International Working Group on Mild Cognitive Impairment. Journal of Internal Medicine, 256(3), 240–246.CrossRefGoogle Scholar
Supplementary material: File

Truong et al. supplementary material

Truong et al. supplementary material 1

Download Truong et al. supplementary material(File)
File 17.1 KB
Supplementary material: File

Truong et al. supplementary material

Truong et al. supplementary material 2

Download Truong et al. supplementary material(File)
File 43 KB