Hostname: page-component-7b9c58cd5d-dlb68 Total loading time: 0 Render date: 2025-03-15T23:40:08.090Z Has data issue: false hasContentIssue false
  • English
  • Français

Unsupervised high-frequency smartphone-based cognitive assessments are reliable, valid, and feasible in older adults at risk for Alzheimer’s disease

Published online by Cambridge University Press:  05 September 2022

Jessica Nicosia ,
Andrew J. Aschenbrenner Open the ORCID record for Andrew J. Aschenbrenner [Opens in a new window] ,
David A. Balota ,
Martin J. Sliwinski ,
Marisol Tahan ,
Sarah Adams ,
Sarah S. Stout ,
Hannah Wilks ,
Brian A. Gordon  and
Tammie L. S. Benzinger
...Show all authors
Jessica Nicosia
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
Andrew J. Aschenbrenner
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
David A. Balota
Affiliation:
Department of Psychological & Brain Sciences, Washington University in St. Louis, St. Louis, MO, USA
Martin J. Sliwinski
Affiliation:
Department of Human Development and Family Studies, The Pennsylvania State University, University Park, PA, USA
Marisol Tahan
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
Sarah Adams
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
Sarah S. Stout
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
Hannah Wilks
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
Brian A. Gordon
Affiliation:
Department of Psychological & Brain Sciences, Washington University in St. Louis, St. Louis, MO, USA Department of Radiology, Washington University, School of Medicine, St. Louis, MO, USA
Tammie L. S. Benzinger
Affiliation:
Department of Radiology, Washington University, School of Medicine, St. Louis, MO, USA
Anne M. Fagan
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
Chengjie Xiong
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA Division of Biostatistics, Washington University, School of Medicine, St. Louis, MO, USA
Randall J. Bateman
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
John C. Morris
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA
Jason Hassenstab*
Affiliation:
Charles F. and Joanne Knight Alzheimer Disease Research Center, Department of Neurology, Washington University, School of Medicine, St. Louis, MO, USA Department of Psychological & Brain Sciences, Washington University in St. Louis, St. Louis, MO, USA
*
Corresponding author: Jason Hassenstab, email: hassenstabj@wustl.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

Smartphones have the potential for capturing subtle changes in cognition that characterize preclinical Alzheimer’s disease (AD) in older adults. The Ambulatory Research in Cognition (ARC) smartphone application is based on principles from ecological momentary assessment (EMA) and administers brief tests of associative memory, processing speed, and working memory up to 4 times per day over 7 consecutive days. ARC was designed to be administered unsupervised using participants’ personal devices in their everyday environments.

Methods:

We evaluated the reliability and validity of ARC in a sample of 268 cognitively normal older adults (ages 65–97 years) and 22 individuals with very mild dementia (ages 61–88 years). Participants completed at least one 7-day cycle of ARC testing and conventional cognitive assessments; most also completed cerebrospinal fluid, amyloid and tau positron emission tomography, and structural magnetic resonance imaging studies.

Results:

First, ARC tasks were reliable as between-person reliability across the 7-day cycle and test-retest reliabilities at 6-month and 1-year follow-ups all exceeded 0.85. Second, ARC demonstrated construct validity as evidenced by correlations with conventional cognitive measures (r = 0.53 between composite scores). Third, ARC measures correlated with AD biomarker burden at baseline to a similar degree as conventional cognitive measures. Finally, the intensive 7-day cycle indicated that ARC was feasible (86.50% approached chose to enroll), well tolerated (80.42% adherence, 4.83% dropout), and was rated favorably by older adult participants.

Conclusions:

Overall, the results suggest that ARC is reliable and valid and represents a feasible tool for assessing cognitive changes associated with the earliest stages of AD.

Keywords

digital biomarkersmobile testingpreclinical Alzheimer’s diseaseecological momentary assessment
Type
Research Article
Information
Journal of the International Neuropsychological Society , Volume 29 , Issue 5 , June 2023 , pp. 459 - 471
Copyright
Copyright © INS. Published by Cambridge University Press, 2022

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Barthélemy, N. R., Li, Y., Joseph-Mathurin, N., Gordon, B. A., Hassenstab, J., Benzinger, T. L., Buckles, V., Fagan, A. M., Perrin, R. J., Goate, A. M., Morris, J. C., Karch, C. M., Xiong, C., Allegri, R., Chrem Mendez, P., Berman, S. B., Ikeuchi, T., Mori, H., Shimada, H., . . . McDade, E., & the Dominantly Inherited Alzheimer Network. (2020). A soluble phosphorylated tau signature links tau, amyloid and the evolution of stages of dominantly inherited Alzheimer’s disease. Nature Medicine, 26, 398–407.CrossRefGoogle ScholarPubMed
Bateman, R. J., Benzinger, T. L., Berry, S., Clifford, D. B., Duggan, C., Fagan, A. M., Fanning, K., Farlow, M. R., Hassenstab, J., McDade, E. M., Mills, S., Paumier, K., Quintana, M., Salloway, S. P., Santacruz, A., Schneider, L. S., Wang, G., & Xiong, C. (2017). The DIAN-TU next generation Alzheimer’s prevention trial: Adaptive design and disease progression model. Alzheimer’s & Dementia, 13, 8–19.CrossRefGoogle ScholarPubMed
Benedict, R. H. B., Schretlen, D., Groninger, L., & Brandt, J. (1998). Hopkins verbal learning test? Revised: Normative data and analysis of inter-form and test-retest reliability. The Clinical Neuropsychologist (Neuropsychology, Development and Cognition: Section D), 12, 43–55.CrossRefGoogle Scholar
Braak, H., & Braak, E. (1991). Neuropathological stageing of Alzheimer-related changes. Acta neuropathologica, 82, 239–259.CrossRefGoogle ScholarPubMed
Bruton, A., Conway, J. H., & Holgate, S. T. (2000). Reliability: What is it, and how is it measured? Physiotherapy, 86, 94–99.CrossRefGoogle Scholar
Calamia, M., Markon, K., & Tranel, D. (2012). Scoring higher the second time around: Meta-analyses of practice effects in neuropsychological assessment. The Clinical Neuropsychologist (Neuropsychology, Development and Cognition: Section D), 26, 543–570.CrossRefGoogle ScholarPubMed
Calamia, M., Markon, K., & Tranel, D. (2013). The Robust reliability of neuropsychological measures: Meta-analyses of test–retest correlations. The Clinical Neuropsychologist (Neuropsychology, Development and Cognition: Section D), 27, 1077–1105.CrossRefGoogle ScholarPubMed
Desikan, R. S., Ségonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., Buckner, R. L., Dale, A. M., Maguire, P., Hyman, B. T., Albert, M. S., & Killiany, R. J. (2006). An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage, 31, 968–980.CrossRefGoogle ScholarPubMed
Dikmen, S. S., Heaton, R. K., Grant, I., & Temkin, N. R. (1999). Test–retest reliability and practice effects of expanded Halstead–Reitan neuropsychological test battery. Journal of the International Neuropsychological Society, 5, 346–356.CrossRefGoogle ScholarPubMed
Dodge, H. H., Zhu, J., Mattek, N. C., Austin, D., Kornfeld, J., & Kaye, J. A. (2015). Use of high-frequency in-home monitoring data may reduce sample sizes needed in clinical trials. PLoS One, 10, e0138095.CrossRefGoogle ScholarPubMed
Donohue, M. C., Sperling, R. A., Salmon, D. P., Rentz, D. M., Raman, R., Thomas, R. G., Weiner, M., & Aisen, P. S. (2014). The preclinical Alzheimer cognitive composite: measuring amyloid-related decline. JAMA Neurology, 71, 961–970.CrossRefGoogle ScholarPubMed
Dorociak, K. E., Mattek, N., Lee, J., Leese, M. I., Bouranis, N., Imtiaz, D., Doane, B. M., Bernstein, J. P. K., Kaye, J. A., & Hughes, A. M. (2021). The survey for memory, attention, and reaction time (SMART): Development and validation of a brief web-based measure of cognition for older adults. Gerontology, 67, 740–752.CrossRefGoogle ScholarPubMed
Edgar, C. J., Vradenburg, G., & Hassenstab, J. (2019). The 2018 revised FDA guidance for early Alzheimer’s disease: Establishing the meaningfulness of treatment effects. The Journal of Prevention of Alzheimer’s Disease, 6, 223–227.Google ScholarPubMed
Fagan, A. M., Mintun, M. A., Mach, R. H., Lee, S.-Y., Dence, C. S., Shah, A. R., LaRossa, G. N., Spinner, M. L., Klunk, W. E., Mathis, C. A., DeKosky, S. T., Morris, J. C., & Holtzman, D. M. (2006). Inverse relation between in vivo amyloid imaging load and cerebrospinal fluid Aβ 42in humans. Annals of Neurology, 59, 512–519.CrossRefGoogle Scholar
Fischl, B., & Dale, A. M. (2000). Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proceedings of the National Academy of Sciences, 97, 11050–11055.CrossRefGoogle Scholar
Fischl, B., Van Der Kouwe, A., Destrieux, C., Halgren, E., Ségonne, F., Salat, D. H., Busa, E., Seidmann, L. J., Goldstein, J., Kennedy, D., Caviness, D., Makris, N., Rosen, B, & Dale, A. M. (2004). Automatically parcellating the human cerebral cortex. Cerebral Cortex, 14, 11–22.CrossRefGoogle ScholarPubMed
Food and Drug Administration. (2018). Early Alzheimer’s disease: Developing drugs for treatment: guidance for industry. Food and Drug Administration.Google Scholar
Gills, J. L., Glenn, J. M., Madero, E. N., Bott, N. T., & Gray, M. (2019). Validation of a digitally delivered visual paired comparison task: Reliability and convergent validity with established cognitive tests. GeroScience, 41, 441–454.CrossRefGoogle ScholarPubMed
Güsten, J., Ziegler, G., Düzel, E., & Berron, D. (2021). Age impairs mnemonic discrimination of objects more than scenes: A web-based, large-scale approach across the lifespan. Cortex, 137, 138–148.CrossRefGoogle ScholarPubMed
Hassenstab, J., Aschenbrenner, A. J., Balota, D. A., McDade, E., Lim, Y. Y., Fagan, A. M., Benzinger, T. L. S., Cruchaga, C., Goate, A. M., Morris, J. C., Bateman, R. J., & the Dominantly Inherited Alzheimer Network. (2020). Remote cognitive assessment approaches in the dominantly inherited Alzheimer network (DIAN) using digital technology to drive clinical innovation in brain-behavior relationships: A new era in neuropsychology. Alzheimer’s & Dementia, 16, e038144.CrossRefGoogle Scholar
Hassenstab, J., Chasse, R., Grabow, P., Benzinger, T. L. S., Fagan, A. M., Xiong, C., Jasielec, M., Grant, E., & Morris, J. C. (2016). Certified normal: Alzheimer’s disease biomarkers and normative estimates of cognitive functioning. Neurobiology of Aging, 43, 23–33.CrossRefGoogle ScholarPubMed
Hassenstab, J., Nicosia, J., LaRose, M., Aschenbrenner, A. J., Gordon, B. A., Benzinger, T. L., Xiong, C., & Morris, J. C. (2021). Is comprehensiveness critical? Comparing short and long format cognitive assessments in preclinical Alzheimer disease. Alzheimer’s Research & Therapy, 13, 1–14.Google Scholar
Lancaster, C., Koychev, I., Blane, J., Chinner, A., Chatham, C., Taylor, K., & Hinds, C. (2020). Gallery game: Smartphone-based assessment of long-term memory in adults at risk of Alzheimer’s disease. Journal of Clinical and Experimental Neuropsychology, 42, 329–343.CrossRefGoogle ScholarPubMed
Langbaum, J. B., Hendrix, S. B., Ayutyanont, N., Chen, K., Fleisher, A. S., Shah, R. C., Barnes, L. L., Bennett, D. A., Tariot, P. N., & Reiman, E. M. (2014). An empirically derived composite cognitive test score with improved power to track and evaluate treatments for preclinical Alzheimer’s disease. Alzheimer’s & Dementia, 10, 666–674.CrossRefGoogle ScholarPubMed
Lo, A. H. Y., Humphreys, M., Byrne, G. J., & Pachana, N. A. (2012). Test-retest reliability and practice effects of the Wechsler memory scale-III. Journal of Neuropsychology, 6, 212–231.CrossRefGoogle ScholarPubMed
Mackin, R. S., Insel, P. S., Truran, D., Finley, S., Flenniken, D., Nosheny, R., Ulbright, A., Comacho, M., Harel, B., Maruff, P., & Weiner, M. W. (2018). Unsupervised online neuropsychological test performance for individuals with mild cognitive impairment and dementia: Results from the Brain Health Registry. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 10, 573–582.Google ScholarPubMed
Matar, E., Shine, J. M., Halliday, G. M., & Lewis, S. J. (2020). Cognitive fluctuations in Lewy body dementia: Towards a pathophysiological framework. Brain, 143, 31–46.CrossRefGoogle ScholarPubMed
Mishra, S., Gordon, B. A., Su, Y., Christensen, J., Friedrichsen, K., Jackson, K., Hornbeck, R., Balota, D. A., Cairns, N. J., Morris, J. C., Ances, B. M., & Benzinger, T. L. S. (2017). AV-1451 PET imaging of tau pathology in preclinical Alzheimer disease: Defining a summary measure. Neuroimage, 161, 171–178.CrossRefGoogle ScholarPubMed
Morris, J. C. (1993). The clinical dementia rating (CDR): Current version and scoring rules. Neurology, 43, 2412–2414.CrossRefGoogle ScholarPubMed
Nicosia, J., Aschenbrenner, A. J., Adams, S., Tahan, M., Stout, S. H., Wilks, H., Balls-Berry, J. E., Morris, J. C., & Hassenstab, J. (2021). Bridging the technological divide: Stigmas and challenges with technology in clinical studies of older adults. Frontiers in Digital Health, 4, e880055. CrossRefGoogle Scholar
Öhman, F., Hassenstab, J., Berron, D., Schöll, M., & Papp, K. V. (2021). Current advances in digital cognitive assessment for preclinical Alzheimer’s disease. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 13, e12217.Google ScholarPubMed
Papp, K. V., Rentz, D. M., Orlovsky, I., Sperling, R. A., & Mormino, E. C. (2017). Optimizing the preclinical Alzheimer’s cognitive composite with semantic processing: The PACC5. Alzheimer’s & Dementia: Translational Research & Clinical Interventions, 3, 668–677.Google ScholarPubMed
Papp, K. V., Samaroo, A., Chou, H. C., Buckley, R., Schneider, O. R., Hsieh, S., Soberanes, D., Quiroz, Y., Properzi, M., Schultz, A., García-Magariño, I., Marshall, G. A., Burke, J. G, Kumar, R., Snyder, N., Johnson, K., Rentz, D. M., Sperling, R. A., & Amariglio, R. E. (2021). Unsupervised mobile cognitive testing for use in preclinical Alzheimer’s disease. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 13, e12243.Google ScholarPubMed
Pereira, D. R., Costa, P., & Cerqueira, J. J. (2015). Repeated assessment and practice effects of the written symbol digit modalities test using a short inter-test interval. Archives of Clinical Neuropsychology, 30, 424–434.CrossRefGoogle Scholar
Pratap, A., Neto, E. C., Snyder, P., Stepnowsky, C., Elhadad, N., Grant, D., Mohebbi, M. H., Mooney, S., Suver, C., Wilbanks, J., Mangravite, L., Heagerty, P. J., Areán, P., & Omberg, L. (2020). Indicators of retention in remote digital health studies: A cross-study evaluation of 100,000 participants. NPJ Digital Medicine, 3, 1–10.CrossRefGoogle ScholarPubMed
Price, J. L., McKeel, D. W. Jr., Buckles, V. D., Roe, C. M., Xiong, C., Grundman, M., Hansen, L. A., Petersen, R. C., Parisi, J. E., Dickson, D. W., Smith, C. D., Davis, D. G., Schmitt, F. A., Markesbery, W. R., Kaye, J., Kurlan, R., Hulette, C., Kurland, B. F., & Morris, J. C. (2009). Neuropathology of nondemented aging: Presumptive evidence for preclinical Alzheimer disease. Neurobiology of Aging, 30, 1026–1036.CrossRefGoogle ScholarPubMed
Price, P. C., Jhangiani, R. S., & Chiang, I. C. A. (2015). Reliability and validity of measurement. Research Methods in Psychology-2nd Canadian Edition.Google Scholar
Raykov, T., & Marcoulides, G. A. (2006). On multilevel model reliability estimation from the perspective of structural equation modeling. Structural Equation Modeling, 13, 130–141.CrossRefGoogle Scholar
Raz, N., Lindenberger, U., Ghisletta, P., Rodrigue, K. M., Kennedy, K. M., & Acker, J. D. (2008). Neuroanatomical correlates of fluid intelligence in healthy adults and persons with vascular risk factors. Cerebral Cortex, 18, 718–726.CrossRefGoogle ScholarPubMed
Ritchie, K., Ropacki, M., Albala, B., Harrison, J., Kaye, J., Kramer, J., Randolph, C., & Ritchie, C. W. (2017). Recommended cognitive outcomes in preclinical Alzheimer’s disease: Consensus statement from the European prevention of Alzheimer’s dementia project. Alzheimer’s & Dementia, 13, 186–195.CrossRefGoogle ScholarPubMed
Sheehan, B. (2012). Assessment scales in dementia. Therapeutic Advances in Neurological Disorders, 5, 349–358.CrossRefGoogle ScholarPubMed
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1–32.CrossRefGoogle ScholarPubMed
Singh, V., Chertkow, H., Lerch, J. P., Evans, A. C., Dorr, A. E., & Kabani, N. J. (2006). Spatial patterns of cortical thinning in mild cognitive impairment and Alzheimer’s disease. Brain, 129, 2885–2893.CrossRefGoogle ScholarPubMed
Sliwinski, M. J. (2008). Measurement-burst designs for social health research. Social and Personality Psychology Compass, 2, 245–261.CrossRefGoogle Scholar
Sliwinski, M. J., Mogle, J. A., Hyun, J., Munoz, E., Smyth, J. M., & Lipton, R. B. (2018). Reliability and validity of ambulatory cognitive assessments. Assessment, 25, 14–30.CrossRefGoogle ScholarPubMed
Smith, P. J., Need, A. C., Cirulli, E. T., Chiba-Falek, O., & Attix, D. K. (2013). A comparison of the Cambridge automated neuropsychological test battery (CANTAB) with “traditional” neuropsychological testing instruments. Journal of Clinical and Experimental Neuropsychology, 35, 319–328.CrossRefGoogle ScholarPubMed
Smyth, J. M., & Stone, A. A. (2003). Ecological momentary assessment research in behavioral medicine. Journal of Happiness Studies, 4, 35–52.CrossRefGoogle Scholar
Snitz, B. E., Tudorascu, D. L., Yu, Z., Campbell, E., Lopresti, B. J., Laymon, C. M., Minhas, D. S., Nadkarni, N. K., Aizenstein, H. J., Klunk, W. E., Weintraub, S., Gershon, R. C., & Cohen, A. D. (2020). Associations between NIH Toolbox Cognition Battery and in vivo brain amyloid and tau pathology in non-demented older adults. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 12, e12018.Google ScholarPubMed
Sperling, R. A., Aisen, P. S., Beckett, L. A., Bennett, D. A., Craft, S., Fagan, A. M., Iwatsubo, T., Jack, C. R., Kaye, J., Montine, T. J., Park, D. C., Reiman, E. M., Rowe, C. C., Siemers, E., Stern, Y., Yaffe, K., Carrillo, M. C., Thies, B., Morrison-Bogorad, M., . . . Phelps, C. H. (2011). Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on aging-Alzheimer’s association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimer’s & Dementia, 7, 280–292.CrossRefGoogle ScholarPubMed
Su, Y., D’Angelo, G. M., Vlassenko, A. G., Zhou, G., Snyder, A. Z., Marcus, D. S., Blazey, T. M., Christensen, J. J., Vora, S., Morris, J. C., Mintun, M. A. & Benzinger, T. L. (2013). Quantitative analysis of PiB-PET with freesurfer ROIs. PloS One, 8, e73377.CrossRefGoogle ScholarPubMed
Su, Y., Flores, S., Hornbeck, R. C., Speidel, B., Vlassenko, A. G., Gordon, B. A., Koeppe, R. A., Klunk, W. E., Xiong, C., Morris, J. C., & Benzinger, T. L. (2018). Utilizing the Centiloid scale in cross-sectional and longitudinal PiB PET studies. NeuroImage: Clinical, 19, 406–416.CrossRefGoogle ScholarPubMed
Su, Y., Flores, S., Wang, G., Hornbeck, R. C., Speidel, B., Joseph-Mathurin, N., Vlassenko, A. G., Gordon, B. A., Koeppe, R. A., Klunk, W. E., Jack, C. R., Farlow, M. R., Salloway, S., Snider, B. J., Berman, S. B., Roberson, E. D., Brosch, J., Jimenez-Velazques, I., van Dyck, C. H., . . . Benzinger, T. L. (2019). Comparison of Pittsburgh compound B and florbetapir in cross-sectional and longitudinal studies. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 11, 180–190.Google ScholarPubMed
Thompson, L., Harrington, K., Roque, N., Strenger, J., Correia, S., Jones, R., Salloway, S., & Sliwinski, M. (2022). A highly feasible, reliable, and fully remote protocol for mobile app-based cognitive assessment in cognitively healthy older adults. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 14, e12283. https://doi.org/10.1002/dad2.12283 Google ScholarPubMed
Van Strien, N. M., Cappaert, N. L. M., & Witter, M. P. (2009). The anatomy of memory: An interactive overview of the parahippocampal–hippocampal network. Nature Reviews Neuroscience, 10, 272–282.CrossRefGoogle ScholarPubMed
Weintraub, S., Besser, L., Dodge, H. H., Teylan, M., Ferris, S., Goldstein, F. C., Giordani, B., Kramer, J., Loewenstein, D, Marson, D., Mungas, D., Salmon, D., Welsh-Bohmer, K., Zhou, X-H., Shirk, S. D., Atri, A., Kukull, W. A., Phelps, C., & Morris, J. C. (2018). Version 3 of the Alzheimer disease centers’ neuropsychological test battery in the uniform data set (UDS). Alzheimer Disease and Associated Disorders, 32, 10.CrossRefGoogle ScholarPubMed
Weintraub, S., Salmon, D., Mercaldo, N., Ferris, S., Graff-Radford, N. R., Chui, H., Cummings, J., DeCarli, C., Foster, N. L., Galasko, D., Peskind, E., Dietrich, W., Beekly, D. L., Kukull, W. A., & Morris, J. C. (2009). The Alzheimer’s disease centers’ uniform data set (UDS): The neuropsychological test battery. Alzheimer Disease and Associated Disorders, 23, 91.CrossRefGoogle Scholar
Wilks, H. M., Aschenbrenner, A. J., Gordon, B. A., Balota, D. A., Fagan, A. M., Musiek, E., Balls-Berry, J., Benzinger, T. L. S., Cruchaga, C., Morris, J. C., & Hassenstab, J. (2021). Sharper in the morning: Cognitive time of day effects revealed with high-frequency smartphone testing. Journal of Clinical and Experimental Neuropsychology, 43, 825–837. https://doi.org/10.1080/13803395.2021.2009447 CrossRefGoogle ScholarPubMed
Woodford, H. J., & George, J. (2007). Cognitive assessment in the elderly: A review of clinical methods. QJM: An International Journal of Medicine, 100, 469–484.CrossRefGoogle ScholarPubMed
Woods, S., Delis, D., Scott, J., Kramer, J., & Holdnack, J. (2006). The California verbal learning test – second edition: Test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms. Archives of Clinical Neuropsychology, 21, 413–420.CrossRefGoogle ScholarPubMed
Supplementary material: File

Nicosia et al. supplementary material

Tables S1-S2 and Figures S1-S3

Download Nicosia et al. supplementary material(File)
File 577.2 KB