Evidence of cognitive training in mild to moderate dementia

Summary

There is urgent need to search for a dementia treatment that can delay its progression and reduce its financial and societal burden. Despite lacking evidence, there is a large number of commercial brain-training products on the market that claim they improve cognition. The Cochrane review under consideration looks at whether cognitive training maintains or improves cognition in those with mild to moderate dementia compared with control and alternative interventions. This commentary puts its findings into clinical perspective.

LEARNING OBJECTIVES

After reading this article you will be able to:

• • recognise the impact that healthcare staff's work has on them and their patients, and how stress may originate

• • describe contemporary conceptual approaches to understanding the psychosocial experiences of healthcare staff and the components of good psychosocial care to mitigate their needs

• • understand how lessons from research and experience might be used to improve employers' evidence-informed capabilities for caring for their staff.

The defining feature of the early stages of dementia, particularly Alzheimer's disease, is cognitive impairment. Non-pharmacological interventions aiming to maintain or improve cognition are known as ‘cognitive-oriented treatments’. They include cognitive training (guided practice on a set of standard tasks designed to target specific cognitive functions), cognitive stimulation (engaging in a range of activities to improve general cognition and social functioning) and cognitive rehabilitation (addressing disability in daily functioning due to cognitive impairment).

A key assumption underlying cognitive training is that repeated practice can improve or at least maintain performance of specific cognitive functions, which will then generalise to improvement in functional outcomes (Lampit 2014).

The latest guidelines from the UK's National Institute for Health and Care Excellence (2018) recommend cognitive stimulation and cognitive rehabilitation for people with mild to moderate dementia, but not cognitive training. This is largely because identified evidence on cognitive training was limited and of low to moderate quality. This finding is similar to that of the Lancet Commission on Dementia Prevention, Intervention and Care (Livingston 2017).

Despite this, there is a burgeoning industry of commercial brain-training products that claim they improve cognition. Against these backgrounds, it is important that clinicians have the most up-to-date review of the literature on cognitive training for persons with mild to moderate dementia.

Summary of the Cochrane review

The review by Bahar-Fuchs et al (2019) analysed 33 randomised controlled trials (RCTs) of cognitive training compared with control conditions or alternative interventions involving a total of 1924 participants with mild to moderate dementia in 12 countries. They found that people completing cognitive training, compared with control conditions, may show some benefits in overall cognition, as well as in more specific cognitive abilities such as verbal fluency. These benefits lasted for at least a few months. However, compared with alternative treatments, they found no evidence that cognitive training was superior.

Definition of the clinical question

The population assessed comprised older adults with mild to moderate dementia (most commonly Alzheimer's dementia), mostly residing at home, and their primary caregivers. The intervention was cognitive training. When reported, each session lasted for 1–1.5 h, 1–5 times a week for 2 weeks to 12 months, with one trial having maintenance sessions every 6 weeks for almost 2 years.

Cognitive training was compared with either control conditions or alternative treatments. The control conditions were active (e.g. social support groups, activities similar to the experimental intervention but with a passive approach, unstructured conversation/discussion, unstructured or non-specific cognitive activity with the same intensity and duration as the experimental intervention) or passive (e.g. waiting list, no contact, placebo medication, or usual care). The alternative treatments included new medication, cognitive stimulation, cognitive rehabilitation, occupational therapy, reminiscence therapy, aerobic exercise, mindfulness, muscular relaxation and music therapy.

There were two primary outcomes (both were for the participants with dementia): (a) global cognition immediately after the intervention; and (b) clinical disease severity in the medium term (3–12 months after the intervention). Global cognition was evaluated using a composite score (Box 1) derived from all cognitive measures within each trial, plus an additional analysis of a screening tool, typically the Mini-Mental State Examination (MMSE).

BOX 1 Composite score

A composite score combines information on multiple individual measures into a single score, to simplify complex information and to provide more reliable data with less bias than would be obtained using a single screening measure such as the MMSE.

The secondary outcomes for the participants with dementia were: (a) global cognition in the medium term; (b) clinical disease severity immediately after the intervention; the following assessed both immediately and in the medium term: (c) domain-specific cognitive status, (d) self-reported and informant-reported meta-cognition, (e) mood, (f) ability to perform activities of daily living, (g) behavioural and psychological symptoms of dementia, and (h) general health or quality of life; and (i) participant burden as per retention rate immediately after the intervention. A further three secondary outcomes were for the primary caregivers: mood, burden of care and quality of life, immediately and in the medium term.

Method

Bahar-Fuchs et al searched a wide range of databases for RCTs on the above clinical question. These included ALOIS (the Specialised Register of the Cochrane Dementia and Cognitive Improvement Group), MEDLINE, Embase, PsycINFO, CINAHL, LILACS, the Web of Science Core Collection, ClinicalTrials.gov and the World Health Organization's portal ICTRP, with no restriction to dates. ALOIS contains records of clinical trials from various databases, including numerous trial registries and grey literature sources updated monthly. They screened reference lists and contacted experts in the field to request additional RCTs not identified by the search.

Screening procedure and risk-of-bias assessments were completed by two of the review authors, with disagreements resolved by a third reviewer. The overall quality of the evidence was assessed using the GRADE criteria, completed by two of the review authors, with disagreements resolved by discussions until a consensus was reached.

The authors used standardised mean differences (SMD) (Box 2) when the same outcome was assessed by different measures.

BOX 2 Standardised mean difference

When the same outcome is measured in various ways (e.g. using different psychometric tests) in different studies, the standardised mean difference (SMD) is used to standardise the results to a uniform scale. The SMD is the ratio of the difference in mean outcome between groups relative to the variability observed in the study:

$$ {\rm SMD} = \displaystyle{\hbox{Difference in mean outcome between groups}} \over {\hbox{Standard deviation of outcome among participants}} $$

SMD values are hard to interpret clinically but rules of thumb in their interpretation suggest that:

• 0.2 represents a small effect

• 0.5 represents a moderate effect

• 0.8 represents a large effect.

(Cohen 1988)

Results

When compared with control conditions, the authors found moderate-quality evidence showing a small to moderate effect of cognitive training on global cognition immediately after the intervention (SMD = 0.42, 95% CI 0.23–0.61) and high-quality evidence showing moderate effect on verbal semantic fluency (SMD = 0.52, 95% CI 0.23–0.81) immediately after the intervention and in the medium term.

When compared with alternative treatments, however, they found that cognitive training had little to no effect on global cognition immediately after the intervention (SMD = 0.21, 95% CI −0.23 to 0.64), but the quality of evidence was low. No evidence was available to assess the outcome of clinical disease severity in the medium term. One single trial of moderate quality showed cognitive training to be associated with improved mood of the caregiver at the end of treatment.

The quality of evidence measured using GRADE varied from very low to moderate. For most of the secondary outcomes, the majority of which were assessed only in the medium term, the quality of evidence was poor. Therefore, the authors were unable to determine whether cognitive training had any meaningful effect on these outcomes.

Discussion

The study population

The evaluation of 33 RCTs with a total of 1924 participants across 12 countries makes this the largest systematic review of this topic to date. Interestingly, 11 out of the 33 studies were conducted in Italy. No eligible studies were identified in primarily English-speaking countries such as the UK, Canada or Australia and no eligible studies were identified in the USA since 2003. Therefore, the extent to which the results of this review apply to individuals in other countries remains unclear.

The sample was mostly representative of patients with mild to moderate dementia, Alzheimer's type, residing in the community. However, the authors also included RCTs with <15% of participants with questionable or severe dementia. A small proportion of participants were living in nursing home facilities, which may indicate greater disease severity. In a few cases, this was corroborated by MMSE scores of as low as 7. Studies with participants residing long-term in psychiatric hospitals were excluded, and there were no details about dementia in people with intellectual disabilities.

Risk of bias

Many of the RCTs provided insufficient details on allocation concealment and randomisation processes. This can introduce selection bias whereby, for example, more unwell participants may be assigned to the intervention arm rather than the control arm. Nevertheless, randomisation and allocation concealment (Box 3) can still be adequately performed in psychosocial RCTs. Unlike in the evaluation of drug treatments, in which the highest level of evidence is double-blinded RCTs (Box 3), masking (blinding) of intervention is not possible in passive-control (e.g. usual care) psychosocial RCTs, although it can be attempted in active-control or alternative treatment RCTs. If masking is inadequate and participants in the placebo group are aware of their status, they could become demotivated and unwilling to perform their best (performance bias), which may have an impact on the outcome measures. Masking of outcome assessments can still be done adequately, but in 25% of the RCTs the outcomes were assessed by unmasked personnel. This may lead to detection bias.

BOX 3 Masking (blinding) and allocation concealment

Masking (blinding) is a way of preventing researchers, clinicians and patients in a clinical trial from knowing which study group each patient is in, so that they cannot influence the results.

In a single-blind study, patients are unaware of which study group they are in. In a double-blind study, neither the patients nor the researchers/doctors know which study group the patients are in. In a triple-blind study, the patients, clinicians and researchers do not know which treatment patients are getting.

Allocation concealment is a different concept, whereby trial investigators who are assigning participants to groups are kept unaware of the allocation sequence in the randomisation process, so that they cannot influence the allocation of participants. Allocation concealment is possible in all types of trial, even in non-blinded ones.

Clinical heterogeneity

The authors acknowledged that it can be difficult to differentiate different types of cognitive-oriented treatment (COT). Trials of complex treatments that have elements of cognitive stimulation or rehabilitation were included only if the review authors agreed by consensus that cognitive training was clearly the predominant component. However, if the intervention being compared was similar to the alternative treatment (e.g. mixed COT compared with a specific COT), the study may have been underpowered to detect an effect.

The cognitive training interventions were clinically heterogeneous. Some targeted a single domain whereas others targeted multiple domains. The mode of delivery (pencil and paper versus computer), settings (home versus hospital/out-patient), format (individual versus group), frequency, doses and duration (2–104 weeks) of the interventions also varied widely.

The observed clinical heterogeneity most likely contributed to statistical heterogeneity. In fact, large and moderately significant heterogeneity was the most common reason for downgrading of immediately post-intervention evidence of cognitive training compared with control conditions. Despite this, several pre-specified subgroup analyses performed to explore potential effect modifiers (e.g. type of cognitive training, duration, dose) found none to be significant enough to warrant a separate meta-analysis.

Statistical analyses and outcomes

There was no mention of a subgroup analysis on the format of intervention (group versus individual). It would be interesting to see whether this influenced the effect of the intervention, as group cognitive stimulation has been shown to be more effective than individual cognitive stimulation, leading to interpretations that significant benefits are derived from the social aspect of being in a group rather than its content (Livingston 2017). On a related note, assessment of therapeutic relationship when there is a consistent facilitator might shed more light into its role in such interventions. Assessment of the level of experience of the facilitators may also be helpful to see whether it influences the effectiveness of the intervention.

The review authors conducted a random-effects meta-analysis (focusing on the average treatment effect estimate and its 95% confidence interval), as this method allows for heterogeneity in treatment from study to study (Box 4). It might have been helpful to have had a 95% prediction interval (Box 4) to quantify the potential effect of the treatment delivered in an individual study setting, as this may be different from the average effect across settings.

BOX 4 Random-effects model and 95% prediction interval in meta-analysis

A random-effects meta-analysis model allows for heterogeneity between studies.

It assumes that observed estimates of treatment effect can vary across studies owing to real differences in the treatment effect in each study and not just owing to pure chance.

In terms of interpreting the results in practice, it is important to note that the summary result gives the best estimate of the average treatment effect, not the most likely effect. Therefore, the average treatment effect may be different in an individual setting. Calculating a 95% prediction interval helps to quantify whether the effects will always remain consistent in individual treatment settings.

(Riley 2011)

The primary outcome of global cognition was assessed using a composite score (Box 1) from all cognitive measures. Although this method allowed for more trials and total participants to be included, which increased statistical power and confidence in the findings, the use of composite scores in general can be problematic as components can be unreasonably combined, inconsistently defined and inadequately reported (Cordoba 2010). The review authors reported that they generally used data from a test that was most commonly used in other studies, preferring established measures. It might have been helpful to have had a list of the measures used to provide clarity for the reader.

It is worth noting that the second primary outcome – change in disease progression in the medium term (3–12 months), which may be more clinically relevant – was studied in only two trials with a total of 98 participants and using a dementia severity rating scale. Having a small sample increases the likelihood of a type 1 error (or false positives), so the authors could not be certain of the findings. In fact, sample size for outcomes in the medium term were frequently small, and imprecision (caused by this and confidence intervals crossing the ‘no effect’ threshold) was the most common reason for downgrading of medium-term evidence.

Clinical relevance of the key finding

The key assumption underlying cognitive training – that practice in a given task can lead to generalised improvements beyond training context into functional outcomes – is not reflected in the findings of this review. The authors are moderately confident that cognitive training did not lead to any improvements in ability to perform activities of daily living, mood, or behavioural and psychological symptoms of dementia. This is clinically relevant because as dementia progresses, individuals are more likely to face greater functional decline and increasing behavioural and psychological symptoms of dementia. This suggests that cognitive training may not delay the psychosocial impact of dementia progression in the medium term.

As features of dementia progression include decline in functioning, it may be helpful for future studies to explore the effects of cognitive training on independence (by measuring increase in package of care and admission to residential care), as well as other clinically relevant outcomes such as increase in the dosage and augmentation of dementia drugs, quality of life and caregiver burden over a longer period after the intervention.

Treatment adherence and harm versus benefit

The authors recognised that, although adherence to lifestyle interventions is a common problem, especially with unsupervised, home-based interventions in a population of individuals with cognitive impairment, adherence was referred to by only six recent studies. Therefore, interpretation of the results may be biased owing to inadequate adherence. Similarly, all but four did not mention steps taken to ensure fidelity to intervention. This can create problems in replicating psychosocial intervention studies.

However, cognitive training was not associated with increased participant burden, as evidenced by the discontinuation rates, although the quality of the evidence was low. This may be an important finding, as such interventions can have unintended side-effects, for example fatigue and frustration, which may lead to participants dropping out of treatment. It may give cognitive training an advantage over approved medications that are associated with adverse effects. It may be helpful for future studies to include evaluations of the number needed to harm (NNH) and number needed to treat (NNT) to allow for more direct clinical interpretation of harm versus benefits of cognitive training.

Conclusions

As individuals with dementia face various levels of difficulties due to symptoms in areas including cognition, neuropsychiatric symptoms and activities of daily living, a holistic approach to treatment focusing on the needs of the individual at the stage of their illness is necessary. The extent to which the observed effects of cognitive training on global cognition and functioning in the medium term may be interpreted as being clinically meaningful is small owing to the poor quality of evidence. The decision to enrol patients into cognitive training should therefore be made in the wider context of other alternative interventions, bearing in mind its very modest benefits and possible lack of local availability.