Evaluating the quality of care received in long-term care facilities from a consumer perspective: development and construct validity of the Consumer Choice Index – Six Dimension instrument

Abstract

The Consumer Choice Index – Six Dimension (CCI-6D) is a new instrument designed specifically to evaluate the quality of care received in long-term care from a consumer perspective. This study aims to demonstrate the construct validity of the CCI-6D. Older residents living in long-term care facilities and proxy family carers (where severely impaired cognition precluded resident consent) participated as consumers of long-term care. Data collected included the CCI-6D instrument, quality of life, physical function and characteristics of the care facility. Relationships between these variables and the CCI-6D dimensions were assessed and analysed through chi-squared and Kruskal–Wallis tests to assess the construct validity of each dimension. Of 430 eligible consumers, a total of 253 completed the questionnaire, of whom 68 (27%) were residents and 185 (73%) were informal carer proxy participants. There was strong evidence of construct validity of the dimensions relating to adequacy of individual care time, access to outside and gardens, access to meaningful activities and flexibility of care. There was more moderate evidence of validity of the home-like own room and shared spaces items, which may be in part due to difficulty in identifying strong discriminatory variables for comparison with these items. The results also indicate a strong association between ‘processes’ of care delivery (as measured by the CCI-6D) and quality of life of care recipients.

Background

Long-term care for older people can be described as support and care activities undertaken to ensure people with loss of function and capacity are able to maintain their wellbeing (Organisation for Economic Co-operation and Development (OECD) 2005). This can encompass care provided within the individual's own home, or care provided in institutional settings (nursing or aged-care homes). The long-term care sectors of many countries are going through processes of transformation, partly in response to increasing calls to change from a ‘one-size-fits-all’ approach to a flexible system with care individualised to the needs and preferences of the consumer (Alakeson 2010; Glendinning et al. 2008). In discussing the consumers of long-term care, it should be acknowledged that this encompasses residents themselves, first and foremost, but also can be considered to include the informal family member carers of residents. Family members are often highly involved in providing informal care and decision making for health and social care for older people in residential care, especially where the resident has severe cognitive impairment or where decision making is formally passed on to them (World Health Organization 2012). Cognitive impairment is highly prevalent among long-term care users (Bjork et al. 2016; Ribbe et al. 1997). Currently over half of residents in Australian long-term care facilities have a diagnosis of dementia (Australian Institute of Health and Welfare 2016).

Long-term care costs remain a significant source of public expenditure, varying from 0.2 to 3 per cent of Gross Domestic Product in OECD participating countries, with institutionally based care (such as nursing or residential homes) accounting for the greatest proportion of costs (OECD 2005). As with all public monies, there is a need to ensure that this is a worthwhile use of these funds to ensure ‘value for money’ for the public (Drummond et al. 2005). In the health sector, such considerations of worthwhileness have traditionally included the perspectives of the general population, since in many countries health systems are financed by significant funding from governments using tax revenue ultimately generated from its citizens (Brazier et al. 2017). Increasingly for the health and social care sectors, however, the views of care recipients and in some cases informal carers are being considered important to this process due to the unique perspective and increased familiarity they have with the benefits of these services, in addition to the fact they are being asked increasingly to co-fund their care in some countries (Brazier et al. 2017; Coast 2004). Measuring the quality of care provided in these services is an important component to the promotion of efficiency and ensuring the appropriate use of public expenditures (Castle and Ferguson 2010). Although many definitions of quality of care exist, salient features of the quality of care in this context may include the organisational culture, medical/clinical care, physical, cognitive and emotional functioning, psycho-social support, and fulfilment of the resident's basic rights including dignity, autonomy and privacy (Castle and Ferguson 2010; Kane 2001; Killett et al. 2016).

Donabedian (1985) proposed a theoretical framework for indicators of quality of care including structures (i.e. organisational characteristics associated with provision of care), processes (i.e. tasks undertaken with or for the resident) and outcomes (i.e. the desired states the care is aiming to achieve). Traditionally, the concepts of quality of care adopted in long-term care settings have not strongly incorporated the resident's own voice and have tended to focus predominantly on the measurement of organisational and clinical processes as quality indicators (Castle and Ferguson 2010; Nakrem et al. 2009). Concerns have been raised that such indicators produce a high level of focus on paper compliance rather than activities which are supporting the resident's wellbeing, and that structural and clinical care focused measures are not well correlated with improvement in quality of life for residents (Kane 2001; Rahman and Applebaum 2009).

With the recent introduction of consumer-directed care in the Australian aged-care sector and other similar policy initiatives internationally, the perspectives of consumers of long-term care services have become increasingly important to concepts of care quality (Castle and Ferguson 2010). Previous studies have indicated aspects of care provision in long-term care that are of importance to residents and their family members, including characteristics of the care provided and the staff providing the care, support for the personhood of the individual, the physical and social environment, and level of autonomy for the individual (Bergland and Kirkevold 2006; Custers et al. 2012; Kane et al. 1997; Schenk et al. 2013).

In order for long-term care facilities to better meet the needs and expectations of residents and their family member carers, a set of quality indicators are needed to evaluate the quality of care from their perspective. Previous work has indicated that instruments specifically designed for this purpose, for application with consumers in long-term care facilities, may perform better than generic instruments, such as those designed for measuring generic health-related quality of life (Forder and Caiels 2011).

The Consumer Choice Index – Six Dimension (CCI-6D) instrument was designed to fill this gap. The CCI-6D represents the first instrument in Australia and internationally developed from its inception with consumers (including both residents and family member carers) to evaluate the quality of care received in long-term care facilities from a consumer perspective. The aim of this paper is to describe the process of development and assessment of the validity of the CCI-6D instrument descriptive system.

Methods

Development of the CCI-6D

A five-stage process was undertaken to develop the CCI-6D instrument following the stages described by Stevens and Palfreyman (2012). An initial literature review highlighted that the vast majority of conceptual frameworks had been developed without any co-production or co-development involving people with dementia. Hence we firstly undertook a qualitative study with people with dementia, on the key determinants of quality in long-term care facilities, to identify potential candidate dimensions (stage 1) (see Milte et al. 2016). The first four stages of development of the CCI-6D instrument are described in detail in elsewhere, but in brief we undertook a data collection (stage 2) via one-to-one interviews with residents supplemented by a series of focus groups undertaken with family members of the people with more moderate and severe cognitive impairment, using a semi-structured approach (stage 3) with questions focusing around the characteristics in a care home that would make it a place people either wanted to reside in or did not. The data were analysed (stage 4) in batches as the data collection progressed until ‘data saturation’ (i.e. the point where no new themes within the data were identified) was achieved. A number of key dimensions were identified in the analysis, including ‘choice, freedom and self-determination’, ‘access to meaningful activities’, ‘feeling useful and valued’, ‘respect of possessions and personal space’, ‘individualised care’, ‘supportive health care and medical management’, ‘home-like spaces’ and ‘supportive safety techniques’ (Milte et al. 2016). These dimensions were also found to align with the concepts of individualised care (Chappell, Reid and Gish 2007), personhood for dementia care (Kitwood and Bredin 1992), and flexibility and autonomy in resident daily life (Boyle 2004). As the instrument needed to be parsimonious for ease of use whilst also encapsulating the key dimensions of importance, these key concepts were distilled down by the research team to those that were most often and consistently raised, to generate six candidate dimensions for inclusion within the CCI-6D instrument. These six dimensions were then presented to our project advisory group (N = 10, comprising consumer representatives from the Alzheimer's Australia Consumer Dementia Research Network (who were informal care-givers), clinicians, health services researchers and representatives of long-term care providers) for confirmation of consumer-friendly wording and face validity.

The fifth and final stage involved generating the descriptive system. The CCI-6D instrument comprises six main dimensions reflecting quality of care from the consumer perspective: (a) time spent attending to individual needs (Care Time), (b) home-like and non-institutional shared spaces (Shared Spaces), (c) home-like and non-institutional own rooms (Own Room), (d) access to outside and garden space (Outside and Gardens), (e) access to participation in meaningful activities individualised for the person (Meaningful Activities), (f) flexibility in the time that care activities were undertaken (Care Flexibility). These dimensions were operationalised as the six questions of the CCI-6D instrument, each with three different levels of measurement ranging from low to high levels of performance. In addition, the research team plans to develop a preference-weighted scoring algorithm, to complement the descriptive system of the CCI-6D as a second stage of this project, to enable incorporation of the instrument into an economic evaluation framework.

Approach for assessment of validity

For a newly developed instrument such as this, it is important to evaluate whether the instrument measures what was originally intended (i.e. in the case of the CCI-6D the quality of care in a long-term care facility). There are a number of different types of validity, including content, construct and criterion-related validity (Bowling 2005). Content validity refers to the extent to which the content of the instrument appears to be measuring the full scope of the characteristics it is intended to measure. This was provided initially through our qualitative interview process with older people and family member carers, and assessed through consultation with our project advisory group later. Criterion validity refers to the evaluation of a new instrument against an existing gold-standard, however, in the case of the quality of care received in long-term care facilities there is no such recognised instrument. Accordingly, our assessment of validity for the CCI-6D follows that taken by Coast et al. (2008) and Malley et al. (2012), and focuses on the construct validity of each of the dimensions, i.e. whether the six dimensions of the new instrument exhibited expected relationships with other variables and concepts. Previous studies have noted that for preference-weighted instruments the requirements for assessing validity differ from the method for psychometric assessment of instruments more generally (Brazier, Deverill and Green 1999; Coast et al. 2008; Malley et al. 2012). That is, it requires consideration of the validity of the instrument's content or its descriptive system, the valuations for the preference-weights and the empirical validity of the final preference-weighted scores. This paper aims to evaluate the validity of the CCI-6D descriptive system. The validity of the valuations and the empirical validity of the final preference-weighted scores are to be reported upon in a subsequent paper.

Study design

This study was nested in a large cross-sectional, observational study (INSPIRED) designed to evaluate the specialised dementia care currently being provided in Australian nursing homes. Participants were recruited from nine nursing homes across Australia, including both metropolitan and rural locations, over an eight-month period in 2015–2016. The study was approved by the Flinders University Social and Behavioural Research Ethics Committee (project numbers 6732 and 6753). Eligibility criteria included that the resident had been living in the home for at least 12 months, and was not currently receiving palliative care. Prior to completing the CCI-6D questionnaire, residents were administered the Psychogeriatric Assessment Scales – Cognitive Impairment Scale (PAS-Cog) by a trained research nurse, a cognitive functioning assessment tool with excellent reliability and validity, and widely used in the Australian long-term care sector for assessing cognition (Jorm et al. 1995). The PAS-Cog is scored on a scale between 0 and 21, where a higher score indicates greater cognitive impairment (Jorm et al. 1995). A score equal to or greater than 5 was shown to identify more than 80 per cent of dementia cases (Jorm et al. 1995). Scores can be further categorised to indicate no or minimal cognitive impairment (scores 0–3), mild impairment (4–9), moderate impairment (10–15) and severe impairment (16–21) (Department of Health 2017). Residents with no to mild cognitive impairment (indicated by a PAS-Cog score of between 0 and 9) were then asked to self-consent and self-complete the CCI-6D. Where residents had a more severe level of cognitive impairment (i.e. either indicated by a PAS-Cog score greater than 9 or judgement of a trained data collector that they did not understand the requirements of the study), a suitable proxy was approached to give informed consent. For those cases of proxy consent, questionnaires were either completed by the resident if able, or by a proxy completing on behalf of the resident in those cases where the resident had more severe cognitive impairment which prevented them from completing the questionnaire. For this study, a proxy assessor was defined as a person who had a close relationship with the individual, who visited the person regularly and assisted with making decisions on their behalf. This was usually a spouse, sibling or offspring of the individual. Following informed consent, participants took part in a face-to-face interview with trained data collectors. If a proxy participant was unable to attend a face-to-face interview, arrangements were made for them to complete the questionnaire via telephone or post. Data collected from the resident and/or proxy included the CCI-6D, quality of life (as measured by the Euroqol 5 Dimension 5 Level instrument (EQ-5D-5L) utility score; Devlin et al. 2016), dimensions of dementia-related quality of life as relevant (measured by the DEMQOL or DEMQOL-Proxy instruments; Mulhern et al. 2013) and social-care related quality of life (as measured by the Adult Social Care Outcomes Toolkit (ASCOT) – Care Home Version 3 Resident or Proxy instrument as required; Towers et al. 2016). Data collected from staff and records of the facility included physical function (Modified Barthel Index; Shah, Vanclay and Cooper 1989), frequency of access to ‘outside and gardens’ for the resident in the past week, and whether the resident had brought in their own furniture for furnishing their room, using standardised questions drawn from a previous study of nursing homes for people with dementia (Palm et al. 2013). Hypothesis testing was used to assess the construct validity of the CCI-6D descriptive system and its expected relationships with variables (see Table 1). These expected relationships were drawn from evidence in the literature where possible, or from the opinions of experts in the research team in the area of person-reported outcomes, and aged care, following a previous methodology (Malley et al. 2012).

Table 1. Variables included and expected associations

Notes: ASCOT: Adult Social Care Outcomes Toolkit. EQ-5D-5L: Euroqol 5 Dimension 5 Level instrument. CCI-6D: Consumer Choice Index – Six Dimension.

Statistical analysis

To assess for relationships between variables of interest, we used Pearson's chi-squared tests (for categorical variables) with use of related tests where appropriate (e.g. Yate's correction for continuity which compensates for the overestimate of the chi-square value in two-by-two tables). These tests have an assumption of minimum expected cell frequency (i.e. that 80 per cent or more of the cells should have a frequency of five or more). Where this assumption was violated, the maximum likelihood ratio chi-square test was used (McHugh 2013). The Kruskal–Wallis test was used to assess for relationships for continuous variables, with Mann–Whitney U-tests undertaken post hoc to test for differences between groups with Bonferoni adjustment. All analysis was undertaken using IBM SPSS Statistics Version 23. Associations significant at the 1 per cent level were taken to be highly suggestive of a relationship between the dimension and the variable. Associations at the 5 per cent level were taken to be weakly suggestive of a relationship between the dimension and the variable.

Results

A total of 661 residents were assessed for eligibility in nine nursing homes across five aged-care organisations in four states of Australia. Of these, 430 (65%) residents were deemed eligible and 254 (59%) consented to participate. Of those who declined participation, the most common reasons were being too busy to participate (26.69%) or a lack of interest in the study (19.32%). One participant did not undertake the CCI-6D questionnaire and was excluded from the analysis; 253 participants undertook a CCI-6D questionnaire and are included in the following analyses, of whom 68 (27%) were residents of long-term care facilities and 185 (73%) were proxy participants due to the severe cognitive impairment of the resident precluding their participation. Basic demographic characteristics of the sample are shown in Table 2.

Table 2. Socio-demographic characteristics of participants

Notes: N = 253. 1. Unless otherwise specified. EQ-5D-5L: Euroqol 5 Dimension 5 Level instrument. SD: standard deviation. PAS-Cog: Psychogeriatric Assessment Scales – Cognitive Impairment Scale.

For the CCI-6D questionnaire, 12 participants had one missing response and one participant each had two or three missing responses. These 14 participants were included in the study with a value imputed for the missing responses based on the mean of their responses to the remaining questions. The responses to the CCI-6D questionnaire are summarised in Table 3 for the total sample, and resident and proxy participants are presented separately. Over half of the participants reported they have the highest level of choice and flexibility in the dimensions of Care Time, Care Flexibility, Shared Spaces and Own Room. Participants indicated more variability and generally lower levels of perceived choice and flexibility in the dimensions of Outside and Gardens and Meaningful Activities. There were statistically significant differences in the distribution of responses between residents and proxy participants for the dimensions Own Room, Outside and Gardens and Meaningful Activities, with a higher proportion of proxy participants indicating lower levels of choice and flexibility in these areas. There were no statistically significant differences in the distribution of responses between proxy and resident participants for the other dimensions.

Table 3. Responses of participants to the Consumer Choice Index – Six Dimension (CCI-6D) questionnaire

Notes: 1. Pearson's chi-squared was used to generate p-values. 2. Or your family member (as appropriate for proxy participants).

CCI-6D and quality of life

We hypothesised there would be strong relationships between all of the CCI-6D dimensions and utility scores for social care-related quality of life (ASCOT) due to its focus on the long-term care setting, with relationships present for health-related quality of life (EQ-5D-5L) due to its generic nature and lack of specificity for the long-term care setting. This hypothesis was developed drawing upon the Donabedian model for assessing quality of health care, which proposed that care quality indicators fall into three main categories, i.e. those focusing on the organisational structures supporting care (structure), the processes through which care is provided (process) and the ultimate outcomes seen in recipients as a result of care (outcomes) (Donabedian 1985). This model suggests that relationships between the three categories are linear in nature, i.e. that good structures of care will lead to good care processes, and that good processes for care will lead to good outcomes (Castle and Ferguson 2010). This link has been further supported more recently, with improvements in outcomes (such as quality of life) identified following improvements to the structures or processes of care provided in nursing homes (Brownie and Nancarrow 2013; Havig et al. 2011; Kane et al. 2007). On the basis of this previous evidence, we hypothesised that improvements in the processes of care, as assessed by the CCI-6D instrument, would be positively associated with higher levels of wellbeing and health and social care-related quality of life for care recipients (as the ultimate outcome of long-term care services). Overall, this hypothesis was supported (see Table 4), with strong relationships identified between social care-related quality of life (ASCOT) and all dimensions of the CCI-6D, and a strong relationship between health-related quality of life (EQ-5D-5L) and two dimensions (Outside and Gardens and Meaningful Activities). There was weaker evidence of a relationship between the EQ-5D-5L and the dimensions Care Time, Own Room and Care Flexibility. There was no evidence of a relationship between EQ-5D-5L and the Shared Spaces dimension. Post hoc tests identified the difference in EQ-5D-5L scores reported for those participants who selected the highest and middle response levels of the Care Time dimension reached statistical significance. Significant differences were also found in EQ-5D-5L scores for the participants selecting the highest and middle response levels of the Outside and Gardens and Meaningful Activities dimensions. Similarly, the difference in EQ-5D-5L scores reported for the highest and lowest response levels for the dimensions Care Time, Outside and Gardens and Meaningful Activities also reached statistical significance. Significant differences in the EQ-5D-5L scores were also found for the middle and lowest response levels of Meaningful Activities dimension, and the highest and middle response levels of the Care Flexibility dimension. Significant differences were identified between social care-related quality of life (ASCOT) scores reported for those participants who selected the highest and lowest response levels of dimensions Care Time, Shared Spaces, Own Room, Meaningful Activities and Care Flexibility. There was also a significant difference in ASCOT scores reported for the participants selecting the highest and middle response levels of the same dimensions. There were also significant differences in ASCOT scores for those selecting the highest and lowest and middle and lowest response levels of the Outside and Gardens dimension.

Table 4. Mean Euroqol 5 Dimension 5 Level instrument (EQ-5D-5L), Adult Social Care Outcomes Toolkit (ASCOT), Modified Barthel Index scores and age by Consumer Choice Index – Six Dimension (CCI-6D) dimension

Notes: 1. Or your family member (as appropriate for proxy participants). 2. Kruskal–Wallis test was used to generate p-values. SD: standard deviation.

CCI-6D and resident function and age

We identified the potential for a relationship between the CCI-6D and age and physical function (Modified Barthel Index). There was a strong relationship between responses to the dimensions Outside and Gardens and Meaningful Activities and physical function (Modified Barthel Index), and a weak relationship with Own Room dimension, but not with the remaining dimensions. Relationships were in the expected direction (i.e. a higher level of physical function for those indicating better access to Outside and Gardens and Meaningful Activities) and post hoc tests indicated significant differences in Barthel scores between the highest and middle and the highest and lowest response levels for both dimensions. Age did not show a significant relationship with responses to any of the dimensions, except for a weak relationship with access to Outside and Gardens.

The results of the tests of the hypothesised relationships between the individual variables and the dimensions of the CCI-6D are described below. The significance of the relationships between the six CCI-6D dimensions and the extent of control and flexibility in care, self-reported psychological wellbeing, social and occupational engagement, and the physical institutional environment are shown in Table 5, with the nature and direction of the association reported in the text.

Table 5. Significance of relationship between Consumer Choice Index – Six Dimension (CCI-6D) dimensions and associated quality of life, and model of care characteristics (p-values¹)

Notes: 1. Pearson's chi-squared was used to generate p-values. ASCOT: Adult Social Care Outcomes Toolkit.

Care Time

As expected, we identified strong associations between the ASCOT Control dimension (p = 0.008) and the DEMQOL Social support dimension (p = 0.003) and the CCI-6D's Care Time dimension, with participants reporting the top level of Care Time generally also reporting better levels of control and social support. Strong associations were also shown between Care Time and ASCOT dimensions Social participation, Accommodation, Occupation and Dignity; DEMQOL dimensions Enjoying life; and DEMQOL-Proxy Hopes and aspirations for the future dimension, which were not hypothesised prior. There were no strong relationships between the Care Time dimension and physical care environment characteristics.

Shared Spaces

We identified a strong association between the CCI-6D Shared Spaces dimension and the ASCOT Accommodation (p = 0.001) and Social participation dimensions (p = 0.001). Relationships were in the expected direction, with higher levels of response to the home-like Shared Spaces associated with better responses on the ASCOT Accommodation and Social participation dimensions. We did not expect the strong associations that were found between the Shared Spaces dimension and the ASCOT dimensions of Control, and Dignity, the DEMQOL Enjoying life dimension, and the DEMQOL-Proxy Feeling useful dimension. There were no strong relationships between the dimension and any other variables.

Own Room

Residents bringing their own furniture in to furnish their room and the ASCOT Accommodation dimension were expected to be strongly related to the Own Room dimension. We identified strong relationships with the ASCOT Accommodation dimension (p ⩽ 0.001), but not with residents bringing their own furniture in to furnish their room (p = 0.377). Strong relationships were also found with the Own Room dimension and the ASCOT dimensions of Control and Occupation, and the DEMQOL-Proxy Feeling useful dimension, which were not previously hypothesised. Weaker associations were found with the Own Room dimension and the ASCOT Social participation dimension, DEMQOL Hopes and aspirations for the future dimension, and frequency of resident visiting the gardens.

Outside and Gardens

As expected, we found a strong relationship between the Outside and Gardens dimension and the frequency residents were reported to use the outside and garden area (p ⩽ 0.001). The relationship was in the expected direction (i.e. better responses to the Outside and Gardens dimension was associated with a higher frequency of access to outside and gardens by the resident). We also identified strong relationships between the CCI-6D dimensions and the Control, Social participation, and Occupation dimensions of the ASCOT, the DEMQOL Things that you want to do but can't dimension, and the DEMQOL-Proxy Hopes and aspirations for the future dimension.

Meaningful Activities

We identified strong relationships between the CCI-6D's Meaningful Activities dimension and the ASCOT's Control (p ⩽ 0.001), Occupation (p ⩽ 0.001), and Dignity (p = 0.004) dimensions. We also identified strong relationships between the dimension and the DEMQOL Enjoying life (p ⩽ 0.001), and DEMQOL-Proxy Hopes and aspirations for the future dimensions (p ⩽ 0.001). We did not find a relationship with the DEMQOL Things that you want to do but can't dimension (p = 0.391). We also found strong relationships that were not previously hypothesised between the CCI-6D dimension and the Social participation, and Accommodation dimensions (from the ASCOT), and the frequency of resident visiting the gardens variable.

Care Flexibility

We identified a strong relationship between the CCI-6D Care Flexibility dimension and the ASCOT Dignity dimension (p = 0.014), but no association between Care Flexibility and the Control dimension, or with the DEMQOL Social support dimension. However, we also identified a strong relationship between Care Flexibility and the ASCOT Accommodation dimension, and DEMQOL Enjoying life dimension, and a weak relationship with the DEMQOL Things that you want to do but can't dimension which was not hypothesised previously. The Care Flexibility dimension did not show a significant relationship with the other individual variables tested.

Discussion

This study has provided a preliminary assessment of the practicality and construct validity of the CCI-6D instrument for application in evaluating the quality of care of long-term care facilities from a consumer perspective. The vast majority of study participants were able to complete the CCI-6D instrument, and responses were dispersed across all three possible levels for each dimension comprising the instrument, demonstrating it is responsive to changes in the perception of care provision between individuals. In addition, there was evidence of a strong relationship with responses to the dimensions of the CCI-6D and social care-related quality of life, and a more moderate relationship with health-related quality of life, such that responses indicating greater flexibility and choice in care were associated with higher utility scores for the ASCOT and EQ-5D-5L instruments. This provides evidence of both the convergent validity of the CCI-6D instrument and of the clear associations first identified by Donabedian, between ‘process’ (reflected in the CCI-6D instrument) and ‘outcome’ (i.e. improved quality of life for individual residents) in defining the quality of care received in long-term care facilities from a consumer perspective.

Evidence has been presented for the construct validity of the CCI-6D dimensions (i.e. strong correlations with the expected variables) and discriminant validity (i.e. lack of relationship with variables not considered associated with that construct). Where relationships were not evident, reasonable explanations could be readily identified. We identified a strong correlation between the level of physical functioning of residents (as assessed by the Modified Barthel Index) and the dimensions of Outside and Gardens and Meaningful Activities, but not to the other dimensions. This finding may be explained by residents with greater physical functioning having greater access to outside and gardens and undertaking activities due to their greater independence, while those residents with functional impairments may find it more difficult to participate in activities offered as current practice in nursing homes. They may also be restricted in the activities they can undertake as a way of managing a risk of falls, or have lower levels of independence resulting in a reliance on others (e.g. staff) for access to the provided outside and garden areas. Given the high level of functional impairments by definition among residents, this level of dissatisfaction with these two dimensions highlights a need for action in these domains.

There are some limitations to this study which need to be considered. The study design facilitated the administration of the CCI-6D instrument to participants at a single time-point only. It was therefore not possible to assess the test re-test validity of the CCI-6D as a component of this study. Further research is needed to assess re-test validity and the responsiveness of the CCI-6D to change over time, a key requirement for assessing its potential for application as an outcome measure. We consider it very important to facilitate maximum participation of residents in evaluations of improvements to long-term care to ensure their voice is incorporated in design of services which directly impact upon them (Alzheimer Europe 2011; Castle and Ferguson 2010). However, residents of nursing homes often are experiencing a variety of cognitive, sensory and functional impairments which impact on their ability to respond to questionnaires of this nature (Helvik et al. 2015; Löfgren et al. 1993). Therefore, an important consideration for future work will be developing administration methods to facilitate involvement of residents with a greater variety of cognitive or functional impairments, e.g. through use of visual representations of the dimensions and their respective levels. Methods such as talking mats or visual representations of dimensions and levels have shown promise in this regard and should be investigated further (Murphy et al. 2005; Turnpenny et al. 2015).

In addition, as the majority of residents in long-term care facilities have functional and/or cognitive limitations, as expected a priori, a relatively large proportion of residents (75%) were unable to participate in the study themselves and therefore participation from a close relative was sought on their behalf. However, there are known systematic differences in the way that older people respond to questionnaires and how proxies respond even when they are asked to respond from the perspective of the older person (Alzheimer Europe 2011; Coucill et al. 2001). For example, self-reported quality of life has been found to be significantly higher for people with dementia relative to proxy assessors (Addington-Hall and Kalra 2001). The current study represented the first application of the CCI-6D in practice. It is important to explore the reliability of this measure in participants with cognitive impairment, with a view to supporting their participation in evaluating quality of care in the sector. Future research should focus on assessing the extent of inter-rater agreement between residents and their proxies, as well as focusing on determining the range of residents with differing levels of cognitive impairment that can be supported to offer self (as opposed to proxy) assessment in completing the CCI-6D instrument. Currently, there is some evidence for thresholds for self-completion for instruments measuring quality of life such as the DEMQOL (Smith et al. 2007) and the EQ-5D (Hounsome, Orrell and Edwards 2011), with these instruments showing acceptable reliability or completion rates among participants with mild or moderate cognitive impairment. However, the cognitive threshold for completion of an instrument is likely to depend upon its characteristics, including complexity of language and concepts presented, in addition to its method of administration (Sansoni et al. 2007). It can be postulated that the cognitive functioning threshold for self-completion may be lower for a simple and short questionnaire such as the CCI-6D, as compared to other more complex instruments.

Conclusions

The analyses presented in this paper present evidence of construct validity for the individual dimensions of the CCI-6D instrument, as assessed through the relationships with variables hypothesised to be associated with the dimensions of the instrument. Therefore, the CCI-6D shows promise in its ability to evaluate the quality of care processes from the perspective of consumers in long-term care facilities. The CCI-6D may be readily applied to evaluate the current levels of performance against key quality indicators identified by consumers for nursing home care (e.g. to provide a benchmark for the sector), or to evaluate the effect of systematic changes, interventions and quality improvement activities. As a measure of the quality of care processes in long-term care, the CCI-6D can be used in evaluations of innovations in long-term care to add to perspectives gained from the use of other more generically focused instruments, e.g. the ASCOT social care-related quality of life measure. In order to facilitate the potential for incorporation into an economic evaluation framework, the next stage of the research will involve the development of a preference-weighted scoring algorithm for application with the CCI-6D instrument.