P76: Agreement between nursing-home caregivers’ observations of residents’ depression, wellbeing, and quality of life

Abstract

Objective:

In nursing home residents, outcomes are often assessed using proxies, especially in residents with severe cognitive problems. Although caregivers are commonly involved as proxies, studies assessing their agreement for proxy measures are scarce. Therefore, secondary analysis was performed on a dataset with proxy-reported scores of several scales in Dutch and Flemish nursing home residents with and without dementia.

Methods:

To assess the agreement between the observations of 81 pairs of caregivers, we calculated Cohen’s Kappa, Weighted Kappa, and Prevalence- and Bias-Adjusted Kappa (PABAK and PABAK- OS for ordinal data) coefficients for the items on the Nijmegen observer-rated depression scale for detection of depression in nursing home residents (NORD), the social wellbeing of nursing-home residents scale (SWON-3), and two subscales (i.e., “social relations” and “having something to do”) of the QUALIDEM. In addition, coefficients were calculated for the item concerning subjective judgment of the residents’ depressive symptoms (“no,” “yes, mild or light,” or “yes, severe”) and for the item concerning whether the caregivers believed a depression diagnosis had been established (“yes,” “no”).

Results:

In general, PABAK and PABAK-OS coefficients were higher than the Cohen’s and Weighted Kappa coefficients, suggesting a considerable amount of prevalence or bias. For the total sample, most items were above .40, indicating acceptable agreement. The results showed higher levels of agreement for proxy scores of residents with lower levels of dementia, compared to residents with more severe dementia.

Conclusion:

The general finding of different levels of agreement between coefficients with and without correction for prevalence and bias, suggest the importance of exploring both values to enable adequate interpretation of the reliability of these items. The result of limited levels of agreement between caregivers concerning residents with more severe dementia underscores challenges for measurements in this population. We believe that practitioners and researchers should be aware of these challenges when using and interpreting scores derived from proxies. Moreover, understanding why different raters reach different conclusions regarding the same residents is important for interpreting the meaning of proxy-reported scores.