Can robots express facial emotions dominantly enough for use in dementia care?

Impaired facial emotion recognition could be a contributing factor in the social and cognitive deterioration observed in persons with dementia (PWD) influencing well-being, social competence, and increasing the stress levels of caregivers. Even in mild stages of dementia, impaired recognition of emotion is frequent if the emotion is not expressed dominantly enough (Spoletini et al., 2008). This work constitutes a step toward robot-assisted dementia care, and the underlying concept is that robots can express certain emotions quite dominantly.

Our objective is to evaluate the recognition, and denomination of the six basic emotional facial expressions as displayed by iSocioBot (Supplementary Figure, DOI: 10.5281/zenodo.3834225) to PWD, and to compare it with the results from the evaluation of static photographs of humans from the Paul Ekman database (Supplementary Method, DOI: 10.5281/zenodo.3834225) in order to investigate the differences in recognition rates among the two stimuli. iSocioBot’s facial expressions were previously validated by healthy adults (Tan et al., 2018). This study can be replicated using robots with screens, and its novelty lies in the unprecedented use of a robot for that purpose.

This is a cross-sectional study following the between-subjects approach, conducted on PWD (with a score <29 in the mini-mental state examination) at three care homes in Denmark. Participants, anonymously and voluntarily, were prompted to answer the forced-choice question “What emotion do you think the face is showing?”, by selecting from the six basic emotions in a random order. Confusion matrix and Fisher’s exact test were used in the analysis. Informed written consent was obtained, and the study was approved by the Research Ethics Committee of the North Denmark Region.

Twenty PWD of mean age 82.3 years (age range 69–92) were randomized to the photo (n:9) and the robot condition (n:11). PWD recognize dominantly the positive emotional robotic expressions of “happiness” and “surprise,” as well as the negative emotional expression of “sadness,” but “fear,” “disgust,” and “anger” are often confused (Supplementary Table, DOI: 10.5281/zenodo.3834225). Cohen’s Kappa level of agreement is considered fair (K _robot = 0.260) for the robot condition and moderate (K _human = 0.467) for the human condition. No significant difference (p = 0.461) is found in the recognition of emotions in the two different conditions.

Since this was the first study using a robot for facial expression recognition with PWD in real time, we can only discuss our findings in relation to studies using human photographs. In Henry et al. (2008), PWD had difficulty in denominating fearful, angry, and happy expressions. We confirm that the emotions of “fear” and “anger” had low recognition rates in both stimuli, but the recognition rates for the positive emotions (“happiness” and “surprise”) were high confirming Guaita et al. (2009). In Hargrave et al. (2002), PWD failed to denominate the emotions of “sadness,” “surprise,” and “disgust.” Our results confirm that “disgust” is an emotion with low recognition rates independent of stimulus, in contrast to “surprise” one of the most recognizable emotions.

Our findings suggest that robots with the capacity to convey specific dominant facial emotions may be used in dementia care complementing human presence without reducing, or replacing it.