Our emotions do not always surface into our awareness, making it difficult to manage them and communicate them to others. Even when emotions do not reach our awareness, they still express themselves as physiological changes, often unperceived by ourselves and others. To aid in emotion self-regulation and increase the bandwidth of emotion communication, I designed a programmable affective sleeve that translates physiological aspects of emotions into material haptic action. The affective sleeve has been developed as a case study for Affective Matter. Affective Matter suggests a method for human-material interaction that enhances health and wellbeing.

I first discuss the three foundations of Affective Matter underlying the design of the affective sleeve: Embodiment, Entrainment, and Material Intelligence. I then proceed to the methods and results of an exploratory study I developed and conducted that tests the psychophysiological impact of the sleeve on 36 participants. The study results suggest that the pace of the affective sleeve’s haptic action can be programmed to regulate the wearer’s breathing pace to either have a calming or a stimulating impact on the wearer. The results also show varied affective responses to distinct haptic stimuli. Discussion of the results suggests future research directions and therapeutic applications for the benefit of individuals with mental health and neurodevelopmental disorders.

The majority of research in speech emotion recognition (SER) is conducted to recognize emotion categories. Recognizing dimensional emotion attributes is also important, however, and it has several advantages over categorical emotion. For this research, we investigate dimensional SER using both speech features and word embeddings. The concatenation network joins acoustic networks and text networks from bimodal features. We demonstrate that those bimodal features, both are extracted from speech, improve the performance of dimensional SER over unimodal SER either using acoustic features or word embeddings. A significant improvement on the valence dimension is contributed by the addition of word embeddings to SER system, while arousal and dominance dimensions are also improved. We proposed a multitask learning (MTL) approach for the prediction of all emotional attributes. This MTL maximizes the concordance correlation between predicted emotion degrees and true emotion labels simultaneously. The findings suggest that the use of MTL with two parameters is better than other evaluated methods in representing the interrelation of emotional attributes. In unimodal results, speech features attain higher performance on arousal and dominance, while word embeddings are better for predicting valence. The overall evaluation uses the concordance correlation coefficient score of the three emotional attributes. We also discuss some differences between categorical and dimensional emotion results from psychological and engineering perspectives.