We present two practice-situated participatory investigations using networked wearable sensors to develop movement-responsive collectively playable musical instruments: a series of four collocated workshops for expert dancers and a distance learning course in which students use wearable technology to enhance embodied learning and feelings of connectedness telematically. We reflect on our exploration of techniques for structuring ensemble improvisations augmented with bespoke digital musical instruments using aggregate statistical measures, such as variance of participants’ physical orientation as an index of group intention. Participatory design exchanges top-down design methodologies with bottom-up approaches consulting actors’ interests. We follow this approach by evolving our instruments through abductive experiments and trial-and-error tinkering, without strong theories, methods, or models, using elementary signal processing techniques that are meaningfully understood and modified by participants. Our experiences suggest useful scaffolding techniques for educational transdisciplinary research-creation communities seeking to explore relational ensemble dynamics in telematic and/or physically collocated settings using accessible wearable technologies. Through creative inquiry and participation, technical objects can become bearers of sense and meaning rather than instating mystifying or alienating relations for the participants.

The performative installation DeviceD utilizes a network of systems toward facilitating interaction between dancer, digital media, and audience. Central to the work is a wearable haptic feedback system able to wirelessly deliver vibrotactile stimuli, with the latter initiated by the audience through posting on Twitter social media platform; the system in use searches for specific mentions, hashtags, and keywords, with positive results causing the system to trigger patterns of haptic biofeedback across the wearable’s four actuator motors. The system acts as the intermediator between the audience’s online actions and the dancer receiving physical stimuli; the dancer interprets these biofeedback signals according to Laban’s Effort movement qualities, with the interpretation informing different states of habitual and conscious choreographic performance. In this article, the authors reflect on their collaborative process while developing DeviceD alongside a multidisciplinary team of technologists, detailing their experience of refining the technology and methodology behind the work while presenting it in three different settings. A literature review is used to situate the work among contemporary research on interaction over internet and haptics in performance practice; haptic feedback devices have been widely used within artistic work for the past 25 years, with more recent practice and research outputs suggesting an increased interest for haptics in the field of dance research. The authors detail both technological and performative elements making up the work, and provide a transparent evaluation of the system, as means of providing a foundation for further research on wearable haptic devices.

Climbing stairs can become a daily obstacle for elderly people, and an exoskeleton can assist here. However, the exoskeletons that are designed to assist stair climbing are actuated in different ways. To find a minimal actuation configuration, we identify the assist phases by evaluating the power deficit of 11 healthy but weak elderly people (72.4 ± 2.1 years; 69–76 years; 1.67 ± 0.10 m; 74.88 ± 14.54 kg) compared to 13 younger people (24.0 ± 1.8 years; 22–28 years; 1.74 ± 0.10 m; 70.85 ± 11.91 kg) in a biomechanical study and discuss moment characteristics. Three-dimensional kinematics and ground reaction forces were collected, and kinematics, kinetics, and power characteristics of each subject for ascent and descent were calculated using inverse dynamics. Significant differences for power between both groups were assessed with statistical parametric mapping method using dynamic time warping. During ascent, the largest significant power deficit of the elderly subjects occurs in the single stance phase (SSP) during pull-up in the knee joint. During descent, significant mean power deficits of 0.2 and 0.8 W/kg for the highest deficit occur in the ankle joint in the beginning of the SSP and also in the knee joint in the same phase. Therefore, an exoskeleton should address the power deficit for knee extension (ascent: 1.0 ± 0.9 W/kg; descent: 0.3 ± 0.2 W/kg) and could assist the ankle during ascent and descent by an additional plantar flexion moment of 0.2 Nm/kg each.

Gesture-based musical performance with on-body sensing represents a particular case of wearable connection. Gloves and hand-sensing interfaces connected to real-time digital sound production and transformation processes enable empty-handed, expressive musical performance styles. In this article, the origins and developments of this practice as well as a specific use case are investigated. By taking the technical, cognitive, and cultural dimensions of this media performance as foundation, a reflection on the value, limitations, and opportunities of computational approaches to movement translation and analysis is carried out. The insights uncover how the multilayered, complex artistic situations produced by these performances are rich in intersections and represent potent amplifiers for investigating corporeal presence and affective engagement. This allows to identify problems and opportunities of existing research approaches and core issues to be solved in the domain of movement, music, interaction technology, and performance research.

This mixed-methods study investigates the use of wearable technology in embodied psychology research and explores the potential of incorporating bio-signals to focus on the bodily impact of the social experience. The study relies on scientifically established psychological methods of studying social issues, collective relationships and emotional overloads, such as sociodrama, in combination with participant observation to qualitatively detect and observe verbal and nonverbal aspects of social behavior. We evaluate the proposed method through a pilot sociodrama session and reflect on the outcomes. By utilizing an experimental setting that combines video cameras, microphones, and wearable sensors measuring physiological signals, specifically, heart rate, we explore how the synchronization and analysis of the different signals and annotations enables a mixed-method that combines qualitative and quantitative instruments in studying embodied expressiveness and social interaction.

In electronic musical instruments (EMIs), the concept of “sound sculpting” was proposed by Mulder, in which imaginary objects are manually sculpted to produce sounds, although promising has had some limitations: driven by pure intuition, only the objects’ geometrical properties were mapped to sound, while effort—which is often regarded as a key factor of expressivity in music performance—was neglected. The aim of this paper is to enhance such digital interactions by accounting for the perceptual measure of effort that is conveyed through well-established gesture-sound links in the ecologically valid conditions of non-digital music performances. Thus, it reports on the systematic exploration of effort in Dhrupad vocal improvisation, in which singers are often observed to engage with melodic ideas by manipulating intangible, imaginary objects with their hands. The focus is devising formalized descriptions to infer the amount of effort that such interactions are perceived to require and classify gestures as interactions with elastic versus rigid objects, based on original multimodal data collected in India for the specific study. Results suggest that a good part of variance for both effort levels and gesture classes can be explained through a small set of statistically significant acoustic and movement features extracted from the raw data and lead to rejecting the null hypothesis that effort is unrelated to the musical context. This may have implications on how EMIs could benefit from effort as an intermediate mapping layer and naturally opens discussions on whether physiological data may offer a more intuitive measure of effort in wearable technologies.

Inclusive musical instruments benefit from incorporating wearable interfaces into digital musical instrument design, creating opportunities for bodily felt experiences and movement-based interactions. In this article, we discuss the evolution of our inclusive design approach behind the design and performance practices of three wearable musical instruments. By focusing on the embodied, somatic, and tacit dimensions of movement-based musical interaction, we evaluate these case studies, combining the third and first-person perspectives. The design and implementation of the wearable sensing, utilizing the additive manufacturing techniques, are discussed for each instrument and its performer in specific cases of musical expression. This article further discusses how our approach integrates music performance as a crucial step into design and evaluation, utilizing these performance practices and such collaborative settings for improved diversity and inclusion. Finally, we examine how our design approach evolves from user-centered design to more participatory practices, offering people with diverse abilities a shared music performance space.