Hostname: page-component-cd9895bd7-dk4vv Total loading time: 0 Render date: 2024-12-27T14:08:54.385Z Has data issue: false hasContentIssue false

Ecomprovisation: Project Markarian 335

Published online by Cambridge University Press:  30 January 2024

Luzilei Aliel
Affiliation:
University of São Paulo. Email: luzaliel@gmail.com
Ivan Simurra
Affiliation:
Federal University of Acre. Email: ivan.simurra@ufac.br
Marcello Messina
Affiliation:
Southern Federal University. Email: messina@sfedu.ru
Damián Keller
Affiliation:
Federal University of Acre, Federal University of Paraíba. Email: dkeller@ccrma.stanford.edu
Rights & Permissions [Opens in a new window]

Abstract

Through an ecological approach to creative practice (henceforth ecomprovisation), this project deals with the expansion of creative strategies applicable to everyday contexts. Within ubiquitous music (ubimus), we target the convergence of sonification methods with the application of ecological models within the context of comprovisation. These conceptual frameworks inform the technological and aesthetic approaches applied in the making of Markarian 335. We describe the creative procedures and the implications of the design choices involved in this artwork. The contributions and shortcomings of our ecomprovisational approach are situated within the context of the current efforts to foster expanded creative possibilities in ubimus endeavours.

Type
Article
Copyright
© The Author(s), 2024. Published by Cambridge University Press

1. INTRODUCTION

Since the publication of the first volume on ubiquitous music research (ubimus) (Keller, Lazzarini and Pimenta Reference Keller, Lazzarini and Pimenta2014), several new perspectives within this expanding field have widened the palette of possibilities for artistic endeavours. One promising trend involves the incorporation of improvisatory strategies to foster the exploration of sonic resources. This practice-led approach to music research was already present in various ubimus projects, gaining a specific conceptual and methodological foothold through the incorporation of ecological frameworks. How does ecological thinking impact the creative practices that target ubiquitous contexts? We tackle this question through a discussion of proposals within ecologically grounded creative practice, providing hands-on examples of the design and deployment of an artistic project using acoustic and computationally based sonic resources.

An aspect of ubimus research that received increased attention during the last few years involves the implementation of support for creative engagement by both musicians and participants who have no technical training. This is a particularly problematic issue because the majority of the extant research on musical interaction is limited to professional musicians and in some cases it centres on the virtuosic performer as the ideal model for interaction design (Wessel and Wright Reference Wessel and Wright2002). The undue stress on a genre-based understanding of virtuosism and on a culturally biased interpretation of expression Footnote 1 has erected serious barriers to the inclusion of untrained participants in music-making. Rather than adopting preconceived genres, while tailoring the design for the needs of a particular niche of musicians, ubimus approaches have strived to avoid the narrow focus of previous musical-interaction proposals while highlighting the need to let the participants determine when and how to deploy resources to attain their creative goals. Furthermore, the concept of creative goal has been put into question. When applied to artistic and educational activities, the utilitarian view of interaction as a problem-solving activity has shown serious limitations (Lima, Keller, Pimenta, Lazzarini and Miletto Reference Lima, Keller, Pimenta, Lazzarini and Miletto2012). This is why ubimus methods tackle both the exploratory aspects of creative music-making and the demands of knowledge-sharing mechanisms in activities involving both trained and untrained participants.

Ecological thinking Footnote 2 has gained strength through ubimus deployments. Since the pioneering proposals dating from the late 1990s (Keller Reference Keller1999, Reference Keller2000), a healthy diversity of ecologically grounded frameworks has been applied to a wide range of artistic endeavours. Nance (Reference Nance2007) introduced eco-based strategies to the realm of acoustic-instrumental writing, hence expanding the palette of sonic resources through the inclusion of recorded instrumental sources. Connors (Reference Connors2015) proposed the concept of ecological performativity, adding another layer of techniques to eco-grounded installation art. Through a series of performance-oriented artworks, Aliel, Keller and Costa (Reference Aliel, Keller and Costa2015) furnish the basis for a fusion between ecologically oriented improvisation and compositional approaches. Labelled ecomprovisation, Footnote 3 this practice-based thread applies ecological thinking to shape group behaviours targeting both synchronous- and asynchronous-distributed musical activities.

Another thread of developments has emerged from the application of ecological models to sound-making. According to Braun (Reference Braun2017: 89), sound-making should be studied in the context of and in relation to the other senses and modalities. He characterises sound studies as a field within the cultural and social sciences, highlighting its historical aspects. This proposal involves an interdisciplinary research agenda that emphasises cross-cultural components. From this perspective, Braun discusses the design and compositional projects of the late 1990s that have pushed the field towards multimodality. Keller and Truax (Reference Keller and Truax1998) implemented a set of techniques employing databases of granular sounds extracted from environmental sources and employed control functions modelled after the behaviours of real-world events. Artistic deployments of this approach can be heard in the eight-channel pieces …soretes de punta (Reference Keller and Truax1998) and touch’n’go (Reference Keller1999). As an example of a multimodal perspective on sound-making, Braun discusses Tyler Kinnear’s analysis of …soretes de punta. He states that the compositional usage of synthesised water sounds highlights the variety of manipulations and interactions applicable to environmental sources in creative endeavours. According to Kinnear’s analysis, the use of ecologically based granular synthesis lets the composer handle the identity of the sonic resources through operations on the acoustical parameters of water drops while targeting their interactions with various surfaces. Sounds derived from this referential identity serve as a creative palette for …soretes de punta. Despite the restricted materials, the manipulation of phase-synchronicities and decorrelations among streams expands the timbral profile of the sonic events and textures, thus enriching the sonic results (Rolfe and Keller Reference Rolfe and Keller2000). Braun (Reference Braun2017: 85) concludes that as a transformative agent, the water stream functions as a thread between a baseline timbral identity – established through the spectral characteristic of the drop collisions – and the environmental sounds utilised in the piece. Footnote 4

Braun’s argument adds weight to the ecological methods of Opie and Brown (Reference Opie and Brown2006) and Gomes et al. (Reference Gomes, Pinho, Lopez, Costa, Dias, Tudela and Barbosa2014) that deal with the integration of perceptual modalities. Other examples of this approach are featured in Connors’s two audiovisual installations, From the Edge and Currents, based on sounds collected through field recordings made at Newfoundland’s East Coast in Canada. Connors uses Barad and Morton’s ideas to articulate a compositional perspective that questions representationalism, adopting a midway position on how to deal with agency within a nature–culture continuum. These installations deploy live-streamed environmental datasets to re-enact various forms of agency. The datasets function as co-creative devices to explore the ecologically grounded performative dimensions of the artworks. The methods used by Opie, Gomes and Connors rely on the extraction of data from environmental phenomena. Rather than treating the usage of data as culturally neutral, they reinterpret the sources through situated sonification strategies. Consequently, these proposals differ from the standard usage of sonification as found in historical computer-music compositions such as Dodge’s Earth’s Magnetic Field (Dodge Reference Dodge1970). Footnote 5

In line with the preceding discussion of sonic-oriented ecological perspectives, model-based sonification involves the creation of acoustic models that generate responses based on inputs from a user or from multiple users. This technique requires a computational model and complementary instructions to guide behaviours towards a sonic outcome. In contrast with the historical sonification approaches previously cited, ecological models involve an active engagement from the stakeholders. Local interactions with the participants become a crucial aspect of this artistic approach. The dynamics of the models are not fixed or strictly dependent on the initial conditions. They rely strongly on environmental factors which are prone to change during deployments. From an acoustic-instrumental perspective, this approach could be interpreted as a virtual musical instrument that can be played by the user to generate the respective sound (Hunt and Hermann Reference Hunt and Hermann2004). Nevertheless, as entities dependent on mutual adaptations (which are very different from passive acoustic instruments), ecological models tend to be shaped by sonic ecologies (Keller Reference Keller and Brown2012; O’Callaghan Reference O’Callaghan2013). Given this functional context, acoustic-instrumental interaction becomes irrelevant. Footnote 6 To emulate real-world agent-object interactions that entail meaningful responses to actuation (usually yielding sounds and involving several temporal layers of events and featuring a range of variations dependent on the chosen modality), the data and the state of the ecosystem need to remain open to emergent processes.

Furthermore, the application of ecological thinking to the realm of sonification may extend the usage of local resources in creative practice. The initial proposals in this field during the late 1990s involved the implementation of ecological models, the emulation of everyday events and targeted artistic experiences based on multimodality. More recently, Opie and Brown (Reference Opie and Brown2006), Hermann (Reference Hermann2011) and Connors (Reference Connors2015) have independently developed techniques that extract information from local events fostering the explorations of sonic models based on human decision-making. How to deploy these resources in improvisation is an open issue partially addressed by recent artistic initiatives.

In this article, we document the artistic project Markarian 335 (henceforth M335). The article targets interaction design and involves the implementation of computational tools, highlighting the application of sonification techniques aimed at improvisation. Based on an ecological approach to comprovisation, the M335 project explores alternatives to expand the creative strategies applicable to everyday contexts. The section 2 of the article engages with the deployment of ecological frameworks within ubimus. We highlight the use of ecologically grounded audio-processing models and point to the potential impact of this perspective on sonification. Then we address the usage of auditory-display techniques as creative strategies for ubimus, indicating crossovers with the frameworks that target multimodal and behavioural ecologies (Keller and Lazzarini Reference Keller and Lazzarini2017). The section 3 is dedicated to the procedures employed in M335, highlighting the design of creativity-support strategies tailored for comprovisational practice. Finally, we deal with the contributions and shortcomings of each strategy to creative endeavours, situating the results within the current ubimus expanded field for novices and experts.

2. RELATED WORK: COMPROVISATIONAL APPROACHES

Comprovisation is an emerging artistic strategy that encompasses composition and musical improvisation. The term comprovisation does not have a consensual definition yet. Its origins go back to the improvisational proposals of the 1960s – including the output of artistic groups such as the Scratch Orchestra and Musica Elettronica Viva (Cardew Reference Cardew1969; Curran and Teitelbaum Reference Curran and Teitelbaum1989). Current comprovisational approaches have also addressed aspects of interactive musical representations (Hannan Reference Hannan2006; Dudas Reference Dudas2010; Fujak Reference Fujak2011). Fujak (Reference Fujak2011) argues that in the activities done by living beings, some scenarios can only be partially planned. A variety of uncontrolled factors can impact the environmental conditions, shaping the behaviours of the stakeholders. Thus, he proposes comprovisation as a metaphor for ways of living. In ecological parlance, interaction units are defined as events which occur in the context of habitats and ecologies (Keller Reference Keller2000; Keller and Capasso Reference Keller and Capasso2006). When resources are deployed, the stakeholders engage in both planned actions and unpredictable behaviours. Plans are ways of projecting relationships between the extant resources, encompassing both material and cognitive activities, which may entail either teleological or exploratory strategies. In this article, we aim to construct a conceptual framework bridging comprovisation and ecologically grounded creative practicality within the ubimus context. To achieve this, we draw parallels with a conceptual tool introduced by Aliel, Keller and Ferraz (Reference Aliel, Keller and Ferraz2018b) and Messina and Aliel (Reference Messina and Aliel2019, Reference Messina and Aliel2023), the Gelassenheit.

Gelassenheit is a term coined by Heidegger (Reference Heidegger1966). Its literal translation is ‘serenity’ although Heidegger’s formulation transcends the literal meaning of this word in English. He proposes two forms of thought: 1) calculative thinking, understood as a science-oriented artistic method involving measuring, collecting data and replicating results; and 2) its counterpart, meditative thinking, entailing an attitude open to unpredictable actions and unexpected outcomes, that is, ‘an openness to mystery’ (Heidegger Reference Heidegger1966: 55). Heidegger proposes Gelassenheit as a stage to be achieved through innovative ways of thinking. From an ecologically oriented perspective and grounded on arguments by Donald (Reference Donald and Turner2006), Aliel, Keller and Costa (Reference Aliel, Keller and Costa2018a) reframe this proposal as a process of adaptation and modification of self-reflective strategies in artistic practice. The absence of control encouraged by Gelassenheit factors tends to trigger unexpected results, setting an atypical frame for potential actions and forcing the stakeholders to adapt their behaviours to unfamiliar conditions. Thus, Gelassenheit may be viewed as a factor fostering evolutionary processes within highly compressed time frames.

Within the concept of Gelassenheit defined by Heidegger, encompassing calculative thinking and meditative thinking, we can hypothesise a method for comprovisation that engages in a dialogue with the ecological perspective through action plans (guideline plan and contingency plan) (Aliel Reference Aliel2022). From an ecological perspective, therefore, comprovisation may be defined as a way to handle creative actions that feature guideline plans (that constitute the core of compositional activities – calculative thinking) and contingency plans (involving elements of improvisation – meditative thinking), having a direct relation to the local environment as source material and involving experimentation within its sphere of practice. A guideline plan establishes rules of interaction that are persistent and depend on material factors such as mass, form and the dynamic properties of materials, acting upon specific structures and resources (such as place and stakeholders). A contingency plan may deal with the same targets as the guideline plans, though it also encompasses volatile resources and processes that lack predictability. A guideline plan includes rules, planned actions, algorithms and resources often obtained through asynchronous methods (and usually linked to pre-compositional activities). A contingency plan targets the unpredictable, the eventualities, the chaotic (involving deterministic but highly complex outcomes) and random events (encompassing either environmental factors or computational strategies), thus highlighting the creative role of both human and non-human errors.

Depending on emerging musical and social needs, comprovisational strategies can adapt. Given specific conditions, guideline and contingency plans may offer opportunities to level the collective access to the creative processes, avoiding preconceived notions of priority of one strategy over another, as suggested by the critical discussions on the application of hierarchical and centralised compositional methods by Bhagwati (Reference Bhagwati, Lasker, Luz and Dack2008), Keller (Reference Keller2000) and Lewis (Reference Lewis2000). This flexibility tends to add weight to contingencies. Consequently, one of the challenges of ubimus design is to provide comprovisational guidelines that are effective both for persistent and volatile resources (Keller Reference Keller2014), while encouraging the exploration of the creative potential of contingencies.

Some comprovisational plans may involve elements of intentionality of the stakeholders. A guideline plan may feature an explicit representation of expected outcomes. Contingency plans usually entail a lack of completeness and rest on implicit knowledge shared through cultural traces (non-idiomatic improvisation is a case in point). Sometimes, both plans may converge. For instance, the absence of rules implies a new rule that limits sharing explicit knowledge. This suggests the usage of tacit or informal knowledge as a way to establish criteria and order to the actions of the agents. At the other extreme, disruptive tendencies in comprovisation may involve an entity imposing a contingent plan opposed to the shared concepts obtained through previous interactions.

For the purpose of creating a new specific neologism related to comprovisation work aligned with ecological practices (guideline plans and contingency plans), we propose the inclusion of the letter ‘e’ into the context of comprovisation, resulting in ‘ecomprovisation’. In this context, the ‘e’ combined with ‘co’ (eco) associates with ‘comp’ (composition) and ‘provisation’ (improvisation), thereby amalgamating three words into a fusion that aims to encompass the concept.

3. M335 PROJECT

In this section, we tackle the analysis of an artistic work, M335. Our focus is on the possibilities of using action plans (guidelines and contingencies) for the creation of an ecomprovisation. We approach the discussion of M335 from the perspective of a professional double bass player. Our analysis highlights technological advancements – involving the Handy S prototype – and approaches centred on musical aspects, such as the utilisation of extended techniques and graphical notations.

M335 is a piece for acoustic or electric bass, inspired by NASA’s discovery of an unidentified object. The object was photographed while distancing from the black hole M335. It has not been catalogued and studies have not yet reached consensus on its composition or nature. Footnote 7 Einstein’s general theory of relativity implies that nothing, not even light, can escape a black hole. The event that took place in M335 pointed at a new phenomenon that remains unexplained. We employ this premise as an artistic inspiration. Metaphorically, the double bass tries to ‘escape’ the sound mass generated by the machine. However, the sonic materials are mostly controlled by the actions exerted on the instrument. This struggle between acoustic and electronic sources fosters a process of sonic immersion involving cycles of feedback between the gestures and the sounds.

3.1. Tools and materials

The computational prototype features two objects from the Pure Data GEM Footnote 8 library, namely [pix_video] and [pix_data]. The [pix_video] object enables the use of a webcam connected to a computer. The [pix_data] object acquires the colour of a specified pixel in an image. This capture is defined by the RGB standard (red, green and blue). During the initial tests of a webcam attached to the double bass, we observed that colour-based capture does not necessarily yield a result directly connected to the musician’s gestures. The capture process and subsequently the sound production were imperfect, even in single-colour conditions. In other words, it was not possible to repeat or refine the capture. After several tests, we concluded that colour-based capture did not contribute to the generation of musically replicable material.

However, during the testing process we also noted that the musician’s movements of the double bass produced sounds he considered more ‘musical’ despite their random qualities. Based on this premise, we proposed an indirect-capture process, allowing more freedom to move along x, y and z axes. Rather than emphasising mappings for each colour; the new approach focused on the relationship between light and darkness enabled by the movements. The musician moved the double bass, the webcam captured the light sources and sounds were produced. Darkness or low illumination yielded no capture and no sound. This approach provided a baseline control of sound ‘on and off’.

Although we had a certain level of control, we considered incorporating randomness under unexpected conditions. In this context, the musician interacts with a sound (on or off) but has no control over other sonic characteristics, forcing them to adapt to the local conditions. Considering the object’s ability to capture RGB, we altered the equation of the three colours (multiplications in some objects) to generate variations in duration, velocity and pitch. Footnote 9

3.1.1. Graphics: target clear and accessible indications for interpretations and actions by the musicians/performers

We use spectrographic scores Footnote 10 to support the process of aesthetic decision-making (Figure 1). Our knowledge-sharing strategy features conceptual hints rather than rigid procedures. We use two graphs to trigger alternative sonic interpretations of data captured by the satellites LIGO Livingston and LIGO Hanford. Footnote 11 These sonic sources are colloquially termed ‘black-hole sounds’.

Figure 1. Spectrographic scores used in M335.

To establish a connection between the spectrogram and instrumental actions, we subdivide each spectrum in three regions. These subdivisions (indicated by green, red and blue lines) are associated with the regions of the double bass’s body and indicate where the left hand should be placed to handle pitch-oriented actions. To facilitate the comprehension by the performer, we have related the increase in intensity (displayed in the spectra) to dynamic references applicable to each segment. The double bassist can choose to play either the left side (LIGO Livingston) or the right side (LIGO Hanford). She can also switch sides at any time between LIGO Livingston or LIGO Hanford, or she may repeat the material indefinitely.

The spectrographic scores are read from top to bottom. Timbral variations are directly associated with bow movements. Thus, if a parameter value is large, the bow gesture will also be large. The wavy notation represents the intensity of the right hand (applied while bowing, pizzicato and so on). This parameter can be read from top to bottom or from bottom to top. Changes in the order during the performance are left to the discretion of the instrumentalist. There is no time restriction for repetitions, but there are restrictions on the dynamics and the strings choices. For instance, on the left side (LIGO Livingston), the strings E and A are indicated, while on the right side (LIGO Hanford), the strings D and G are indicated. As already mentioned, the choice of sides is free.

From a total of eight, four pages are used for each performance. Some pages permit a free choice of pitches, while others have guidelines indicating the pitches to be played or excluded. There are no time indications for page turns. Hence, the instrumentalist may choose to explore one or more pages before handling the complete set.

3.1.2. Extended techniques: furnish accessible and clear indications for interpretations and actions by the performers

Extended techniques based on explicit instructions are featured as part of the guidelines plan. Despite adopting a limited set of instrumental techniques, we tailored the design for: 1) adaptation of sonic processing based on bow movements; and 2) adaptations tailored for the sonification model (more on this later). The variability is governed by rules of action and reaction that depend on the local context. Consequently, small-scale decisions play a decisive role. The implication is that a creative agent cannot expect to replicate previous results because there is no univocal relationship between the audio-processing techniques and the parameters driven by this form of performance. Footnote 12

In the performance, we can observe the musician utilizing some of the proposed features, placing emphasis on the body movement gesture of the double bass to activate the webcam feature, employing extended techniques, and at times referring to the score to guide certain actions (Figure 2; Video Example 1).

Figure 2. M335’s performance.

3.1.3. Usage of the Handy S prototype in M335

The prototype Handy S was used to explore adaptive strategies (Keller, Aliel and Silva Reference Keller, Aliel and Silva2018): 1) parametrisation based on the capture of acoustic-instrumental gestures – a webcam is coupled to the instrument providing a gestural sonic interface that bypasses the typical process of acoustic-instrumental transduction; and 2) the generative sonic results are recycled as material for improvisation.

We conducted preliminary studies that led to the deployed version of M335, involving performers accustomed to improvisation practices who were familiar with contemporary musical repertoire. These studies pointed to strategies to attain greater control of the sonic outcomes: it was observed that some performers began to track the timing of the webcam ‘errors’. Thus, they learned to anticipate when the system generates an update and employed these opportunities to try new behaviours leading to new sounds.

Another strategy involved learning that small changes of position in the x, y and z axes result in greater control of the outcomes. Instead of using manual gestures to trigger modifications (following their standard instrumental training), the performers tried moving the body of the instrument. Since the webcam is attached to the instrument, the movements of the double bass generated more precise sonic outcomes. We can interpret these strategies as human behaviours adapted to specific demands of the ubimus ecosystem. We believe these features were discovered because the musicians searched for apparent flaws in the system that allowed certain materials to be ‘tamed’, becoming more predictable and controllable.

4. M335: SUMMARY OF FINDINGS

M335 features two forms of auditory-display: action and feedback. The interaction model involves reading and interpreting a score and capturing visual data to transition from no sound to sound. The complete cycle involves events initiated by the instrumentalist, visual data captured by the webcam, and sonic outcomes that are not fully determined by the state of the system. While executing the score, the performer triggers multimodal events (Figure 3). Footnote 13 The spectrographic scores are used as a guide for the improvisational processes (see Opie and Brown Reference Opie and Brown2006; Connors Reference Connors2015 for related techniques). These processes do not imply a direct mapping between data and sonic outcome: they yield a new sonic layer that diverges from the acoustic-instrumental material. Furthermore, the player cannot intentionally replicate the processed material because the behaviours are shaped though sonic feedback. Instead of using only the sound of the instrument or applying extended techniques, our ecomprovisational approach promotes the empirical exploration of electroacoustic and acoustic sonic resources.

Figure 3. Diagram of the M335 interaction model.

A second strategy aims to increase the creative relevance of the sonic sources. A fairly entropic level is fostered through a ‘reversal’ of roles. The musician (although still triggering audio processes) relies on events produced by the interactions with the Handy S prototype that may not necessarily involve instrumental sound. The instrumental gestures are captured by the webcam (Figure 4), Footnote 14 triggering processes that may diverge from what is expected by the instrumentalist. Consequently, electronic sounds become central. The outcomes are linked to the interactions between the instrumental sound and the movement-tracking system. Through this dual and synchronous interaction, the audioprocessing parameters introduce new and divergent material based on the double bass sounds.

Figure 4. A consumer-level camera was attached to the double bass to enable data capture and parametric control in M335.

As previously explained, the current prototype retains a certain level of randomness due to the low definition of the visual data and due to the automated ‘choices’ of data capture. These factors, rather than being treated as limitations, constitute a target of design for the M335 ecomprovisational project. Although not planned as a design feature, during the sessions the musician pointed out that the projection of the hands of the performer influenced the way they were playing. At a later stage, this third strategy was integrated into the design (see Aliel Reference Aliel2017). Through the projection of the movements of the hands, the unintentional material is reframed, grounding the assessments of the contingencies on multimodal information and enhancing the immersion in the ecomprovisational experience.

From this perspective, we consider that the M335 prototype follows the proposal by Boden and Edmonds (Reference Boden and Edmonds2009) to apply constraints to promote complex and creative results. These constraints serve to frame the performance leaving room for ‘computational choices.’ Footnote 15 Furthermore, a connection arises between Heidegger’s calculative and meditative thinking, as presented earlier. While calculative thinking permeates the aesthetics and practice based on the acoustic instrument, expanded through the option to turn the electronic sounds on or off, meditative thinking is encouraged by fostering openness to the unexpected and adapting to the local conditions.

If we consider the application of Gelassenheit, we can suggest that the occurrence of unexpected conditions (almost with random characteristics) is not actually a problem. In general, this kind of event is common in ecological conditions (natural events that cannot be controlled by humans or computers). Thus, from this perspective, unexpected elements can bring unique benefits to the process and can be understood as events within the contingency plan. In other words, even a technological parametrisation error will be interpreted as a form of ‘improvisation’. We term these conditions Gelassenheit events (Aliel Reference Aliel2022).

5. FINAL REMARKS AND FUTURE DEVELOPMENTS IN ECOMPROVISATION

Does ecomprovisation provide a path to deal with the demands of future ubimus initiatives? Let us take a moment to reflect on the proposals laid out in Keller, Messina and Oliveira, Reference Keller, Messina and Oliveira2020a and Keller, Simurra and Messina, Reference Keller, Simurra and Messina2020b targeting second-wave ubimus endeavours. Second-wave ubimus may encompass everyday creative musical activities and collective asynchronous musical interaction. The constraints of casual interaction encouraged by ubimus initiatives include: 1) very limited time for preparations; 2) short-term engagements; 3) a wide diversity of stakeholder profiles; 4) unreliable network connectivity; and 5) unrestricted public exposure. While some of these caveats are addressed by the techniques discussed in this article, others constitute targets of future research. Despite envisaging professional-creativity support, some of the strategies implemented in M335 could be adapted to address the needs of casual participation (Keller et al. Reference Keller, Aliel and Silva2018). In particular, the techniques developed for the creation of the artwork M335 provide ways to extract knowledge from local resources that can enhance improvisational approaches. These approaches may potentially include strategies for everyday musical creativity for both trained and untrained participants in the future.

Potential future approaches include:

  1. 1. By using open graphic scores that present basic instructions for actions, the gap between learning and performing is reduced, allowing both groups to employ similar resources (see Aliel Reference Aliel2022).

  2. 2. Since the support involves coupling a technological object to an acoustic instrument, the process of gestural and sonic experimentation could eventually unfold similarly for both novices and more experienced participants, fostering knowledge exchanges through mimicking or imitation.

  3. 3. As unpredictable factors are an important feature of ecomprovisation, both groups need to deal with emergent properties by finding solutions or incorporating new actions into their palette of creative tools. This implies a pressure on the boundaries between everyday musical creativity and professionally oriented approaches. Ecomprovisation may furnish a bridge to explore both perspectives.

  4. 4. In line with the maker-oriented DIY ubimus proposals (Lazzarini, Keller, Otero and Turchet Reference Lazzarini, Keller, Otero and Turchet2020), everyday objects could be repurposed through ecomprovisational strategies. Footnote 16 These techniques could bypass several of the financial barriers faced by potential participants in low-income regions. Owing to the scarcity of professional audiovisual equipment in peripheral countries, gear such as high-resolution video-cameras may be replaced by recycled consumer-level mobile telephones. Android systems, for example, offer a wide range of free or open-source software Footnote 17 that support synchronous streaming of audiovisual data to personal computers.

  5. 5. The incorporation of more refined movement-tracking algorithms (as exemplified by Chakraborty, Yaseen, Timoney, Lazzarini and Keller Reference Chakraborty, Yaseen, Timoney, Lazzarini and Keller2022) also opens interesting possibilities to release the interaction from the bounds of the acoustic instruments. Mid-air interaction techniques involve triggering a multiplicity of sources from a visual-tracking algorithm that supports adaptations to diverse body shapes and positions. No special-purpose equipment is necessary.

Finally, a note of caution is necessary regarding the support for distributed activities. Collective asynchronous musical activities bypass co-located face-to-face interactions. Consequently, they may reduce the social-bonding aspects fostered by co-located music-making. While several comprovisational proposals have targeted synchronous activities, open temporalities tend to be linked to non-hierarchical and asynchronous techniques. Ubimus ecomprovisational techniques could provide a way to deal with asynchronicity and quasi-synchronicity in comprovisation. On the one hand, group music-making lets the stakeholders build a shared knowledge pool that grounds consensual decisions. Alternatively, it fosters the coexistence of divergent worldviews. On the other hand, the participants’ usage of the local resources encourages aesthetic diversity. Diversity within consensus and coexistence of opposed views seem to be attitudes encouraged by ubimus endeavours. Ecomprovisational strategies – as exemplified in M335 – feature forms of interaction aligned with these goals.

SUPPLEMENTARY MATERIAL

To view supplementary material for this article, please visit https://doi.org/10.1017/S1355771823000651

Footnotes

1 See Gurevich and Treviño (Reference Gurevich and Treviño2007) for a critical discussion of the use of the label expression within the context of the conferences on New Instruments for Musical Expression.

2 We use the terms ecological, eco, ecologically grounded, eco-based and derivatives to refer to the creative practices that are inspired by and engage with what has been termed the E4 perspective in cognition (ecological, embodied, embedded and enactive).

3 Ecomprovisation features a fusion of comprovisational methods within the context of ecologically grounded creative practices. We acknowledge the possibility of exploring other forms of fusion of ecologically inspired improvisatory approaches within ubimus, such as exemplified by Connors (Reference Connors2015) and Stolfi, Milo and Barthet (Reference Stolfi, Milo and Barthet2019).

4 See also Basanta’s (Reference Basanta2010) analysis of this work. An informal discussion of the context of this piece can be found in Carson (Reference Carson2020). More background can be found in (Keller Reference Keller1999, Reference Keller2000; Keller and Berger Reference Keller and Berger2001).

5 See an in-depth discussion of the constraints and potentials of applying auditory-display methods in ubimus design in Lazzarini and Keller (Reference Lazzarini and Keller2021).

6 Examples of the acoustic-instrumental paradigm in musicology and in the proposals for ‘new instruments for musical expression’ are: the separation between the public, the composer and the performer; mastery of specific knowledge as a requirement for making music; the myth of the genius as a source of creativity incarnated by the virtuoso, conductor or composer (usually male, white and from a central country). See discussions in Bown, Eldridge and McCormack (Reference Bown, Eldridge and McCormack2009), Keller (Reference Keller2000) and Simurra, Messina, Aliel and Keller (Reference Simurra, Messina, Aliel and Keller2023).

7 The news report can be seen at https://astronomynow.com/2015/10/28/black-hole-markarian335-has-major-flare/ (accessed 12 January 2024).

8 GEM (Graphics Environment for Multimedia) is a set of externals developed for multidimensional matrix processing on Linux, Macintosh or Windows to create and manipulate video and graphics in OpenGL.

9 The Pure Data patch can be viewed at www.journals.cambridge.org/OSO.

10 A spectrographic score is defined as a visual resource that features spectral information to convey musical knowledge. This knowledge-transfer strategy has also been adopted by other ubimus projects, see for instance Stolfi et al. (Reference Stolfi, Milo and Barthet2019).

11 The original spectrum can be seen at www.youtube.com/watch?v=dP6ZWew83_Q.

12 This touches upon issues of replicability in creative practice that have been a focus area of research in ubimus.

13 In Figure 3, the items [pix_videos] and [pix_videos] refer to Pure Data objects.

14 We employed a Logitech webcam, Model C920, in our implementation. Its resolution capability extends up to 1920 ×1080 pixels with H.264 video compression.

15 For example, given a set of numbers from 1 to 10 and (A > B), considering (B = 4) we have (A → 5, 6, 7, 8, 9, 10). The specific choice is left to the machine.

16 In some regions, even standard devices that are usually cheap and accessible – such as high-resolution webcams or MIDI-enabled gadgets – are prohibitively overpriced. For these contexts, DIY replacements may furnish affordable solutions.

17 For instance, DroidCam.

References

REFERENCES

Aliel, L. 2017. Essays on Comprovisations in Sound Ecology: Practical and Theoretical Perspectives. MA dissertation, University of São Paulo.Google Scholar
Aliel, L. 2022. Model-Based Comprovisation: Modeling in Musical Composition and Improvisation Based on Guideline and Contingency Plans. Doctorate thesis, University of São Paulo. https://doi.org/10.11606/T.27.2022.tde-12072022–111800.CrossRefGoogle Scholar
Aliel, L., Keller, D. and Costa, R. 2015. Comprovisation Approaches from Aesthetic Heuristics to Composition. Proceedings of the XV Brazilian Symposium on Computer Music. Campinas, SP. SBCM.Google Scholar
Aliel, L., Keller, D. and Costa, R. 2018a. The Maxwell Demon: A Proposal for Modeling in Ecological Synthesis in Art Practices. Música Hodie 18(1): 103–16. https://doi.org/10.5216/mh.v18i1.53575.CrossRefGoogle Scholar
Aliel, L., Keller, D. and Ferraz, S. 2018b. Perspectives of the Gelassenheit Thought-Form: Applications and Experimentation in Cognitive-Ecological Creative Activities. Revista Vórtex 6 (2): 127 CrossRefGoogle Scholar
Basanta, A. 2010. Syntax as sign: The Use of Ecological Models within a Semiotic Approach to Electroacoustic Composition. Organised Sound 15(2): 125–32. https://doi.org/10.1017/S1355771810000117.CrossRefGoogle Scholar
Bhagwati, S. 2008. Towards Interactive Onscreen Notations for Comprovisation in Large Ensembles. In Lasker, G. E., Luz, M. A., and Dack, J. (eds.) Systems Research in the Arts and Humanities. Windsor: International Institute for Advanced Studies in Systems Research and Cybernetics, 43–9.Google Scholar
Boden, M. A. and Edmonds, E. A. 2009. What is Generative Art? Digital Creativity 20(1–2): 2146.CrossRefGoogle Scholar
Bown, O., Eldridge, A. and McCormack, J. 2009. Understanding Interaction in Contemporary Digital Music: From Instruments to Behavioural Objects. Organised Sound 14(2): 188–96. https://doi.org/10.1017/S1355771809000296.CrossRefGoogle Scholar
Braun, H. J. 2017. An Acoustic Turn? Recent Developments and Future Perspectives of Sound Studies. Avant: Trends in Interdisciplinary Studies 8. https://doi.org/10.26913/80102017.0101.0005.CrossRefGoogle Scholar
Cardew, C. A. 1969. Scratch Orchestra: Draft Constitution. The Musical Times 110 (1516): 617–19.CrossRefGoogle Scholar
Carson, T. 2020. On Ecocomposition: An Interview with Damián Keller. Journal of Digital Media & Interaction 3(5): 112–13.Google Scholar
Chakraborty, S., Yaseen, A., Timoney, J., Lazzarini, V. and Keller, D. 2022. Adaptive Touchless Whole-Body Interaction for Casual Ubiquitous Musical Activities. Proceedings of the International Computer Music Conference (ICMC2022), Limerick, Ireland, 132–8.Google Scholar
Connors, T. M. 2015. Audiovisual Installation as Ecological Performativity. Proceedings of the 21st International Symposium on Electronic Art (ISEA 2015). Vancouver, Canada: ISEA.Google Scholar
Curran, A. and Teitelbaum, R. 1989. Musica Elettronica Viva. Program notes for MEV Festival, the Knitting Factory, New York. www.alvincurran.com/writings/mev.html (accessed 20 December 2023).Google Scholar
Dodge, C. 1970. Program notes to Earth’s Magnetic Field. Nonesuch LP H 71250.Google Scholar
Donald, M. 2006. Art and Cognitive Evolution. In Turner, M. (ed.) The Artful Mind. Oxford: Oxford University Press, 320.CrossRefGoogle Scholar
Dudas, R. 2010. Comprovisation: The Various Facets of Composed Improvisation within Interactive Performance Systems. Leonardo Music Journal 20: 2931.CrossRefGoogle Scholar
Fujak, J. 2011. Comprovisación: Notas para la discusión sobre la validez del concepto. Oro Molido 33: 2430.Google Scholar
Gomes, J., Pinho, N., Lopez, F., Costa, G., Dias, R., Tudela, D. and Barbosa, Á. 2014. Capture and Transformation of Urban Soundscape Data for Artistic Creation. Journal of Science and Technology of the Arts 6(1): 97109.CrossRefGoogle Scholar
Gurevich, M. and Treviño, J. 2007. Expression and Its Discontents: Toward an Ecology of Musical Creation. Proceedings of the 7th International Conference on New Interfaces for Musical Expression, 106–111.Google Scholar
Hannan, M. 2006. Interrogating Comprovisation as Practice-led Research. In Speculation and Innovation: Applying Practice-Led Research in the Creative Industries. Brisbane: Queensland University of Technology,Google Scholar
Heidegger, M. 1966. Discourse on Thinking. New York: Harper & Row.Google Scholar
Hermann, T. 2011. The Sonification Handbook. Berlin: Logos Publishing House.Google Scholar
Hunt, A. and Hermann, T. 2004. The Importance of Interaction in Sonification. Proceedings of the 10th International Conference on Auditory Display, Sydney, Australia.Google Scholar
Keller, D. 1999. touch’n’go: Ecological Models in Composition. Master of Fine Arts thesis, Simon Fraser University, Burnaby, BC. www.sfu.ca/sonic-studio/srs/EcoModelsComposition/Title.html (accessed 20 December 2023).Google Scholar
Keller, D. 2000. Compositional Processes from an Ecological Perspective. Leonardo Music Journal 10: 5560. https://doi.org/10.1162/096112100570459.CrossRefGoogle Scholar
Keller, D. 2012. Sonic Ecologies. In Brown, A. R. (ed.) Sound Musicianship: Understanding the Crafts of Music. Newcastle upon Tyne: Cambridge Scholars, 213227.Google Scholar
Keller, D. 2014. Characterizing Resources in Ubimus Research: Volatility and Rivalry. Cadernos de Informática 8(4): 5768.Google Scholar
Keller, D., Aliel, L., and Silva, C. R. 2018. The Handy Metaphor: Bimanual, Touchless Interaction for the Internet of Musical Things. Annals of the ubiquitous Music Workshop/Proceedings of the Ubiquitous Music Workshop (ubimus 2018). São João del Rei, Brazil: Ubiquitous Music Group.Google Scholar
Keller, D. and Berger, J. 2001. Everyday Sounds: Synthesis Parameters and Perceptual Correlates. Proceedings of the VIII Brazilian Symposium on Computer Music (SBCM 2001). http://gsd.ime.usp.br/∼lku/site-sbcm/2001/ (accessed 20 December 2023).Google Scholar
Keller, D. and Capasso, A. 2006. New Concepts and Techniques in Eco-Composition. Organised Sound 11(1): 5562. https://doi.org/10.1017/S1355771806000082.CrossRefGoogle Scholar
Keller, D. and Lazzarini, V. 2017. Ecologically Grounded Creative Practices in Ubiquitous Music. Organised Sound 22(1): 6172.CrossRefGoogle Scholar
Keller, D., Lazzarini, V. and Pimenta, M. S. (Eds.). 2014. Ubiquitous music. Berlin: Springer.CrossRefGoogle Scholar
Keller, D., Messina, M. and Oliveira, F. Z. 2020a. Second Wave Ubiquitous Music. Journal of Digital Media & Interaction 3(5): 520.Google Scholar
Keller, D., Simurra, I. and Messina, M. 2020b. Toward Anticipatory Ubimus. EAI Endorsed Transactions on Creative Technologies, 7(24). https://doi.org/10.4108/eai.13-7-2018.164664.CrossRefGoogle Scholar
Keller, D. and Truax, B. 1998. Ecologically Based Granular Synthesis. Proceedings of the International Computer Music Conference (ICMC 2010). Ann Arbor, MI: MPublishing, University of Michigan Library.Google Scholar
Lazzarini, V. and Keller, D. 2021. Towards a Ubimus Archaeology. In Proceedings of the 10th Workshop on Ubiquitous Music (UbiMus 2020). Porto Seguro: g-ubimus.Google Scholar
Lazzarini, V., Keller, D., Otero, N. and Turchet, L. (Eds.) 2020. Ubiquitous Music Ecologies. London: Routledge.CrossRefGoogle Scholar
Lewis, G. E. 2000. Too Many Notes: Computers, Complexity and Culture in Voyager. Leonardo Music Journal 10: 3339. https://doi.org/10.1162/096112100570585.CrossRefGoogle Scholar
Lima, M. H., Keller, D., Pimenta, M. S., Lazzarini, V. and Miletto, E. M. 2012. Creativity-centred Design for Ubiquitous Musical Activities: Two Case Studies. Journal of Music, Technology and Education 5(2): 195222.CrossRefGoogle Scholar
Messina, M. and Aliel, L. 2019. Ubiquitous Music, Gelassenheit and the Metaphysics of Presence: Hijacking the Live Score Piece Ntrallazzu 4. Perception, Representations, Image, Sound, Music – 14th International Symposium on Computer Music Multidisciplinary Research, Marseille, France, 14–18 October.Google Scholar
Messina, M. and Aliel, L. 2023. Things, Objects, Subjects and Stuff: IoMuSt and Ubimus Perspectives on AI. IEEE Big Data 2023. 1st Workshop on AI Music Generation (AIMG 2023), Sorrento.CrossRefGoogle Scholar
Nance, R. 2007. Compositional exploratçions of Plastic Sound. PhD thesis, De Montfort University.Google Scholar
O’Callaghan, J. 2013. Orchestration of Ecology, as Ecology. Proceedings of the Music and Ecologies of Sound Symposium, Paris, France. www-artweb.univ-paris8.fr/spip.php?action=acceder_document&arg=1397&cle=76ad47cf4b0914fe3ddb826b37426eff6bdec9e6&file=pdf%2Focallaghan_orchestrationofecologyasecology.pdf (accessed 11 January 2022).Google Scholar
Opie, T. and Brown, A. 2006. An Introduction to Eco-Structuralism. Proceedings of the International Computer Music Conference (ICMC 2006). Ann Arbor, MI: MPublishing, University of Michigan Library, 9–12.Google Scholar
Rolfe, C. and Keller, D. 2000. Decorrelation as a By-Product of Granular Synthesis. Proceedings of the XIII Colloquium on Musical Informatics. L’Aquila: AIMI. https://ccrma.stanford.edu/∼dkeller/pdf/Decorrelation2000.pdf (accessed 20 December 2023).Google Scholar
Simurra, I., Messina, M., Aliel, L. and Keller, D. 2023. Creative Semantic Anchoring: Creative-Action Metaphors and Timbral Interaction. Organised Sound 28(1): 6477. https://doi.org/10.1017/S1355771822000322.CrossRefGoogle Scholar
Stolfi, A. S., Milo, A. and Barthet, M. 2019. Playsound. Space: Improvising in the Browser with Semantic Sound Objects. Journal of New Music Research 48(4): 366–84.CrossRefGoogle Scholar
Wessel, D. and Wright, M. 2002. Problems and Prospects for Intimate Musical Control of Computers. Computer Music Journal 26(3): 1122.CrossRefGoogle Scholar
Figure 0

Figure 1. Spectrographic scores used in M335.

Figure 1

Figure 2. M335’s performance.

Figure 2

Figure 3. Diagram of the M335 interaction model.

Figure 3

Figure 4. A consumer-level camera was attached to the double bass to enable data capture and parametric control in M335.

Aliel et al. supplementary material

Aliel et al. supplementary material 1

Download Aliel et al. supplementary material(Video)
Video 45.6 MB
Supplementary material: File

Aliel et al. supplementary material

Aliel et al. supplementary material 2

Download Aliel et al. supplementary material(File)
File 53.2 KB