Ludified documents the Gamified Audiovisual Performance and Performance Practice (GAPPP) research project (2016–20). The initiative was based at the University of Music and Performing Arts Graz and led by composer Marko Ciciliani. Its focus was the artistic potential of computer-game elements in audiovisual works and how they impact performance and reception. The team used the term ‘ludified’ to describe this introduction of game elements, thereby distinguishing their purposes from those of marketing and education (often associated with the term ‘gamified’). The project facilitated the commission, creation, performance and assessment of several works. It also integrated practice-based compositional research, performance studies and audience-perception research. This tripartite approach is reflected in the book’s editorship: Lüneburg is a performer and performance-studies scholar, while Pirchner is a musicologist, designer and programmer who addresses audience perception. The resulting book provides foundational frameworks and research models for addressing ludified works, and it will be of interest and value to artists and scholars seeking to create, conceptualise or analyse such works. The book’s scholarly reach also has implications for the broader domains of interactive music and mixed-media performance.
The book is creative and handsome in its design, presenting two volumes in one in a playful manner. The first volume discusses the overall research design of GAP, provides a set of theoretical papers and discusses several artworks. The second volume focuses on Ciciliani’s ludifed works. The built-in ribbon bookmark terminates at a chrome USB data stick containing documentation of the works, which is also available on an associated website. The full-colour prints in the book display fine artistic detail. An iphone or tablet app can be downloaded to provide an augmented-reality superimposition for some of the images from Ciciliani’s installation Anna & Marie. The 14 performance artworks presented (and some installation variants) provide a strikingly diverse and imaginative range of approaches.
Volume 1
Ciciliani’s chapter presenting a polar-diagram tool for analysis of gamified audiovisual works provides a particularly valuable contribution to the field, and it is likely be widely adapted by artists and scholars seeking to conceptualise or analyse gamified works. The diagram comprises ten parameters along five axes, organised into two halves: one addressing compositional choices and the other the role of the performer. For example, the determinism axis focuses on the level of determinacy in the compositional domain and on the level of spontaneity vs. planning in the performance domain. The other axes are interface, composition, presence and ludus. Each axis-spoke is given a value, resulting in a spider-web diagram. The axis definitions used in this approach are at times complex and ambiguous, perhaps unavoidably so, given the vast range of potential approaches in the domain. Further, as Ciciliani notes, ‘as each parameter is defined based on qualitative evaluations, there are not absolute criteria according to which the reference values for the individual parameters are selected’ (p. 34). Though the axis values may be highly subjective, they point to relevant questions that can greatly clarify the nature of works and facilitate comparison between them. (Given the tripartite design of GAPPP, this reviewer wonders if and how audience perception could eventually be integrated into such diagrams.)
The next section of the volume, ‘Proposal – Response’, provides a selection of texts derived from a GAPPP symposium, addressing a variety of artworks and issues. The symposium utilised a ‘proposals’–‘responses’ structure. One example ‘proposal’ is Rob Hamilton’s design and composition of a virtual-reality string quartet (Trois Machins de la Grâce Aimante), in which each of the performers have their own VR goggles and view a shared virtual space while performing on virtual ‘hand-held’ instruments (called Corotets). In a separate chapter, Lüneburg considers Hamilton’s piece from the performers’ point of view, in part using Ciciliani’s polar diagram as a framework. Her perspective is insightful, particularly regarding the challenges the performers faced in controlling the virtual instruments and interacting as a quartet; Lüneburg also sheds light on the way the system encouraged the performers to be curiosity based in performing and discovering new expressive gestures. In addition, she discusses the reception of the work. Lüneburg then asks the following: ‘What does the Corotet offer that the physical string instruments cannot? How can we use these qualities in our performance? Is there an alternative to the way game aesthetics in [the work] could be employed to emphasize a ludified approach or involve the audience? Would any of this increase the agency for the players and offer a richer space of possibility that in consequence would result in more meaningful playing?’ (p. 111). Her ruminations on these questions identify salient issues and new paths for exploration, effectively demonstrating the value of integrating GAPPP’s three perspectives.
Volume 2
This volume investigates Ciciliani’s ludified work. In his Introduction to the volume, musicologist and dramaturg Julian Kämper reflects on the rise and significance of integration of game elements into audiovisual art, and he surveys the motivation and scope of Ciciliani’s work in that context. In ‘Virtual 3D environments as composition and performance spaces’, Ciciliani provides brief historical reflection on the use of virtual 3D space in concert-based works and considers (i) how the spatial distribution of sound sources can shape a composition’s time factor, (ii) the potentials of polyspatiality, (iii) challenging the visual exclusivity of projection screens and (iv) data-mapping strategies.
Pirchner’s ‘Investigating audience experience’ examines the attention of the audience when confronted with visual stimuli (performers on stage and/or video projections), music and other media. He describes the development and implementation of the touch-tablet based, open-source interactive real-time measurement of attention (IRMA) system – a continuous-measurement system assessing whether and when the audience is focused on sound, visuals or the performer. Test results are discussed in detail for Ciciliani’s Kilgore.
In the chapter ‘Tokens of enunciation in multimedia performance’, Ciciliani discusses how multimedia performances address their audience, with intent to establish ‘a non-reductionist approach that highlights the complexity of something that is often taken for granted’ (p. 69). This includes exploration of the nature and typology of material-based, spaced-based, reference-based and subject-based ‘enunciations’. Surprisingly, given the setting of the chapter, we do not receive many clues about how these concepts apply to specific works in the volume.
Ciciliani’s Anna & Marie is an installation with a performance element for electric violin (performed by Lüneburg), baroque violin (performed by Susanne Scholz), two 3D worlds, augmented reality, light design and generative electronics. It explores two Enlightenment-era anatomists (Anna Morandi and Marie Bihéron) through elements of ‘educative games’ and ergodic storytelling (with the performers impacting how the story proceeds). The foundational conceit is an imaginary encounter between Morandi and Bihéron.
The chapter ‘Anna & Marie, a performative installation built on ergodic storytelling – two project reports’ (by Ciciliani and Lüneburg) again addresses both composer and performer perspectives. It does so by alternating between statements by the authors. The interleaving of commentary is not linear, with the text sometimes jumping between disconnected issues. While this creatively counters the normal expectation of a scholarly paper, it succeeds in highlighting the different perspectives of the authors and the way these can complexly intertwine in a holistic consideration of the work. The chapter closes with a section on the ‘voice of a visitor’, offering in this case only a brief nod to the promised tripartite approach.
Pirchner’s chapter ‘The real-time score of Anna & Marie: ergodic and emergent qualities’ considers the use of real-time and spatial technologies as strands in Anna & Marie – in which interplay of multiple entities serve as dynamic scores, including real-time generation of a symbolic score and the virtual performance space as a score. The chapter makes the point that there are profound potentials in the creation of dynamic scores, but it understandably has challenges in identifying their ultimate implications.
Aesthetics
For this author, it was viewing the artworks that stimulated the most complex ruminations on the future of ludified artworks. In his chapter (vol. 1), composer Christopher Ressi notes a long-standing resistance among critics to the idea that the gaming medium can produce works of art, suggesting that ‘most artists who use computer games as a medium for their art tend to adopt quite broad notions of what constitutes a game’ (p. 48). Even allowing for this broad notion, the artworks in Ludified seemed to succeed more in highlighting the challenges and wide range of possibilities available than in fully proving the critics wrong.
These are early days for these new approaches. The conceptual, technical, compositional and performative challenges are substantial. One possible stance when operating at such a forefront is to use an avant-garde aesthetic posture to make the task more achievable and manageable. However, this may also obscure what is truly achieved artistically – or what fails. If, for example, one posits that an artistically coherent or impactful experience can emerge from the dynamics of a ludified work, how can one validate that hypothesis if and when the resulting experience is often chaotic or decidedly difficult to ‘read’, or resists establishing dramaturgically graspable relationships? While this may be a necessary interim creative step, the key question in the exploration of such new potentials ultimately becomes this: when does the aesthetic impact become greater than – and fully justify – the novelty and complexity of the means?
It is notable that the GAPPP study chose not to inquire into the actual aesthetic reception of the audience. It addressed relatively objectifiable (albeit quite salient) issues, such as what draws the audience’s attention. Yet developing the full potential of ludified works also demands that a different class of questions be addressed in the future, such as ‘How coherent or incoherent did you find various parts of the performance? What do you feel emotionally about Anne & Marie after experiencing this installation? What elements of this performance most seemed to provide a sense of ‘meaning’, if any, and why? What elements of this performance would you like to experience more of, and why? Which were unrewarding and why?’ If we aspire to take audience reception seriously, perhaps such vital questions cannot be long avoided, even if this challenges the traditional mythos of the autonomous artist.
Pedro González Fernández’s String Mask Overflow (for violin, movement, shadow, video and tape) was a ‘response’ work to Hamilton’s Trois Machins de la Grâce Aimante. This reviewer finds the former one of the most provocatively poetic of the Ludified works, though it also appears to be one of the least ‘ludified’ in its actual implementation. Arising in part from Fernández’s own experience as a string player, it implies a critique of the whole ludified venture and an expression of sympathy for performers. The tasks that the performer (Lüneburg) of String Mask Overflow must undertake suggest a kind of multi-layered dehumanisation (of the performer) with confused and fractured relationships among musical heritage, identity, gaming and machinic control, and the quest for meaningful agency. Even though the reader is not often told so, these issues are implicated throughout Ludified. In Fernández’s final scene, the performer shadow-eats her violin. Is this an act of reconciliation via incorporation and embodiment, or one of destruction? Perhaps it is both. Perhaps we do not yet know. We can only wait to see if and how ludified works can evolve into impactful, inspirational and transformational artworks that embrace the full humanity of the audience, performer and composer alike.
Open access
