Alternative Approaches to Chinese Imagery: Audiovisual aesthetics in A Reflection in the Brook

Abstract

The importance that Chinese composers attach to their nation’s cultural traditions in their electronic music compositions has become a dominant trend in Chinese electronic music. This has generally led to a ‘Chinese imagery’ in Chinese electronic music compositions. Among China-inspired electroacoustic music, the interactive multimedia work A Reflection in the Brook (小青, 2013) shows a unique expression. The author explores how the composer recreated a controversial female figure in Chinese history through a completely real-time audiovisual language: Feng Xiaoqing, thereby presenting Chinese imagery in electroacoustic music through an alternative approach. The audiovisual relationship in A Reflection in the Brook will be analysed through the lens of Michel Chion’s audiovisual theory and the perspective of musical composition techniques, further presenting the audiovisual aesthetics of multimedia electronic music.

1. INTRODUCTION

Jin Ping’s (金平) interactive multimedia electronic music A Reflection in the Brook ¹ is regarded as an iconic work in the development of Chinese multimedia electronic music and is the first work in Jin Ping’s Women’s Trilogy series. ² Beforehand, Jin Ping’s An Enchanted Drum (被施了魔法的鼓, 1998) was the first interactive electronic music in China. This article first discusses the uniqueness of A Reflection in the Brook in expressing ‘Chinese imagery’ from the perspective of creative concept. It overturns some of the historical views of the protagonist Feng Xiaoqing, and uses Western musical instruments, abstract audiovisual presentations and modern dance performances to recreate the image of Chinese female character that is not culturally directed. Based on the ‘The Three Paths’ (Landy 2018) of Chinese composers using Chinese elements as defined by Leigh Landy, this article aims to further propose the possibility of the path-breaking approach in A Reflection in the Brook becoming ‘The Fourth Path’. Second, it analyses the real-time narrative approach of sound and image in A Reflection in the Brook from a creative perspective. Finally, in the light of Michel Chion’s sound theory, this article explores the implications of the audiovisual aesthetics in A Reflection in the Brook for the audiovisual relationship in multimedia electronic music, in the dimensions of ‘audiovisual subordination’ and ‘longitudinal and horizontal perspectives on audiovisual.

2. THE FOURTH PATH

The art of electronic music originated in the West, and only entered the vision of Chinese artists in the 1980s. It has always been the creative pursuit of Chinese composers to use new artistic languages and compositional approaches to create artistic expression with vitality. When looking at the trends in Chinese electronic music since its beginning to present day, we find that a very mainstream tendency has developed in many composers as it enters the twenty-first century – the importance they attach to the outstanding cultural heritage of the various Chinese nationalities.

This emphasis has led to a unique phenomenon in the field of Chinese electronic music composition – composers who are mostly inspired by Chinese culture to some extent and who try to present elements or contexts of Chinese culture (e.g., presenting a certain sound, a certain traditional instrument or a certain cultural phenomenon) in electronic music. In short, the tendency to express Chinese culture is evident in the compositions, and this has led to a universal Chinese imagery in Chinese electronic music compositions.

Leigh Landy has proposed ‘China-inspired Electroacoustic Music’ (Landy 2018: 81) in relation to the preceding phenomenon. He found and summarised three paths for Chinese composers to use Chinese elements.

1. 1. Sampling.

2. 2. The use of Chinese instruments and/or musical approaches.

3. 3. Inspiration from Chinese culture. (Landy 2018: 82)

In recent years, many influential works have used these three paths to represent Chinese imagery. These include, but are not limited to, Beijing Aria (北京咏叹) by Zhang Xiaofu (张小夫), which uses the Beijing accents of The Book of Family Names (百家姓) and the Three-Character Scripture (三字经); the use of Peking opera chanting, rhyming and sprechstimme in The Ghosts of the Mountain (山鬼); the use of Tibetan voices and Lama chanting in the multimedia symphony Nuo Ri Lang (诺日朗); the humorous and witty lyrics of Wuxi nursery rhymes in Fan Ling’s Grandpa (老伯伯), using chosen linguistic vocals with distinctive ethnic characteristics as sampling; Jin Ping’s Ferry Tale (陈三娘) and Tao Yu’s Ah Q (阿Q) both use elements of local Chinese opera. Many other works such as Zhang Xiaofu’s Dancing Ink (墨韵灵飞), Zhuang Yao’s (庄曜) Reverie of the Mountain (山的遐想), Li Pengyun’s (李鹏云) Orchid-liked Temperament (气如兰) and Ink Shennongjia (水墨神农架), and Guan Peng’s (关鹏) Universal Peace (普安). These works are either composed using a combination of traditional Chinese folk instruments and electronic music or by using samples of folk instruments and special playing methods. The forms of expression in these works are different, but there is a clear commonality in the use of Chinese culture as the main subject of expression.

Among the many ‘China-inspired electroacoustic music’, composer Jin Ping’s interactive multimedia electronic music A Reflection in the Brook is unique. It reimagines a controversial female figure in Chinese history through the lens of new media, but without using any culturally ‘directional’ or ‘iconic’ elements in its sound and visual presentation. Although it is partially similar to the third path of ‘Inspiration from Chinese culture’ as interpreted by Landy, in terms of creative inspiration, it does not conform to ‘focuses on Chinese extra-musical cultural ideals being reflected musically, whether they are to do with Taoist avoidance of formalisation or the ability to interpret a poem of Li Po in a wide variety of manners‘ (Landy 2018: 82).

Rather than expressing certain Chinese cultural ideals, A Reflection in the Brook is a modern subversion of historical perspectives, which is in a way ‘critical’ in itself. This might add a fourth path to ‘China-inspired electroacoustic music’, based on Landy’s ideas – the subversion and reconstruction of traditional culture. Although this path is a minority one, it is an alternative way of conveying Chinese imagery in electronic music.

The work is inspired by the stories of ancient Chinese characters. The main character, Feng Xiaoqing, is a woman from the late Ming Dynasty (ad 1617–35), who was quite controversial because of her tragic life and unusual behaviour. A talented poet, Xiaoqing died of depression after a life of misfortune in which she took her own shadow as her love interest. In the three hundred years since her death, many scholars have written stories of her, and plays about her life have been performed on stage.

In the Ming and Qing dynasties, there are plays about Xiaoqing, such as The Jealousy-curing Soup (疗妒羹) by Wu Bing, The Shadow of Spring Waves (春波影) by Xu Yejun, Merry Court (风流院) by Zhu Jingfan, A Long-lived Love (情生久) by Chen Xiufang, Snow at the West Lake (西湖雪) by an anonymous author and Dream of Plum Blossoms (梅花梦) by Zhang Dao. It has become a widespread ‘Xiaoqing phenomenon’, with literary figures writing about the subject of Xiaoqing.

During the Republican period (ad 1912–49), renowned sociologist Pan Guangdan (潘光旦), in Feng Xiaoqing’s Psychosexual Disorder Revealed (冯小青心理变态揭秘), analysed Feng Xiaoqing’s ‘shadow love’ psychologically with the help of Freud’s psychoanalytic theory and the sexual psychology. He draws an analogy with Narcissus, a beautiful man in ancient Greek mythology, and argues that Xiaoqing, as the real-life embodiment of Narcissus, who, as a result of her seclusion and sexual repression on the lonely mountain, developed a psychological distortion from ‘ego-love’ to ‘shadow-love’.

In 1927, the scholar Wen Yiduo (闻一多) created the picture Looking in the Mirror (对镜) (Figure 1) for a book by Pan Guangdan (Pan 1990: 5):

生动地表现了冯小青的悲剧命运和变态心理。作者选取了小青晨妆对镜的瞬息…….画面上冯小青披着睡衣，左肩半露，镜中人面容憔悴，鬓发散乱，双眉微颦，郁结难开，塑造了封建社会不幸女性的内在性格 ³

Which vividly represents Feng Xiaoqing’s tragic fate and distorted psychology. The author has chosen a fleeting scene of Xiaoqing in her morning makeup facing the mirror … The picture shows Xiaoqing in her nightgown, her left shoulder half exposed, her face haggard, her hair scattered, her eyebrows knitted, and her depressed knottedness difficult to relieve, which portrays the inner character of an unfortunate woman in feudal society. (Pan 1990: 6)

Figure 1. Looking in the Mirror by Wen Yiduo.

This is also the only painting of a figure by Wen Yiduo that has survived, and is therefore particularly valuable.

It is easy to see that the historical figure of Feng Xiaoqing is a miserable woman, unhappy in her marriage, unable to express her talent, obsessed with her own ‘shadow love’ and ultimately dying alone. The attitude of the literati from the Ming and Qing dynasties to the Republic of China towards Feng Xiaoqing also coincided with pity, sympathy and hatred of the feudal shackles. In the computer music A Reflection in the Brook, however, composer Jin Ping does not continue this expression, he does not directly recreate and perpetuate traditional views or use new technology to tell old stories. He ‘reinvents’ it with a modern ideology and aesthetic concept, and even this ‘reinvention’ is based on the subversion and reversal of ‘tradition’, which is a reinterpretation of traditional culture under a new media perspective.

3. REAL-TIME AUDIOVISUAL NARRATIVE

A Reflection in the Brook has a very distinct narrative, its figure, plot and conflict gives it many characteristics similar to those of theatre and dance. In terms of techniques of narration, the work does not use any pre-programmed audio or video, but is entirely in real time – the sound and visual images are generated by live performance and computer in real time, using pre-programmed Max/MSP/jitter patches to sample, capture, morph and generate the instrumentation and movement of the actors on the fly and to create multiple interactions between multiple artistic elements (Figure 2). This method of multilinear ‘real-time narration’ by multiple media is not only a new attempt in the creation of multimedia electronics, but also undoubtedly brings great challenges to both technology and art.

Figure 2. The layout of the performance of A Reflection in the Brook.

The whole work uses time as a clue to present the ups and downs of Xiaoqing’s fate and experiences. It expresses the psychological state of the main character Xiaoqing at different times through four sections of themes as shown in Table 1.

Table 1. Themes and content at four sections of A Reflection in the Brook

3.1. Real-time narrative of sound

In A Reflection in the Brook, the live performance of the violin plays an important role. On the sound level, it is the source of the material for the real-time sampling and morphing of the max audio program, which is the ‘morphing base’ for all the electronic music sound in the work. On the other hand, some of the parameters output by the real-time sound sampling and analysis module are also transmitted to the real-time video processing computer (program) via a wireless network. These data are used to synchronise the live audio with the live video and to visualise some of the audio signals, in short, the violin is also used to trigger and control the video changes in real time.

The opening of the work is introduced by the interplay of several sets of figures that using special bowings for the violin and real-time audio. After a brief atmospheric buildup, the solo violin, like a storyteller, begins to play the lyrical, long melody of the first section a free rhythmic E–F♯–B–E motive of the theme (Figure 3). The Max real-time audio program samples the melodic theme in real time and then adds multiple delays and reverbs at different time positions to make the distorted electronic music sound like echoes in the mountains. The music in the introductory passage focuses on the heroine, Xiaoqing, living alone in the lonely mountains. The composer has effectively created the lonely situation of Xiaoqing, through the melody of the song, the multilayered overlapping delayed echoes, the changes in the surround sound field and the simplicity of the layering of volumes, which sets the scene for the conflict that follows.

Figure 3. Core materials of the melodic theme E–F♯ –B–E in violin part.

The ‘Mirror’ section continues the same musical introduction as the introductory section, except that the music begins to take on a more ‘dissonant’ character. Here the composer focus on the dialogue between Xiaoqing and herself in the mirror. This ‘mirroring’ is multifaceted, with the violin playing representing the real Xiaoqing, a series of thematic figures consisting of special bowings – including ricochet on the bridge with the hair of the bow, ricochet with the wood of the bow, and continual pizzicato – creating special sound effects that represent a mirror image of Xiaoqing (Figure 4), and a completely abstract sound that has been distorted by various computerised real-time audio processing techniques to symbolise another, more distant mirror image of reality. The violin playing in this section is highly improvised – the violinist plays the first thematic figure and then, based on real-time feedback from the computer, randomly selects a subsequent figure from a number of pre-designed figures. The computer analyses the data on the pitches played to realise four feedback modes: separation of the performance fragments into finely divided and dense figures; change the pitch of phrases and regroup them; simulated pizzicato strings; and random generation of harmonic sounds. The violin and the computer are constantly ‘listening’ and ‘answering’, creating a dialogue and interaction between the human and the computer. The sound also alternates between ‘reality’ and ‘distortion’, with layers of metamorphosis and increasing tempo, foreshadowing the conflicting and complex feelings of the protagonist as she constantly asks questions of her ‘mirror’ self in her search for identity but cannot find the answers.

Figure 4. Core materials of the ‘mirror theme’ in violin part.

‘Death of Xiaoqing’ is the climax of the work and is completed by the ‘Death’ and the ‘Epilogue’, which symbolises the sublimity of the soul. In the process of expression of the protagonist’s death, Xiaoqing’s struggle is intense – the music develops in a highly dissonant manner: the dissonance of the timbre, the density of texture, the rhythmic complexity and the loudness all increase. The Max audio program is overlaid in real time with delay effects with a certain rhythm and highly irregular delayed voices, creating a disordered, dense rhythm. Later in the ‘Death’ section, some MSP formant processing is added to the real-time sound processing, intensifying the tension of the timbres changes, making the sound increasingly torn, thick and discordant, presenting an orchestral, group effect, and with a few rapid violin glissandos pushing the discordant sound to the extreme, heralding the arrival of the “Death” moment. The extremely dark, dissonant sound energy quickly diminishes as the violin and the computer’s real-time audio section begin to play long hymn tunes together, swirling around the entire sound field, diminishing in volume and fading away, as if Xiaoqing’s soul is leaving the body.

After a moment of silence, a lighter motive which is symbolic of the ‘Soul Sublimation’ of the protagonist leads into the ‘Epilogue’ section. The E–F♯–B–E scale, with E as the dominant note, is in fact a variation on the ‘Introduction’ theme. And there are many similarities between the musical processing of the entire ‘Epilogue’ section and the ‘Introduction’ section, both introduce and create a mutual echo between the beginning and the end. The tone of the epilogue is bright, warm and harmonious, in contrast to the highly dissonant ‘Death’ section that precedes it, revealing a certain sense of ‘purification’. The violin is dominated by a melodic and functional harmonic sound, and the sound processing in the Max audio program continues the method of adding multiple time delays to real-time audio samples as in the ‘Introduction’ section (Figure 5). The difference here is that the delayed part gradually increases with each sample node and are carefully designed in advance in terms of rhythms, which are added in real time to vary the pitch of the different ranges based on the pitch frequency of the live performance, thus constantly gathering the power of the part and creating a group-like sound effect. The music of the ‘Epilogue’ is significantly brighter and more powerful than that of the ‘Introduction’. The music here is hopeful and makes the ‘Epilogue’ the most glorious section of the piece, forming two juxtaposed sections of climax with the preceding ‘Death’, the contrast between them breaking the conventional ‘arch’ structure. It is easy to see the composer’s approach to the ‘Death of Xiaoqing’ in the arrangement of the music – death is not the end, but the beginning of a new life, and this beginning is open-ended. It gives the audience space for expectation and imagination of Xiaoqing’s ‘rebirth’.

Figure 5. Adding multiple time delays to real-time audio samples in Max/Msp (Wang 2021: 88)

3.2. Real-time narrative of images

The artistic presentation of A Reflection in the Brook is multilinear. Each medium is different in its approach and presentation, and the mode of interaction between the mediums changes according to the needs of the performance, thus expanding the expressive space of the work.

As a highly dramatic work, the image presentation of A Reflection in the Brook is particularly important. It not only provides the thread of the story, moves the plot forward and switches scenes, but also helps the music to convey the character’s inner struggles and intense dramatic conflicts to the audience. All the images in A Reflection in the Brook are based on real-time generation and real-time interaction. Some of them are triggered by the sound signals from the scene, others are generated by capturing the actors’ movements. The ‘real time’ makes it difficult to create and perform, requiring a precise connection between sound and image and a rhythmic synchronisation. From a creative conceptual point of view, this narrative approach also gives the work the character of new media art, reflecting the composer’s aesthetic awareness of reshaping traditional national culture with a modern mindset.

The four main visual modules of the work’s images correspond to the ‘Introduction’, ‘Mirror’, ‘Death’ and ‘Epilogue’. The narrative themes in the four sections, symbolising the different psychological states of the protagonist in the four periods. The author has briefly summarised the basic logic of the meaning and technical realisation of the visual themes in the work, as shown in Table 2.

Table 2. The meaning and basic technical logic of visual themes

As can be seen from Table 2, the four visual modules with clear symbolic meanings are all completed by the jitter programming system, with each thematic module corresponding to a different programming that reflects the changing fortunes of the protagonist through different visual representations.

The ‘line drawing’ theme in the first visual module tends to an atmospheric description, where the composer does not aim for complete synchronisation of sound and image, but keeps the two together in keynote. The music here is simple, reflecting a mood of solitude. The vision uses a simple presentation – a camera captures the image of the dancer’s performance and triggers the jit.qt.grab object in the vision program to convert the image into matrix data in real time, where the parameters inside the cell are monitored and analysed. Since the difference between the cell parameters is a direct result of the visual difference between light and dark. The lower the cell parameters, the greater the number of lines generated. The jit.gl.mesh object dynamically generates line patterns in low parameters areas, creating an effect similar to the ink lines in traditional Chinese painting. These black and white lines are intertwined with light and dark, setting off a cold and bleak visual tone, indicating the loneliness and anguish of the characters.

The second section focuses on the creation of the ‘Mirror’ effect, which is an important plot in the piece. Historical interpretations of Feng Xiaoqing’s ‘shadow love’ are based on her particular fondness for looking in the mirror – her obsession with the water reflection and herself in it. In contrast to previous scholars’ views, Jin Ping offers an alternative interpretation of this act. The ‘mirror image’ passage reflects Xiaoqing’s continuous search for herself – in the face of the unchangeable reality of her time, she tries to discover herself by ‘asking herself questions’ in the mirror. The music and images here present a sense of exploration, symbolising the awakening of the protagonist’s feminine consciousness. Here the inner conflict of the characters is revealed, but not to the extent of an intense conflict. In the visual process, the author has designed two different ‘mirroring’ effects for Xiaoqing’s ‘self-searching’. The first is a real-time delay of the image: the ‘jit.qt.grab’ object acquires the image of the actor’s performance from the camera. The 30 frames are pre-stored into a ‘buffer’ using the ‘jit.matrixset’ object, the five ‘jit.matrixset’ objects are used to play the 30 frames at staggered times, adjusting the speed of the delay in real time to create a ‘staggered’ delay effect. It is a gentler mirroring effect, as if to speak of the inner confusion of the character. The second is a more complex ‘multi-person dance’ effect, showing Xiaoqing’s constant search for ‘who am I?’ in multiple mirror images. In terms of technical means, a combination of camera-delayed recording playback and real-time performance by the actors is used. The camera records the first theme performed by the dancers in its entirety, then gradually fades into the previously recorded first theme as the dancers perform the second theme and combines it with the second theme video being recorded in real time, thus achieving a real-time interactive effect between the dancer’s performance and the multiple digital images.

In ‘Death of Xiaoqing’, the composer uses a completely abstract ‘illuminated geometry’ to express the intense inner struggle of the character. The image begins to interact with the sound in a highly synchronised, real-time manner. The position of the light is driven by the sound signal, the iPad remote control and the kinect joint position. The switching speed of the graphics is directly controlled by the bow speed of the violin. It continues through to the epilogue. The bright, dazzling graphic tones contrast with the darkness of the visual backdrop, as if plunging into an abyss, making the ‘Death’ the darkest part of the piece. The abstract graphics here twist, split, distort and switch in rapid succession to the extremely incongruous music. A dazzling light constantly shines against the dark and depressing backdrop, as if the protagonist is in fierce confrontation with the hideous forces of reality. Ultimately, these struggles seem in vain – Xiaoqing is in a frenzied emotion as she dies. The violin’s fierce bowing shifts the digital image rapidly between eight still images and a black screen, matching the extremely tense body movements of the dance, until the node symbolising the death of Xiaoqing comes to an abrupt end. The theme of ‘Particle Eruption’ is presented, with large, black, dynamic particles rising in an eruption behind the ‘dead’ Xiaoqing, a metaphor for the gushing of blood, representing the ultimate release of emotion and catharsis (Figure 6). Here the composer uses kinect ⁴ to position the dancer’s body and uses the body coordinates to control the coordinates of the radiation source, thus ensuring that the particles are always emitted from the dancer’s back as she moves.

Figure 6. The actress playing Xiaoqing moving with particles erupting from her back in the performance.

The ‘Epilogue’ section is the brightest of the piece, and one that is rich in the spirit of ‘romanticism’. After a moment of silence, the dancer rises slowly from the stage. Abstract shapes made up of luminous dots and lines appear, accompanied by music and dance body movements, and become geometric halos of ‘hope’ (Figure 7). For the first time, these halos of light appear in bright colours other than black and white, symbolising the rising of Xiaoqin’s soul in hope (Figure 8). In the visual processing of Xiaoqing’s death, the composer breaks away from the traditional impression of an oppressed woman dying of depression in a feudal society. Her extreme calmness and beauty express a feminine bravery that beyond reality and embodies her silent yet powerful resistance. From the ‘dancer’s resurrection from the dead’ to the ‘bright abstract figure of rebirth’, it is clear that the meaning of Xiaoqing’s death is more of a yearning for rebirth and light. The author’s anticipation of Xiaoqing’s rebirth is full of positive expectations. This expectation goes beyond life and death in the secular sense and explores the continuation of spiritual forces beyond the material and physical.

Figure 7. In the finale section of the performance, Xiaoqing interacts with a bright abstract pattern, signifying ‘new life’.

Figure 8. Xiaoqing’s rebirth.

4. AUDIOVISUAL AESTHETICS

As a multimedia electronic music work based on multiple interactive technologies and real-time computer technology. The real-time narrative of multimedia provides a richer dimension for the artistic expression of A Reflection in the Brook, and also brings some insights into the ‘audiovisual relationship’ of multimedia electronic music.

4.1. Visu-audition

In multimedia works, the relationship between the visual and auditory senses is commonly perceived as a superposition of the two senses, but this perception is still based on the idea of the visual and auditory senses as separate entities. Michel Chion, a leading French sound theorist, has argued that ‘audiovisual’ is a total perception, not a simple addition of two different perceptions, sound and image. Like a chord in music, it is not a superposition of three different notes, but a holistic form, defined by the intervals between the notes (Chion 2014: 1). In order to demonstrate that the two senses are not simply additive, equal or symmetrical, Chion cites two concepts: ‘audio-vision’ and ‘visu-audition’ (Chion 2014: 198-99), ⁵ indicating two different audiovisual subordination relationships in the phenomenon of ‘audiovisual association’.

From previous listening experiences, most audiovisual representations in multimedia electronic music tend to be more in favour of the latter (visu-audition), that is, ‘the type of perception deliberately focuses attention on the sense of hearing, and our listening is accompanied, emphasized and aided by an image content, which influences what we hear and may lead us to project a particular perception onto it’ (Chion 2014: 199). To be more precisely, unlike many works that begin with a visual presentation and then create a sound for the visual, the subject of multimedia electronic music is sound, and the visual presentation is an extension of the sound presentation, a visual representation of the meaning of the sound in the mind of the creator. And this audiovisual relationship applies to A Reflection in the Brook as well, even though it is a fully real-time multimedia production.

Although the sounds and images in A Reflection in the Brook occur ‘simultaneously’ in terms of expression, there is a sequence of first sound first and then an image in the composition. The visual motifs that most embody the conflict and inner struggle – the ‘glowing abstract geometry’ – are based on a computer program that visualise the audio signal in real time (Visualisation Routine), thus achieving a highly synchronisation and connection with the musical content. The images in the ‘Introduction’ and ‘Mirror’ section, although not directly generated from the sound, are based on the visual design of the sound theme and remain consistent with the tone and image of the sound. In the climax, Xiaoqing’s ‘Death’ section, a large number of abstract shapes twist, split, distort and switch in the darkness of the backdrop, reflecting not only the intense inner torment and struggle of the protagonist on an emotional level, but also giving the audience a visual guidance on a perceptual level – a directional and spatial guide to the movement of sound. As Chion explains, ‘Our listening is accompanied, emphasized, and assisted by an image content’ (Chion 2014: 199). Thus, the images in A Reflection in the Brook are still an extension of sound, an image of sound, but in a mature work this ‘sequential’ order of audiovisual elements is intentionally fused or balanced, and the audience’s experience is an artistic whole in which the audiovisual adds value to each other.

4.2. Imitation vs counterpoint

The idea that sound is not an isolated auditory art has been extensively researched, and this is clearly evident in the theory and practice of film sound. In many cases, there is an interaction between the auditory and visual senses and even between the other senses, a phenomenon known as ‘interoception’, which is based on the ‘transmodal perception’ ⁶ of the human being. Thus, people’s understanding of sound imagery can often rely on the experience of vision or even other senses. Conversely, sound can also be acted upon by other perceptions – influencing the perception of form, speed, time, intensity and space. This multidimensional sensory ‘association’ can effectively help to match sound imagery with visual images, making the ‘invisible’ sound ‘tangible’.

Therefore, in the creation of multimedia electronic music, two more common logics of audiovisual integration have developed. One is the association of a figurative, clearly symbolic sound theme with an equally figurative visual theme that directly represents the content conveyed by the sound. Similar to the reverse application of sound to the direct reproduction of visual images in film (e.g., hearing a car – seeing a car.) This is one of the more common techniques used by most Chinese composers in multimedia electronic music composition, where the establishment of visual images relies on previous visual experience of sound, which is mostly derived from cultural and historical factors, and the meaning it conveys is more intuitive. The other is the multiplication of abstract sound and abstract visual images in terms of colour, space, form and texture, correlating the morphological features of the sound with the stylistic elements of the visuals, such as ‘linear’, ‘granular’ and ‘fragmented’. This approach is more focused on the combination of the modalities.

In terms of composition, these two approaches are similar to the ‘mimetic’ approach to music composition, where one medium imitates another in terms of image, or form. In A Reflection in the Brook, however, the combination of audio and visual takes a different approach. Sound and image are like two separate ‘parts’, which, in terms of their horizontal temporal development, are presented as narrative clues, each presenting four thematic sections: ‘Introduction’, ‘Mirror’, ‘Death’ and ‘Epilogue’. In vertical audiovisual relationships, there are points of close integration – although not in a figurative way. For example, at the point of Xiaoqing’s death, the sound is expressed as a fading, ‘dissipating’ multivocal long tone, signifying that Xiaoqing’s soul is leaving her body. The image uses a large eruption of black particles to symbolise the gushing of blood and the release of emotion. The long tones and the particles are not unified or even contradictory in terms of visual image or form, but they constitute a kind of narrative complement here, and together they emphasise a moment, create a rhythm, co-construct time and space for the imagination. If the same analogy of musical composition is used, this approach is similar to the ‘counterpoint’ of polyphonic music. The vocal parts are both closely linked and varied, independent but also forming a vertical correspondence in terms of important beats.

5. CONCLUSION

The interactive multimedia music A Reflection in the Brook is a profoundly thoughtful work. In previous compositions of all kinds about the historical figure of Feng Xiaoqing, the judgement of the value of her life has focused on a universal criterion, including marriage, offspring, social status and wealth. Composer Jin Ping offers a new perspective, that is, from the inner being of Xiaoqing. From this viewpoint, the composer focuses his work on Xiaoqing’s ‘search’ and ‘struggle’, which comes from the ‘confusion’ of her inability to find her self-worth in the social environment of the time, therefore she frequently asks herself ‘who am I’ in the mirror. This process of self-questioning is a reflection of Xiaoqing’s desire to find who she really is, as well as an awakening of ‘feminist consciousness’. The creation of Xiaoqing is quite forward-looking, from the artistic concept to the technical presentation, which in 2013 would have seemed ‘ahead of its time’. It is interesting to note that about five years later, the debate and calls for women’s rights and values in Chinese society have become more heated, which in another way seems to confirm the ‘foresight’ of this work.

The Chinese imagery embodied in electronic music is the emotional expression of Chinese composers based on their own culture. Many years ago, Zhang Xiaofu expressed the idea of ‘using one’s native language to interpret and construct one’s own musical context’ (Zhang 2020). For the Chinese composer, who is in constant aesthetic search in the practice of music composition, the essence of traditional culture is the source of the artistic ‘native language’. While the composition of musical contexts is personal, depending on the composer’s own aesthetic ideas, cultural opinions and expressive preferences. The inspiration for A Reflection in the Brook is related to Chinese culture, but the way in which the Chinese imagery is presented is different from most Chinese productions of its kind. Although no audiovisual elements symbolising the culture of the nation were used, it did not affect the audience’s understanding of its connotation and spirit, but moved many audiences. It is evident that the cultural connotations of music are not presented in a one-dimensional, fixed way and that a multidimensional expression may be more constructive for the audience, provoking deep understanding and reflection on the culture of the people. It is also more conducive to the integration of Chinese electronic music into the aesthetic paradigm of world music culture in a Chinese context.