Feeling disabled: Vowel quality and assistive hearing devices in embodying affect

Abstract

Previous research has proposed that phonetic variation may index affect prior to indexing other social meanings. This study explores whether the affective indexicality of vowels identified in previous studies can also be observed among deaf or hard-of-hearing speakers, in this case, speakers of Taiwan Mandarin. The results suggest that /i/ backing is invoked to signal negative affect. This study also demonstrates how assistive devices like hearing aids and cochlear implants can be considered semiotic resources. For deaf or hard-of-hearing speakers, assistive hearing devices enter into a process of bricolage with linguistic and other symbolic resources, generating new potentials for the embodiment of affect. (Affect, iconicity, Taiwan Mandarin, embodied sociolinguistics, deafness)*

INTRODUCTION

In the recent affective turn in the humanities, affect has been conceptualized as psychological and sensational experiences that are situationally emergent, informed by discourse, and mediated by sociocultural contexts (Wetherell 2015). Affect is not an isolated individual experience but always a relational product of one's interaction with the society. For instance, in Pratt's study of a San Franciscan high school (2019:334), students of technical theatre are perceived to be ‘secretive, sketchy, and deviant’ by other students. They achieve this persona in part through affective displays. Affect is an interactive quality rather than an internal state.

The performative aspect of affect can mobilize the deployment of linguistic resources. Pratt (2019) found that these ‘tech’ students embody affective toughness through a greater constriction of the tongue body for the English lot-vowel (Hall-Lew 2013) and word-initial /l/. Pratt argues that the articulatory setting of lingual constriction may be iconic of bodily containment, rendering it available to vocalize the affect of toughness (Pratt 2018). These findings not only show that certain linguistic features have the semiotic potential to signal affect, but that speakers’ affective experiences are a key aspect of how speakers navigate social-semiotic landscapes (Eckert 2019).

The current study looks at how the affect of (not) feeling disabled is embodied through a semiotic combination of assistive hearing devices and vowel qualities among (oral) deaf or hard-of-hearing (D/HH) speakers of Taiwan Mandarin. In this study, D/HH speakers invoke linguistic resources to embody the affect of a disabled self when they take off assistive hearing devices. In clinical studies on D/HH speech, researchers see linguistic performances with and without assistive hearing devices as purely mechanistic products of presence and absence of auditory feedback (see Methodology). The underlying logic is that D/HH persons do not exhibit agency with respect to the spoken language, but only passively respond to the body as a mechanistic system. In this article we argue that D/HH speakers, like all speakers, use spoken language agentively (also see Wan 2021a).

This study invited D/HH persons to participate in a device-on/off experiment where they were required to read a list of sentences with and without the full use of their assistive devices (‘auditory enhancement’ and ‘auditory deprivation’, respectively). We argue that there are changes in vowel qualities under auditory deprivation which can be explained through the lens of affect and indexicality. The changes in vowel quality under auditory deprivation should be seen as a semiotic resource in a process of bricolage (Hebdige 1979; Eckert 2008) where linguistic resources work together with the physical body to display affect.

This article begins by introducing the competing models of disability that have been applied to the relationship between D/HH persons and hearing technologies. We introduce a study design where a familiar device-on/off experiment from clinical linguistics is re-theorized, specifically with respect to the D/HH communities in Taiwan. The discussion illuminates an epistemological gap for variationist sociolinguists to contribute to the discussion of assistive hearing devices. The research is thus in the broader theoretical context of affect and sociolinguistics, and responds to the intellectual project of embodied sociolinguistics (Bucholtz & Hall 2016).

ASSISTIVE HEARING DEVICES AND DISABILITY STUDIES

In the discourse of medical professionals, disability is considered an individual problem that can be tackled by medical interventions (Ladd 2003). In contrast, the discourse of social activism views disability as a social situation where people with non-normative bodies are oppressed by people with normative bodies (Silvers 2010).

Hearing technologies have been developed to assist D/HH persons in hearing environmental and speech sounds. They can be seen as the medical response to deafness (Virdi 2020). There are two major types of assistive hearing devices—hearing aids (HAs) and cochlear implants (CIs). For D/HH persons whose deafness is evaluated by clinicians as ‘too severe’ to be assisted by HAs, they may be offered CIs. Following the CI surgery is a rehabilitation process which usually includes speech-language therapy (see Snell 2015; Mauldin 2016). Unlike HAs which amplify sounds, CIs replace the ear's own capacity to hear and directly stimulate the auditory nerve. CIs have been discursively framed as a technology that restores hearing and thus ‘cures’ deafness (Mauldin 2016; Lin 2019). Consequently, CIs are also regarded as an assistive technology that can better foster speech development in children with ‘profound deafness’ (or ‘mild hearingness’) (see Bouchard, Ouellet, & Cohen 2009).

In contrast, CIs are considered by some deaf theorists as a politically problematic technology developed by hearing people to colonise deaf bodies (Valente 2011). CIs are a technology that perpetuates the oppression of D/HH communities. Emphasizing how CIs foster speech development, in either popular or academic discourse, is seen an ideological strategy to promote the oralist ideology that spoken language is superior to signed language (Campbell 2009; Valente 2011).

Criticisms of the social activist model of disability argue that it elides the subjective experiences of impairment and presupposes a dichotomy between impairment as biological and disability as social, which ignores potential identity politics, individual resistance, and a dialogic relation between impairment and disability (Meekosha & Shuttleworth 2017). Just as experiences of biological sex do not exist in a social vacuum, neither do experiences of physical impairment (Shakespeare 2006). For instance, Ghosh (2010) describes how disabled women in Bengal experienced the markedness of impaired bodies more and more as they grew up and learned that they were not considered ideal women in the gendered society because of the ‘inability’ to reproduce and engage in household work.

Relatedly, a social model approach to assistive technologies like CIs has failed to engage with the ways in which D/HH persons relate to their assistive devices. The dominant discourses are either that assistive devices hegemonically dominate D/HH people (as suggested by the social model) or altruistically save D/HH people from disability (as suggested by the medical model) by their overwhelming material power. The material effects of assistive devices are taken as physiologically given, and thus can be physiologically determined and measured. Assistive devices are usually considered material objects; some studies have focused on the subjective experience of assistive devices (e.g. Christie & Bloustien 2010; Snell 2015), but few have emphasized their semiotic potential.

We take the perspective that the body is not a mechanistic system, independent from the mind (Toombs 1988; Bucholtz & Hall 2016; Henner & Robinson 2021a), of which linguistic outputs are simply passive products of auditory inputs. Users do not passively respond to the material effect of assistive devices. As Bucholtz & Hall (2016:188) suggest, technology ‘changes not just the way we interact but also our sense of self’. The body is a semiotic assemblage consisting of linguistic resources and material assistive devices and works as a stylistic whole, actively producing images of the self. Material reality can be independent; yet, when human beings engage with material reality, such engagement cannot be objective. The material reality mutates in our perception when interacting with us (Pennycook 2018).

In this article, we suggest that linguistic variation correlated with the presence or absence of assistive technologies may reflect speech adjustments made in response to mechanistic changes, but might equally reflect stylisations made in response to different experiences of the body.

THEORETICAL CONTEXT

Recent disability studies have pointed out that disabled people do not feel disabled because of non-normative bodies per se. The non-normative body is socially constructed as disabled and therefore undesirable (Ghosh 2010; Goodley, Liddiard, & Runswick-Cole 2018). In this article, we demonstrate how D/HH persons experience auditory deprivation as a transformation of the body. This then is the embodiment of disability, the affect of disability. Phonetic variation associated with auditory deprivation indexes the affect, not the deprivation.

Affect

In this study, we focus on the affect displayed by the participants when they lose full access to auditory enhancement.

When people display affect toward something, they are positioning themselves within semiotic landscapes as the persons who typically display that kind of affect toward that particular thing (Du Bois & Kärkkäinen 2012). In this research, when D/HH participants display negative affect toward the auditorily deprived body and its associated self-image, for a hearing audience (including the researchers), such affective display further indexes a particular discourse of disability in which disabled people demonstrate negative attitudes toward the disabled body (Lane 1988). In contrast, if participants appear to be neutral about disability, or even perform ‘overcoming’ the impact of disability on their life, their neutrality may be construed by a hearing audience as aligning with a discourse of disability in which disabled people are inspirational (Chrisman 2011). Either way, affective displays are constrained by these two opposing narratives of auditory disability.

In the current research, the D/HH participants take part in a device-on/off reading experiment in the presence of a hearing researcher, and comparisons are drawn between the read speech styles in each condition. Although sociophonetic work on affect has typically analysed spontaneous speech (e.g. Eckert 2010; Pratt 2019), read speech is equally available for meaning making (e.g. Silverstein 2003; Gafter 2016; Hall-Lew, Cardoso, & Davies 2021; Wan 2021a,b). The ‘self-conscious’ nature of read speech enables the researcher to see how participants actively embody affect through linguistic resources (see Schilling-Estes 1998). Embodiment can be broadly understood as the process in which people materialize (here, vocalize) a non-material aspect of the self (here, affect). We argue that phonetic differences between the ‘device-on’ and ‘device-off’ read speech contexts are mobilized by changes in affect (or lack thereof). Specifically, linguistic practices align with the affective persona that the speaker associates with the auditorily deprived body.

Vowel and affective qualities

Iconicity represents a similarity between a signifier and the signified that is considered ‘natural’. Affective iconicity has been documented for various linguistic features, both segmental (e.g. Eckert 2010; Calder 2019) and suprasegmental (e.g. Starr, Wang, & Go 2020; Pratt 2021; Esposito & Gratton 2022). The current study focuses on vowel quality. For instance, compared to a back vowel, a front vowel may be iconic of smallness. Some have argued that the iconicity of a linguistic feature may be an evolutionary legacy; for instance, large threatening animals have a lower-pitched sound, so lower sound frequencies (e.g. a back vowel) are iconic of largeness, and thereby powerfulness (Ohala 1983). Sociolinguists tend to see the iconicity of a linguistic feature as a product of ideologization (Eckert 2019; D'Onofrio & Eckert 2021). Recent sociolinguistic work has proposed that iconicity may stem from how we use the body (Podesva 2016; Pratt 2019). That is, certain linguistic features can embody affective qualities because the linguistic features semiotically acquired the affective indexicalities by being contingencies of the articulatory settings expressing those affective qualities. This theory has been evidenced by several empirical studies: vowels front when the facial expression of smile is made by US English speakers (Podesva 2016) and Mandarin speakers in China (Tang, Xu Rattanasone, Yuen, & Demuth 2017); vowels back and raise when Cantonese speakers show the facial expression of disgust (Chong, Kim, & Davis 2018).

A growing body of sociophonetic work has demonstrated the relationship between vowel variation and affective qualities in English-speaking communities, especially in the United States: generally, vowels back and raise when a negative affect is expressed, and vowels front and lower when a positive affect is expressed (Eckert 2010; Wong 2014; Podesva 2016; Pratt 2019). In experimental linguistics work on Mandarin (Erickson, Zhu, Kawahara, & Suemitsu 2016), it was found that variation in vowel anteriority is more strongly associated with performed emotions across speakers: vowel fronting for happy speech, and vowel backing for sad speech. Notably, variation in vowel quality only provides semiotic potential for these affective qualities (Bucholtz & Hall 2016); local sociocultural contexts can override such potential. This might be especially likely in contexts where the variants index local social meanings that take precedence over any potential affective qualities (Hall-Lew et al. 2021).

METHODOLOGY: DEVICE-ON/OFF EXPERIMENT

Laboratory informed

In the paradigm of the ‘device-on/off’ experiment developed by audiologists and speech-language pathologists, speakers are asked to read material with and without auditory feedback, by turning on and off their assistive devices. The ‘device-on/off’ experiment is adopted to see how auditory feedback modifies speech production (Perkell, Guenther, Lane, Matthies, Perrier, Vick, Wilhelms-Tricarico, & Zandipour 2000). When D/HH speakers are deprived of auditory feedback, they can instead rely on existing kinaesthetic memory to produce speech (Perkell et al. 2000). Researchers have found both hyperarticulation and hypoarticulation with respect to vowel production under auditory deprivation (Poissant, Peters, & Robb 2006).

Most clinical research attributes variation in assistive hearing devices to patterns of articulatory variation, rather than considering non-physiological factors. For example, hypoarticulation during auditory deprivation may be argued to stem from the disruption of the speaker's ‘sense of appropriate tongue placement’ (Higgins, McCleary, & Schulte 2001:38); however, a lack of changes in speech production during auditory deprivation is also considered due to the ‘extensive use of the CI’, which allowed the speakers to build robust kinaesthetic memory (Turgeon, Trudeau-Fisette, Fitzpatrick, & Ménard 2017:94). These analyses betray a contradictory ideology: in each case, the assistive device is framed as beneficial, and D/HH people as owners of the assistive device play no role in the demonstrated stylistic practices during auditory deprivation.

Ethnography informed

It is important to study assistive hearing devices ethnographically and historically, so we can understand how the material objects become ideologically linked with oralism (Holmström, Bagga-Gupta, & Jonsson 2015; Virdi 2020; Loh 2022). The current study argues that the ‘device-on/off’ experiment paradigm, in relation to language production, must also be understood as situated in a particular sociocultural context. The first author is Taiwanese and became familiar with the D/HH community, mostly oral D/HH people, through personal networks as well as previous fieldwork experiences, beginning around 2014. The oral D/HH population considered in this article speak Mandarin as their dominant language. Most of them identify as tīngzhàngzhě ‘people with hearing impairment’ or tīngsŭnzhě ‘people with hearing loss’ (the latter is becoming more widely used than the former), rather than lóngrén ‘deaf people (who sign)’. Considering how ‘hearing impairment/loss’ invokes an audist ideology and is dispreferred in English academic writing (Pudans-Smith, Cue, Wolsey, & Clark 2019), here we used the term deaf or hard-of-hearing (D/HH) to encompass all of the Mandarin identity terms, used by our participants, as the term ‘hard-of-hearing’ also highlights the comparison between a non-normative hearing status and the normative one.

Due to a pressure to perform hearingness under audism (Henner & Robinson 2021b), only a small portion of D/HH children attend a deaf school, where Taiwan Sign Language is the medium of instruction. Since the 1980s, ‘inclusive education’ has been implemented in Taiwan. Most D/HH students are enrolled in mainstream schools instead of deaf schools. In 2016, the only deaf school (including primary and secondary education) in the Taipei Metropolitan Area had 141 students, less than 15% of the total of number of D/HH students in the area (N = 1,014; Executive Yuan 2016). In addition, as Taipei is the largest metropolitan area in Taiwan, D/HH people in Taipei have greater access to medical resources including speech-language therapy and assistive technologies like cochlear implantation. Thus, they are more likely to be orally educated than those in other parts of Taiwan.

The first author's earlier work with middle-aged and elder D/HH persons in the Taipei community (Wan 2021b) included the case study of a cochlear implant (CI) user and advocate, Xiao-Kun,¹ a volunteer of a CI support group at a local hospital. Compared to her positive attitudes towards her CI, Xiao-Kun explicitly described hearing aids (HAs) as “shameful”, and she did not like wearing them before receiving the CI. Xiao-Kun presented two distinct images of the self with and without turning on her CI. When using the CI, Xiao-Kun appeared very confident and continually claimed the floor in the conversation where her two friends, who were HA-only users, were present. When she relied only on her HA, she became a drastically less confident person who refused to interact with her interlocutors. Xiao-Kun's two distinct presentations of self suggest that D/HH persons experiencing an auditory deprivation may experience a robust contrast between two embodied experiences. Such a contrast is not simply about the mechanistic consequences of the presence or absence of auditory feedback; it is also socially construed.

Without prompting from the researcher, Xiao-Kun spontaneously performed speech differences between her use of only-HA and her use of the CI, by reading aloud the consent form twice. She shifted to a higher pitch departure of the high-falling lexical tone after turning on the CI, indicating a more ‘standard’ pronunciation. She pointed out this difference as evidence to prove how useful the CI is, compared to the HA (Wan 2021b).

Speaking different linguistic varieties ‘produces different bodies’ (Bucholtz & Hall 2016), and this is not just a metaphor. The way we categorize a body in social space is based on both visual and aural information. Speaking ‘with a deaf accent’, for example, may be interpreted by hearing people as ‘speaking from a disabled body’. When it is believed that the use of a hearing assistive device results in a hearing-like accent, the link between a medical device and a linguistic variety is naturalized. Thus, medical devices become iconic of abled-bodiedness, when in actual fact, the speech variety used with an assistive device is one which the user is only eventually able to produce through the process of speech training that accompanies rehabilitation.

The contrast between auditory modes also indexes the contrast between life experiences associated with those modes. Different auditory modes accrue different affective experiences (Christie & Bloustien 2010) through what the user experiences under each auditory mode. In the example above, Xiao-Kun displays negative affect toward disability and hearing aids. She used to refuse to put on hearing aids, because she thought they were shameful and not useful, while she sees the useful CI as part of an abled life she finds satisfying. She embodied a negatively disabled life by a process of bricolage: the material existence of HA, the conversational strategy of preventing interaction, and the lower onset pitch of the high-falling tone. Xiao-Kun's variable production of the falling tone indexes those two bodies and their associated affective qualities. This observation motivated the current study, where we argue that linguistic variation should be considered part of a semiotic assemblage within which D/HH persons make sense of themselves in the broader social context of ableism.

STUDY DESIGN

Linguistic variables and hypotheses

This study examines the positions of the three Taiwan Mandarin corner vowels—/a/, /i/, and /u/. These vowels are consistently examined in clinical assessments of the relationship between assistive hearing devices and speech variability. Yet, compared to centralized vowels, theoretically they are less susceptible to ‘speech degradation’ during auditory deprivation (Perkell et al. 2000:241), thereby leaving much space for a sociolinguistic exploitation.

Mandarin and its vowels have undergone indigenisation in Taiwan since the 1940s, driven by contact with various local languages. Compared to the Standard Mandarin used in China, Taiwan Mandarin shows a more raised /a/, backed /i/, and fronted /u/ (Chang 1999; Sanders & Uehara 2007), resulting in a more compressed vowel space. This regional variation is not subject to metalinguistic commentary, but the variants indexing speakers from China are anecdotally perceived by Taiwanese listeners to be more standard and clearly pronounced. Building on the cross-linguistic and Mandarin-specific studies of vowel quality and affect discussed above, we may expect to see vowel fronting/lowering indexing positive affect, and vowel backing/raising indexing negative affect. We therefore posit that the indexical field (Eckert 2008) of fronted /u/ in Taiwan Mandarin consists of non-standardness and positive affect, whereas the indexical fields of backed /i/ and raised /a/ consist of non-standardness and negative affect.

Recent linguistics work (Hung, Lee, & Tsai 2017) reports that the inter-speaker variation among D/HH people in Taiwan Mandarin shows /i/ varying acoustically along the dimension of anteriority, /a/ varying along the dimension of verticality, and /u/ varying the least. It is found that /i/ is the most acoustically variable among the three corner vowels across both D/HH speakers and hearing speakers (Figure 1). This work also suggested that variability of /i/ does not seem to be a physiological product of deafness; instead, they proposed it may be due to the difference in signal conduction methods, in that people who rely on bone rather than air to conduct sounds may hear ‘distorted’ /i/ sounds that are located at lower frequencies (Hung et al. 2017). Considering the style axiom, intra-speaker variation usually derives from and echoes the variation which exists between speakers (Bell 1984). The anteriority of /i/ is theoretically the variable that has the most potential for signalling affective qualities that are social.

Figure 1. Interspeaker variability for the three corner vowels; /i/ is the most variable vowel among D/HH speakers, and D/HH speakers have a much more variable production of /i/ than hearing speakers (plotted based on the data from Hung et al. 2017:8).

This article foregrounds the subjectivity of D/HH speakers, in addition to the material effects of assistive devices, to illustrate that our understanding about what happens to linguistic production under auditory deprivation can benefit from a sociolinguistic perspective. Based on the above literature, we propose three hypotheses.

• Hypothesis 1: During the auditory deprivation, D/HH speakers may realise the linguistic variable differently, compared to their realisations with auditory feedback.

• Hypothesis 2: For any linguistic variable that is realised differently, the interspeaker variability can be accounted for by the affective display during the auditory deprivation.

• Hypothesis 3: For any interspeaker variability that is statistically predicted by the affective display, vowel backing is correlated with negative affect, and vowel fronting is correlated with positive affect.

The results support all three hypotheses with respect to the most variable of the three corner vowels, /i/.

Methods

The current research adopts the experimental design of short-term auditory deprivation, where the participants first read aloud the sentence list in the ‘device-on’ condition and then read the sentences again immediately after entering the ‘device-off’ condition. Adopting this method allows the speakers to experience the contrast between a body with technological assistance and a body without it, and in a way that they are familiar with from their experience in clinical settings. Participants knew from the recruitment text that they would be asked to read material with and without access to full assistive devices.

‘Full assistive devices’ refers to the default auditory mode with which participants rely on in daily life. Here, this default auditory mode is referred to as ‘device-on’, and the auditory mode used while in the auditory deprivation condition is referred to as ‘device-off’. For many CI-using participants, the ‘device-off’ condition includes a secondary audio device that is still on, such as a hearing aid.

Satisfaction survey

To transform the device-on/off experiment into a site where individual agency can be investigated, we did a satisfaction survey (Likert scale with six points) on the participants’ assistive devices prior to completing the reading task, to encourage the participants to reflexively consider their relationship with the assistive device (see Appendix A). The survey is adapted from Cox & Alexander's (1999) version and consists of four major categories of statements, presented in random order: (i) whether the device improves the user's speaking (statements 6, 13; hereafter as ‘speaking satisfaction’); (ii) whether the device improves the user's understanding of others’ speech (statements 1, 5; hereafter as ‘listening satisfaction’); (iii) whether the user finds the device carries negative social meanings (statements 8, 12, 14; hereafter as ‘social satisfaction’); and (iv) whether the user finds the device is effective in general (statements 2, 3, 7, 10, 11; hereafter as ‘effectiveness satisfaction’). Participants’ responses summed for each of the four satisfaction indicators and operationalized as continuous variables in statistical modelling.

Reading materials

Some of the previous short-term auditory deprivation research tested isolated vowels (Turgeon et al. 2017); others used carrier sentences (e.g., “Say ___ please” or “It's a ___ please”) in which different vowels were repeated ten to fifteen times (Svirsky, Lane, Perkell, & Wozniak 1992; Lane, Matthies, Guenther, Denny, Perkell, Stockmann, Tiede, Vick, & Zandipour 2007). The current study does not adopt carrier sentences. Target words in carrier sentences receive prosodic emphasis because they are under focus and located in the same word position; thus, we take focus and word position into consideration, instead of adopting carrier sentences. Instead, this study uses more naturalistic sentences, taking focus and word position into consideration in their design, and analysing other words in the sentences besides the focused word.

The reading task consisted of a list of fourteen unrelated sentences written in traditional Chinese orthography containing the three monophthongs /i/, /a/, and /u/ in focused and unfocused words. /i/ and /u/ only appear in open syllables; /a/ is in either open syllables or closed syllables with the coda /n/ (see Table 1).² The sentence-initial words are the topics of the sentences, and they are put in quotation marks to encourage focus (hereafter, focused words). With this design, word position and focus are the same thing. All of the sentences are declarative statements (see Appendix B). For example, 「地球」是太陽系中的第三顆行星 ‘“The earth” is the third planet in the solar system’. The use of sentences with controlled prosody helps avoid any effects of listing prosody that might arise with isolated word lists.

Table 1. Number of syllables with target segmental features (content words, both focused and non-focused) in the read sentences.

Before the recording, participants were asked to practice reading the sentence list to make sure there was no Chinese character that they did not know how to pronounce. While reading practice may encourage a less-spontaneous speech style, it gives the speaker the opportunity to agentively produce an intended read speech style, and that is the target of this study. After the reading task, the participants engaged in a metalinguistic interview. They were invited to share what differences they felt between the two auditory modes, and then were asked more generally about how deafness impacts their life.

Statements made during the metalinguistic interview indicate two different ways in which participants orient to deafness and embodied disability. We first describe that interspeaker difference and then demonstrate how it informs a model of variation in vowel production.

SPEAKER GROUPS IN THIS STUDY

In this section, we describe how affective displays during the ‘device off’ condition indicate experiences of living as disabled persons. We find that participants broadly align with one of two affective displays: ‘negative’ or ‘neutral’.

A total of nineteen participants were recruited through social media and word of mouth between January and August 2020 (see Table 2). The average age of the participants at the time of recording was 25.2 (max = 33, min = 18). There were ten women, and nine men. They were all residents of Taipei and varied in their type of assistive hearing device: CI for CI users, HA for non-CI users. The interviewer (the first author) is a hearing person (twenty-seven years old, man) from Taipei. The interviewer and the participants all identify as Han Taiwanese. Interviews were conducted one-on-one in a quiet room in a public space. Recordings were made on a Zoom H5 (primary; at a 44.1 kHz sampling frequency) and the interviewer's Google Pixel phone (backup), using built-in microphones. Although data recording occurred during the Covid-19 pandemic, rates in Taipei were very low at that time, and none of the participants nor the interviewer was wearing a facemask or face shield at the time of data collection.

Table 2. Participants and information on their default assistive device, their self-described gender, and their affective display.

Analysis of affective display was done by examining the speakers’ responses to the question, ‘did you feel any physiological or psychological difference between the two auditory modes during the reading tasks?’. If the participants (N = 9) mention anything psychologically negative, they are categorized as displaying negative affect (hereafter, ‘negative’ speakers); otherwise, they are categorized as displaying neutral affect (N = 10) (hereafter, ‘neutral’ speakers). None of the nineteen participants mentioned positive psychological feelings. Note that this coding does not imply that the ‘neutral’ participants experienced nothing psychologically negative internally. Rather, this categorization emphasizes the performative aspect of affect, that is, the fact that the participants did not display negative affect in front of the hearing researcher.

To explore whether the affective display is statistically dependent on the default assistive devices the participants rely on (CI vs. non-CI; Table 2), we applied a chi-square test of independence to the data. The result shows that there is no significant association between affective display and the participant's default assistive device, (χ² [1, N = 19] = 3.315, p = 0.068). We also checked whether the affective display directly reflects participants’ levels of satisfaction with their default device and found no correlation for any of the four satisfaction indicators.

Based on the case of Xiao-Kun described earlier, we know that D/HH people's responses toward different assistive devices are not only about the devices in a medical sense. When using a particular device, they also think of the life experiences associated with that device, and these inform ideologies towards both the device and disability. Here, we use the participants’ personal narratives of how deafness influences daily life to make sense of the displayed affect in relation to their life experiences. We demonstrate how their affective displays are performative, as are their meta-linguistic narratives of disability.

Speakers signalling neutral affect

‘Neutral’ participants are those who did not mention anything psychologically negative when being asked to comment on the difference between the two auditory modes. Note that demonstrating affective neutrality is an affective display, too (cf. Pratt 2019). It does not mean there is no affect involved; rather, given the prevalent discourses of stigma toward disability, the avoidance of negative affect is potentially highly agentive. While they might internally experience a negative self-image indexed by auditory deprivation, these ‘neutral’ participants did not display any such affect to the researcher.

The personal narratives of ‘neutral’ participants did not highlight deafness as impacting their life and instead argued that disability is only one axis of identity. They also tend to explicitly reject negative ideological framings of disability. For example, Eda said:

I was born deaf, so I just deal with it peacefully. Actually, I don't think it's anything shameful. … I'm lucky because I grew up in an environment that doesn't make me feel pessimistic. Yes, my friends and others … they don't treat me as special just because I'm in this situation.

我天生就是這樣，那我就是和平的跟它共處這樣子。因為其實我真的覺得沒有什麼好丟臉的啦。… 我算運氣好因為我生長的環境並沒有讓我覺得很悲觀，對，因為我身邊的朋友什麼的啊 … 他們也不會覺得說，你是這樣的狀況，然後就特別對你怎麼樣 (emphasis added)

Eda clearly constructed herself as a person who does not feel negative about deafness. Another example is Anxin.

I'm emotionally stable … I don't easily have too many negative emotions … I know that some D/HH children, under this inclusive education paradigm, the schools do not always have the resources they need, so they have poor academic performances … it is always the case that they are excluded by their peers … or being bullied (4 secs) but I didn't have this kind of experience. … I read some papers saying that, in Taiwan, if your academic performance is great, no one would mess with you.

我脾氣滿平穩的 … 我滿不容易有太多負面情緒 … 我是有大概有知道就是一些聽障小朋友在這種回歸主流教育的狀況，學校其實不一定真的可以給他需要的資源，然後他成績會落後 … 被排擠是一定會有 … 或被霸凌 (4 秒) 但我是沒有碰到這樣的狀況 … 我好像有看到一些論文是寫說，台灣的狀況是如果你成績夠好，基本上沒有人會去鬧你 (emphasis added)

In this quote, Anxin first rejects the identity of being a person “having too many negative emotions”, itself an explicit display of neutral affect. He then contrasted his experiences with those of the other D/HH persons. He also highlighted that in the Taiwanese context, disability is secondary to academic experience, which backgrounds the role of disability in his life.

Some ‘neutral’ participants stated that they experienced an inability to monitor their pronunciation under auditory deprivation, but that they did not perceive their unmonitored speech negatively. For instance, Annie said that she was not worried about her speech under auditory deprivation, since for her, “that's also one of my voices”. Others even stated that they did not feel any difference in auditory feedback between the two auditory modes. Rain is an example. He also mentioned that he was seldom treated differently by hearing people, and that disability never bothered him.

Neutral affect was displayed by these participants to explicitly counter the negative affective qualities (e.g. not emotional, not pessimistic) present in the wider social discourse. That is, they demonstrate the affect of ‘not feeling disabled’. Such affective neutrality is what Bucholtz (1999:211) describes as negative identity practice that individuals adopt to ‘distance themselves from a rejected identity’. Here, what is rejected by the ‘neutral’ participants is a negative stereotype of D/HH people (Lane 1988). In other words, the ‘neutral’ participants reject a negative ontology of disability.

Speakers signalling negative affect

‘Negative’ participants were those who, in presence of the researcher, displayed negative affect toward the self-image indexed by the auditory deprivation. Such negative affect is verbally anchored as feeling “insecure”, “unsteady”, “empty”, “unconfident”, “anxious”, “powerless”, and “frustrated”. For example, Wei said that she felt “insecure” under auditory deprivation because she had been criticized by hearing people for being too noisy when not using her CI.

By examining their narratives, we found that ‘negative’ participants orient to a self-image that is composed of negative experiences concerning disability. They tended to emphasize how deafness negatively affected their life, in general. Some pointed out that they used to be at a low point of their life and gradually became more positive due to particular life events.

For some, disability heightened anxiety around academic performance. Grace expressed anxiety over whether her academic performance was actually on par with other students at her university or if she had been admitted due to lower entrance requirements for students with disabilities. Grace also cared about how hearing people view her.

I often think about why I'm trying so hard to make myself looking like a normal person. … OK, I'm conflicted. On one hand, when I hear people saying I don't look like D/HH, or I speak clearly … I feel happy. On the other hand, I'm angry. Why should I adapt to you and talk like you.

我自己還是會覺得我為什麼要那麼努力讓自己像個正常人 … 好吧，這也是很矛盾，一方面當我聽到別人稱讚我說，就我很不像聽障，或者是我講話很清楚 … 我覺得很開心，一方面我就很憤世嫉俗的會覺得說，我為什麼要適應你們 … 然後講話要像你們一樣 (emphasis added)

Grace pointed out that she felt “unsteady” and “empty” during the auditory deprivation. In this case, her expressions of negative affect against her auditorily deprived body were in response more to the image of a negatively disabled self, indexed by auditory deprivation, than to an immediate lack of auditory feedback.

This pattern was especially noticeable among participants who divided their life into a negative part and a positive part, with the dividing line of their life as their receipt of the CI. This narrative of ‘a new life’ is frequently reported in previous studies on CI users’ experiences (e.g. Finlay & Molano-Fisher 2008). While some HA users might also convey negative affect when turning off their HAs, the narrative of assistive technology being a dividing line did not occur in these data, likely because individuals usually received their first HAs at a very young age. In contrast, many CI users did not receive cochlear implantation until adulthood. For example, in Chenyu's first year during undergraduate study he had an HAs but had not yet received his CI, and he describes having low self-esteem and a timid personality. He considered the later CI implantation as a chance to start a new stage of his life and to adjust his personality.

The cochlear implantation was more like a turning point for me to change my mindset. It's like when I used to use HA, I was less courageous. But after I got my CI, I thought I couldn't give up or waste this opportunity to change myself … back to the CI, objectively speaking, it doesn't really improve my hearing much.

開電子耳代表在心態上有一個時間點去轉變自己的心態。就是以前可能裝助聽器的時候，我本身個性是比較退縮的。不過開完電子耳後，就會想說，不能放棄或是浪費改變自己的機會 … 不過回到電子耳本身，它在客觀上可能並不是增加了很多的聽力這樣子 (emphasis added)

Notably, Chenyu stated that the auditory improvement from receiving CI was actually very limited. Yet, with CI, he said he managed to become a “braver” and “more positive” person who treasured each opportunity given in his career. Describing how he felt during the auditory deprivation, he said:

When I turn off my CI, or sometimes it's out of battery, and I don't have the battery with me, I only have HA. … I kind of psychologically feel disabled. … Probably I'm afraid that people think I cannot hear.

平常關掉電子耳或是沒電，但是一時還沒有辦法換電池的時候，我就只有助聽器 … 這個時候心理上就覺得有一種失能的感覺吧 … 就可能這時候就很害怕別人覺得自己聽不到什麼吧 (emphasis added)

The description corresponds to how he adjusted his personality after receiving his CI. Chenyu's case crystallises how affective negativity is performed toward an auditorily deprived body in this group and is consistent with their emphasis on the negative aspects of deafness in their personal narratives. That is, a momentary affect during auditory deprivation reflects a more general habitus (Bourdieu 1991); such affect is not individualized (life experiences are highly heterogenous), but rather an embodied ideology resulting from the everyday interaction between D/HH individuals and the ableist society.

ACOUSTIC ANALYSIS

Previous research on vowel production between the ‘device-on’ and ‘device-off’ conditions comes from a clinical perspective. Here, we follow classic practice in both clinical work and sociolinguistics, operationalizing variation in vowel quality based on single-point vowel measures.

Analytical technique

All occurrences of /i/, /a/, and /u/ in both focused and unfocused words (see Table 1) were manually segmented in Praat (Boersma & Weenink 2019). Only content words were included. F1 and F2 values were automatically extracted from the midpoints of labelled intervals. Formant values of vowels were Nearey-normalised using the vowels package (Thomas & Kendall 2017) in R (R Core Team 2019). Vowel plots were made through the phonR package (McCloy 2016), and the data were analysed with linear mixed-effects modelling using the lme4 package (Bates, Mächler, Bolker, & Walker 2015). All models included by-speaker and by-word intercepts as random effects. By-speaker random slopes were included where possible.

Each maximal model included the following independent variables: affect (negative/neutral), auditory mode (AM) (device-on/device-off), default assistive device (CI/no-CI), gender (men/women), hometown (i.e. whether one was raised in Greater Taipei) (yes/no), the four satisfaction indicators (continuous, scaled, and mean-centred), duration (continuous and log-transformed; see e.g. Kondaurova, Bergeson, & Dilley 2012), nasal coda (yes/no), word focus (focused/unfocused), and lexical tone height (low [tone 2 and 3]/high [tone 1 and 4]). Independent continuous variables are mean-centred. Model comparison proceeded as a forward, ‘add-one’ process. Interaction terms were theoretically driven, including two-way interactions between AM and every social factor (i.e. affect, device, gender, hometown, and satisfaction indicators). We also considered a three-way interaction among AM, gender, and affect. An interaction term was only included if it improved the model fit, and the optimal model was chosen when no further independent factor improved the model fit.

Results

Figure 2 shows the vowel plots by affective displays. Based on the qualitative vowel visualizations, during auditory deprivation, the vowel /i/ appears to be backed by both groups, but ‘negative’ speakers show greater /i/-backing. There also seems a raising process in the vowel /a/ between auditory modes. Neither group shows much variation in the vowel /u/ between conditions. Statistical analysis later supports these observations. Based on the model results, no fixed effect of auditory mode or its interaction with affect was significant for predicting /a/-anteriority, /i/-height, or either dimension of /u/, and these dimensions of variation will not be discussed further.

Figure 2. Vowel plot by affective display under ‘device-on’ and ‘device-off’ conditions.

/a/ height

The best-fitting model for the F1 of /a/ is as follows (see Table 3):

F1 ~ Duration + Nasal coda + AM * Gender + (1 | Word) + (1 + Nasal coda + AM | Speaker)

Table 3. Results of a linear mixed-effects regression modelling the height of /a/.

Vowel duration is a main effect, but contrary to phonetic predictions (Toivonen, Blumenfeld, Gormley, Hoiting, Logan, Ramlakhan, & Stone 2015), longer durations correlate with a higher /a/ vowel. More in line with phonetic expectations is a main effect of coda type, with a nasal coda raising /a/ relative to a zero coda. Neither device nor affect was retained in the final model; that is, no evidence shows that CI users and non-CI users, nor ‘neutral’ and ‘negative’ participants, produce /a/ height differently. Auditory mode and gender are both significant main effects, and so is their interaction. Women produce /a/ higher under the ‘device-off’ condition, while men's /a/ barely shifts.

/i/ anteriority

The best-fitting model for the F2 of /i/ is as follows (see Table 4):

F2 ~ Word focus + AM * Affect + AM * Gender + AM * Speaking satisfaction + AM * Listening satisfaction + (1 | Word) + (1 + Word focus + AM | Speaker)

Table 4. Results of a linear mixed-effects regression modelling the anteriority of /i/.

As shown in Table 4, word focus is a significant predictor for /i/, such that non-focused syllables are more centralized than focused syllables (Gu, Mori, & Kasuya 2003).

The best-fitting model finds a significant AM-by-affect interaction, and a significant AM-by-gender interaction, but no significant three-way interaction. While speakers across the dataset generally produce relatively backed /i/ during auditory deprivation, this effect is significantly reduced among ‘neutral’ speakers, on the one hand, and men, on the other. Note that affect and gender do not co-vary (χ²[1, N = 19] = 0.46, p = 0.498). These results (Figure 3), show that the only group not showing /i/ backing in the ‘device-off’ mode are the ‘neutral’ men. However, the model encourages us to consider an indexical analysis that keeps gender and affect separate at some level.

Figure 3. Vowel positions by gender and affect category.

Two indicators from the satisfaction survey have significant interaction with the effect of AM. ‘Speaking satisfaction’ and ‘listening satisfaction’ among the participants are not correlated here (r(17) = .34, p = .24). This is likely because a speaker may have a relatively stable control of their own speech regardless of auditory input, due to various factors (e.g. age of early intervention). In this case, they may not experience a noticeable shift in their pronunciation during auditory deprivation, even if they lose access to auditory feedback.

The first of these two interactions finds that if a participant self-reports speaking ‘better’ (more ‘intelligible’ to hearing people and more ‘standard’) under auditory enhancement, they tend to back /i/ more during auditory deprivation. Participants whose ‘speaking satisfaction’ with the default device is high seem to be aware of shifting their speech during deprivation. Note that this effect is orthogonal to any effect of affect; ‘negative’ speakers do not self-report speaking satisfaction with the default device higher than ‘neutral’ speakers do. When a person finds themselves speaking in a less standard way without auditory enhancement, they do not necessarily find it a negative thing. For example, as mentioned earlier, Annie explicitly mentioned that she was aware of her lack of control of own pronunciation when the device was off, but she completely accepted it as just another one of her voices.

The second ‘satisfaction’-based interaction suggests that the more a speaker reports they auditorily benefit from a device, the less they back /i/ in the device-off condition. This is consistent with the clinical perspective that when users benefit from auditory enhancement, they may develop a strong motor control in their speech, leading to a relatively stable speech during short-term auditory deprivation (Turgeon et al. 2017).

The two significant interactions of satisfaction indicators represent the mechanical aspect of how the machine can have an impact on one's speech production, based on the user's own experience. In addition to the mechanical effect, we still see the significant effect of affective display.

DISCUSSION

The results of the acoustic analysis are summarised in Table 5. Among the three corner vowels, only the data from /u/ does not support Hypothesis 1: that speakers realise the linguistic variable differently between auditory modes. The height of /a/ and the anteriority of /i/ support Hypothesis 1. Only the variation in /i/ supports Hypothesis 2: that the variation can be explained by affective display.

Table 5. Summary of findings.

The significant effect of negative affect on /i/-backing is consistent with previous work done in English-speaking communities that vowel backing indexes negative affect (e.g. Eckert 2010). This group-level effect is also consistent with previous experimental work on emotional vowels in Mandarin, for example, that vowel backing is observed across speakers for sad speech (Erickson et al. 2016). The current affect effect is not gendered, nor is it influenced by default assistive devices. As we have also controlled for satisfaction indicators which represent how the participants find their default assistive devices useful in terms of ‘improving’ speech production and perception, the mechanical impact on the position of /i/ has also been considered. That is, we can say the effect of negative affect on /i/-backing is a robust effect.

For ‘negative’ speakers, the removal of the technological assistance indexes a self-image in which they experienced frustration with deafness. They embody their negative affect toward such an image of the self by drawing on various resources, including /i/-backing. Here, we see how physical body and phonetic variation work together as a stylizing unit. In contrast, ‘neutral’ speakers did not invoke social practices that treat a body without technological assistance as a different body from their auditorily enhanced body. The change in physiology is performed as having no influence on the psychology.

While we do not propose a mechanism by which vowel qualities are associated with affective qualities, the results of this research may provide some insights. The D/HH participants in this study have limited access to variation in sound, especially in the range of sound frequencies occupied by the F2 of /i/ (Liu & Kewley-Port 2007). Thus, it is less likely that they perceive iconicity in vowel backing via the acoustic signal, as hypothesized by Ohala (1983). The results of the present study may instead support the theory that affective qualities are linked to vowel qualities somatically because of the way we use the body.

The other major finding of this article is the gender effect on both /a/-raising and /i/-backing. Although gender is not our primary focus, its significance is further evidence that changes in speech production due to auditory deprivation are not purely mechanistic. We spend the rest of this section accounting for the observed gender effect in these data.

Recall that Taiwan Mandarin undergoes /a/-raising, /i/-backing, and /u/-fronting relative to Standard Mandarin in China. As mentioned, /u/ is relatively stable across speakers, so it is not surprising that it is not exploited in intra-speaker variation. For /a/, we not only observed a gender effect but also saw a lack of a significant interaction between auditory mode and any satisfaction indicator. That is, the /a/-raising under auditory deprivation may have little to do with a direct, mechanical result of a change in auditory feedback. There has been no research on the social indexicality of /a/ on its own in Taiwan Mandarin, and the /a/-raising distinguishing Taiwan Mandarin from Standard Mandarin was not reported to be gendered (Sanders & Uehara 2007). We therefore suggest looking at it together with /i/-backing as congruent parts of stylistic practice.

Gender contributes to /i/-backing roughly to the same extent as affect does. Speaker gender contributes to /a/-raising, but affect does not. The two variants also index a non-standard style, and sounding standard is an important component of sounding hearing-like or abled-bodied. The two variants work together to index disabled-bodiedness, regardless of whether one considers a disabled body negative or not.

The gender effect on both /a/-raising and /i/-backing adds to the stylistic variation within auditory deprivation. Compared to disabled men, disabled women may be more sensitive to the shift in embodied experiences of being disabled because of the intersectionality of disability and gender oppression (e.g. Ghosh 2010; Kim 2016). Disabled women are socially pressured more to perform abled-bodiedness (here, performative hearingness; Henner & Robinson 2021b). However, the effect of gender is not found to interact with the effect of affect. There is no evidence to argue that D/HH women invoke /i/-backing to embody negative affect more than D/HH men do. And disabled women being more sensitive to the shift in embodied disabled experiences does not mean they more likely consider disability negative than disabled men. A disabled body can receive a neutral or positive ontology for some of the women participants: for example, Ling mentioned how she was often mistaken as a Southeast Asian migrant worker (a stigmatized social status in Taiwan) because of her deaf accent, but she did not find it an unpleasant experience—instead, she disclosed her deafness and also seized the opportunity to educate hearing people not to look down upon Southeast Asian migrant workers.

CONCLUSION

This article demonstrates how assistive hearing devices enter into a process of bricolage with linguistic and other symbolic resources, generating new potentials for the embodiment of affect. Here, we focus on interspeaker differences in affective display as one potential interpretation of differences in vowel production. When a speaker's assistive hearing device is turned off, the removal of the technological assistance and the linguistic resource collaboratively produce a body which is the extension of one of the speaker's selves, one developed through past experiences. When a physical body with technological assistance is contrasted with a physical body without such assistance, the self-images associated with them are also contrasted, and speakers orient to those images in different ways.

This article is not suggesting that there is no mechanistic contribution from the auditory deprivation to the linguistic performance. Instead, we suggest that the assistive hearing devices, in addition to their mechanical effects, also have semiotic potential. An embodied sociolinguistic approach works well to understand the relationship between D/HH persons and assistive hearing devices with respect to (not) feeling disabled.