Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2024-12-25T17:28:04.423Z Has data issue: false hasContentIssue false
  • English
  • Français

Matching the Mismatch: The interaction between perceptual and conceptual cues in bilinguals’ speech perception

Published online by Cambridge University Press:  04 November 2020

Noelle Wig  and
Adrián García-Sierra Open the ORCID record for Adrián García-Sierra [Opens in a new window]
Noelle Wig
Affiliation:
University of Connecticut
Adrián García-Sierra*
Affiliation:
University of Connecticut
*
Address for correspondence: Adrián García-Sierra, E-mail: adrian.garcia-sierra@uconn.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Speech perception involves both conceptual cues and perceptual cues. These, individually, have been shown to guide bilinguals’ speech perception; but their potential interaction has been ignored. Explicitly, bilinguals have been given perceptual cues that could be predicted by the conceptual cues. Therefore, to target the perceptual-conceptual interaction, we created a restricted range of perceptual cues that either matched, or mismatched, bilinguals’ conceptual predictions based on the language context. Specifically, we designed an active speech perception task that concurrently collected electrophysiological data from Spanish–English bilinguals and English monolinguals to address the extent to which this cue interaction uniquely affects bilinguals’ speech sound perception and allocation of attentional resources. Bilinguals’ larger MMN-N2b in the mismatched context aligns with the Predictive Coding Hypothesis to suggest that bilinguals use their diverse perceptual routines to best allocate cognitive resources to perceive speech.

Keywords

bilingualsspeech perceptionevent-related potentialsperceptual-conceptual interaction
Type
Research Article
Information
Bilingualism: Language and Cognition , Volume 24 , Issue 3 , May 2021 , pp. 467 - 480
Check for updates
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aaltonen, O, Niemi, P, Nyrke, T and Tuhkanen, M (1987) Event-related potentials and the perception of a phonetic continuum. Biological Psychology 24, 197–207.CrossRefGoogle ScholarPubMed
Abramson, AS and Lisker, L (1967) Discriminability along the voicing continuum: Cross language tests. Paper presented at the Proc. 6th International Congress of Phonetic Sciences, Prague.Google Scholar
Abramson, AS and Lisker, L (1972) Voice-timing perception in Spanish word-initial stops. Journal of Phonetics 1, 1–8.CrossRefGoogle Scholar
Antoniou, M, Tyler, MD and Best, CT (2012) Two ways to listen: Do L2-dominant bilinguals perceive stop voicing according to language mode? Journal of Phonetics 40, 582–594.CrossRefGoogle ScholarPubMed
Benet-Martínez, V and John, OP (1998) Los Cinco Grades across cultures and ethnic groups: Multitrait-multimethod analyses of the Big Five in Spanish and English. Journal of Personality and Social Psychology 75, 729–750.CrossRefGoogle Scholar
Berlad, I and Pratt, H (1995) P300 in response to the subject's own name. Electroencephalography and Clinical Neurophysiology 96, 472–474.CrossRefGoogle ScholarPubMed
Brady, SA and Darwin, CJ (1978) A range effect in the perception of voicing. Journal of the Acoustical Society of America 63, 1556–1558.CrossRefGoogle ScholarPubMed
Bohn, OS and Flege, JE (1993) Perceptual Switching in Spanish–English Bilinguals. Journal of Phonetics 21, 267–290.CrossRefGoogle Scholar
Bonala, B and Jansen, B (2012) A computational model for generation of the P300 evoked potential component. Journal of Integrative Neuroscience 11, 277–294.CrossRefGoogle ScholarPubMed
Bullmore, ET, Suckling, J, Overmeyer, S, Rabe-Hesketh, S, Taylor, E and Brammer, MJ (1999) Global, Voxel, and Cluster Tests, by Theory and Permutation, for a Difference Between Two Groups of Structural MR Images of the Brain. IEEE Transactions on Medical Imaging 18, 32–42.CrossRefGoogle ScholarPubMed
Casillas, JV, Díaz, Y and Simonet, M (2015) Acoustic of Spanish and English Coronal Stops. In The Scottish Consortium for ICPhS 2015 (ed), Proceedings of the 18th International Congress of Phonetic Sciences. Glasgow, UK: the University of Glasgow.Google Scholar
Casillas, JV and Simonet, M (2018) Perceptual categorization and bilingual language modes: Assessing the double phonemic boundary in early and late bilinguals. Journal of Phonetics 71, 51–64.CrossRefGoogle Scholar
Datta, H, Shafer, VL, Morr, ML, Kurtzberg, D and Schwartz, RG (2010) Electrophysiological Indices of Discrimination of Long-Duration, Phonetically Similar Vowels in Children With Typical and Atypical Language Development. Journal of Speech, Language, and Hearing Research 53, 757–777.CrossRefGoogle ScholarPubMed
Diehl, RL, Elman, JL and McCusker, SB (1978) Contrast effects on stop consonant identification. Human Perception and Performance 4, 599–609.CrossRefGoogle ScholarPubMed
Diesch, E and Luce, T (1997a) Magnetic fields elicited by tones and vowel formants reveal tonotopy and nonlinear summation of cortical activation. Psychophysiology 34, 501–510.CrossRefGoogle Scholar
Diesch, E and Luce, T (1997b) Magnetic mismatch fields elicited by vowels and consonants. Experimental Brain Research 116, 139–152.CrossRefGoogle Scholar
Dmitrieva, O, Llanos, F, Shultz, AA and Francis, A (2015) Phonological status, not voice onset time, determines the acoustic realization of onset F0 as a secondary voicing cue in Spanish and English. Journal of Phonetics 49, 77–95.CrossRefGoogle Scholar
Donchin, E (1981) Surprise!…Surprise? Psychophysiology 18, 493–513.CrossRefGoogle Scholar
Eimas, PD and Corbit, JD (1973) Selective adaptation of linguistic feature detectors. Cognitive Psychology 4, 99–109.CrossRefGoogle Scholar
Elman, JL, Diehl, RL and Buchwald, SE (1977) Perceptual switching in bilinguals. Journal of the Acoustical Society of America 62, 971–974.CrossRefGoogle Scholar
Ernst, MD (2004) Permutation Methods: A Basis for Exact Inference. Statistical Science 19, 676–685.CrossRefGoogle Scholar
Fish, MS, García-Sierra, A, Ramírez-Esparza, N and Kuhl, PK (2017) Infant-directed speech in English and Spanish: Assessments of monolingual and bilingual caregiver VOT. Journal of Phonetics 63, 19–34.CrossRefGoogle Scholar
Fishler, I, Jin, Y, Boaz, T, Perry, N and Childers, D (1987) Brain potentials related to seeing one's own name. Brain and Language 30, 245–262.CrossRefGoogle Scholar
Flege, JE (1982) Laryngeal timing and phonation onset in utterance-initial English stops. Journal of Phonetics 10, 177–192.CrossRefGoogle Scholar
Flege, JE and Eefting, W (1987a) Cross-Language Switching in Stop Consonant Perception and Production by Dutch Speakers of English. Speech Communication 6, 185–202.CrossRefGoogle Scholar
Flege, JE and Eefting, W (1987b) Production and Perception of English Stops by Native Spanish Speakers. Journal of Phonetics 15, 67–83.CrossRefGoogle Scholar
García-Sierra, A, Diehl, RL and Champlin, C (2009) Testing the double phonemic boundary in bilinguals. Speech Communication 51, 369–378.CrossRefGoogle ScholarPubMed
García-Sierra, A, Ramírez-Esparza, N and Kuhl, PK (2016) Relationships between quantity of language input and brain responses in bilingual and monolingual infants. International Journal of Psychophysiology 110, 1–17.CrossRefGoogle ScholarPubMed
García-Sierra, A, Ramírez-Esparza, N, Silva-Pereyra, J, Siard, J and Champlin, CA (2012) Assessing the double phonemic representation in bilingual speakers of Spanish and English: An electrophysiological study. Brain and Language 121, 194–205.CrossRefGoogle Scholar
García-Sierra, A, Rivera-Gaxiola, M, Percaccio, CR, Conboy, BT, Romo, H, Klarman, L, Ortiz, S and Kuhl, PK (2011) Bilingual language learning: An ERP study relating early brain responses to speech, language input, and later word production. Journal of Phonetics 39, 546–557.CrossRefGoogle Scholar
García-Sierra, A, Schifano, E, Duncan, GM and Fish, MS (in press) An analysis of the perception of stop consonants in bilinguals and monolinguals in two different phonetic contexts: a range-base language cueing approach. Action, Perception, & Psychophysics.Google Scholar
Garrido, MI, Kilner, JM, Stephan, KE and Friston, KJ (2009) The mismatch negativity: A review of underlying mechanisms. Clinical Neuropsychology 120, 453–463.Google ScholarPubMed
Gonzales, K, Byers-Heinlein, K and Lotto, AJ (2019) How bilinguals perceive speech depends on which language they think they're hearing. Cognition 182, 318–330.CrossRefGoogle ScholarPubMed
Gonzales, K and Lotto, AJ (2013) A bafri, un pafri: Bilinguals’ Pseudoword Identifications Support Language-Specific Phonetic Systems. Psychological Science 24, 2135–2142.CrossRefGoogle ScholarPubMed
Gray, HM, Ambady, N, Lowenthal, WT and Deldin, P (2004) P300 as an index to self-relevant stimuli. Journal of Experimental Social Psychology 40, 216–224.CrossRefGoogle Scholar
Groppe, DM, Urbach, TP and Kutas, M (2011) Mass univariate analysis of event-related brain potentials/fields I: A critical tutorial review. Psychophysiology 48, 1711–1725.CrossRefGoogle ScholarPubMed
Hazan, VL and Boulakia, G (1993) Perception and Production of a Voicing Contrast by French-English Bilinguals. Language and Speech 36, 17–38.CrossRefGoogle Scholar
Hay, JF (2005) How auditory discontinuities and linguistic experience affect the perception of speech and non-speech in English- and Spanish-speaking listeners (Doctoral dissertation). Available from Proquest Dissertations and Theses database. (UMI No. 3203519).CrossRefGoogle Scholar
Hisagi, M, Shafer, VL, Strange, W and Sussman, ES (2010) Perception of a Japanese vowel length contrast by Japanese and American English listeners: Behavioral and electrophysiological measures, Brain Research 1360, 89–105.CrossRefGoogle ScholarPubMed
Holt, LL (2005) Temporally nonadjacent nonlinguistic sounds affect speech categorization. Psychological Science 16, 305–312.CrossRefGoogle ScholarPubMed
Holt, LL and Lotto, AJ (2002) Behavioral examinations of the level of auditory processing of speech context effects. Hearing Research 167, 156–169.CrossRefGoogle ScholarPubMed
John, OP and Srivastava, S (1999) The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In Pervin, LA & John, OP (eds), Handbook of personality: Theory and research (2nd Ed), New York, NY: Guilford Press, pp. 102–138.Google Scholar
Johnson, R (1988) Scalp-recorded P300 Activity in Patients Following Unilateral Temporal Lobectomy, Brain 111, 1517–1529.CrossRefGoogle ScholarPubMed
Ju, M and Luce, PA (2004) Falling on Sensitive Ears: Constraints on Bilingual Lexical Activation. Psychological Science 15, 314–318.CrossRefGoogle ScholarPubMed
Jusczyk, PW (2000) The Discovery of Spoken Language. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Karmiloff-Smith, A (2010) Multiple Trajectories to Human Language Acquisition: Domain-Specific or Domain-General? Human Development 53, 239–244.CrossRefGoogle Scholar
Keating, PA, Mikos, MJ and Ganong, WF (1981) A Cross-Language Study of Range of Voice Onset Time in the Perception of Initial Stop Voicing. Journal of the Acoustical Society of America 70, 1261–1271.CrossRefGoogle Scholar
Kirsner, K, Brown, HL, Abrol, S, Chadha, K and Sharma, K (1980) Bilingualism and Lexical Representation. Quarterly Journal of Experimental Psychology 32, 585–594.CrossRefGoogle Scholar
Klatt, DH (1980) Software for a cascade/parallel formant synthesizer. Journal of the Acoustical Society of America 67, 971–990.CrossRefGoogle Scholar
Kuhl, PK, Williams, KA, Lacerda, F, Stevens, KN and Lindblom, B (1992) Linguistic experience alters phonetic perception in infants by 6 months of age. Science 255, 606–608.CrossRefGoogle ScholarPubMed
Lagrou, E, Hartsuiker, RJ and Duyck, W (2011) Knowledge of a second language influences auditory word recognition in the native language. Journal of Experimental Psychology: Learning, Memory, and Cognition 37, 952–965.Google ScholarPubMed
Lauro, JG and Schwartz, AI (2017) Bilingual non-selective lexical access in sentence contexts: A meta-analytic review. The Journal of Memory and Language 92, 217–233.CrossRefGoogle Scholar
Linden, D (2005) The P300: Where in the brain is it produced and what does it tell us? The Neuroscientist 11, 563–576.CrossRefGoogle ScholarPubMed
Lotto, AJ, Sullivan, SC and Holt, LL (2003) Central locus for nonspeech context effects on phonetic identification. Journal of the Acoustical Society of America 113, 53–56.CrossRefGoogle ScholarPubMed
Macmillan, NA and Creelman, CD (2005) Detection Theory: A User's Guide (Second ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Luk, G and Bialystok, E (2013) Bilingualism is not a categorical variable: Interaction between language proficiency and usage. Journal of Cognitive Psychology 25, 605–621.CrossRefGoogle Scholar
Maris, E (2012) Statistical testing in electrophysiological studies. Psychophysiology 49, 549–565.CrossRefGoogle ScholarPubMed
Maris, E and Oostenveld, R (2007) Nonparametric statistical testing of EEG- and MEG-data. Journal of Neuroscience Methods 164, 177–190.CrossRefGoogle ScholarPubMed
Näätänen, R (1982) Processing Negativity - an Evoked-Potential Reflection of Selective Attention. Psychological Bulletin 92, 605–640.CrossRefGoogle ScholarPubMed
Näätänen, R (1992) Attention and Brain Function. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Näätänen, R (2001) The perception of speech sounds by the human brain as reflected by the mismatch negativity (MMN) and its magnetic equivalent (MMNm). Psychophysiology 38, 1–21.CrossRefGoogle Scholar
Näätänen, R, Gaillard, AWK and Mantysalo, S (1978) Early selective attention effect on evoked potential reinterpreted. Acta Psychologica 42, 313–329.CrossRefGoogle ScholarPubMed
Näätänen, R and Michie, PT (1979) Early Selective-Attention Effects on the Evoked-Potential - a Critical-Review and Reinterpretation. Biological Psychology 8, 81–136.CrossRefGoogle ScholarPubMed
Ninomiya, H, Onitsuka, T, Chen, C and Sato, E (1998) P300 in response to subject's own face. Psychiatry and Clinical Neurosciences 52, 519–522.CrossRefGoogle ScholarPubMed
Novak, G, Ritter, W and Vaughan, HG Jr. (1992) Mismatch Detection and the Latency of Temporal Judgments. Psychophysiology 29, 398–411.CrossRefGoogle Scholar
Onitsuka, T, Ninomiya, H, Chen, C and Sato, E (1997) P300 in response to subject's own voice. Japanese Journal of EEG EMG 25, 243–248.Google Scholar
Pastore, RE, Ahroon, WA, Baffuto, KJ, Friedman, C, Puleo, JS and Fink, EA (1977) Common-factor model of categorical perception. Journal of Experimental Psychology: Human Perception and Performance 3, 686–696.Google Scholar
Picton, TW (1992) The P300 Wave of the Human Event-Related Potential. Journal of Clinical Neuropsychology 9, 456–479.Google ScholarPubMed
Phillips, C, Pellathy, T, Marantz, A, Yellin, E, Wexler, K, Poeppel, D, McGinnis, M and Roberts, T (2000) Auditory cortex accesses phonological categories: An MEG mismatch study. Journal of Cognitive Neuroscience 12, 1038–1055.CrossRefGoogle ScholarPubMed
Polich, J (1988) Bifurcated P300 peaks: P3a and P3b revisited? Journal of Clinical Neurophysiology 5, 287–294.CrossRefGoogle ScholarPubMed
Polich, J (2003) Theoretical overview of P3a and P3b. In: Polich, J (eds) Detection of Change. Springer, Boston, MA.CrossRefGoogle Scholar
Polich, J (2007) Updating P300: an integrative theory of P3a and P3b. Journal of Clinical Neurophysiology 118, 2128–2148.CrossRefGoogle ScholarPubMed
Ramírez-Esparza, N, García-Sierra, A and Kuhl, PK (2014) Look who's talking: speech style and social context in language input to infants are linked to concurrent and future speech development. Developmental Science 17, 880–891.CrossRefGoogle ScholarPubMed
Ramírez-Esparza, N, García-Sierra, A and Kuhl, PK (2017) The Impact of Early Social Interactions on Later Language Development in Spanish–English Bilingual Infants. Child Development 88, 1216–1234.CrossRefGoogle ScholarPubMed
Rivera-Gaxiola, M, Csibra, G, Johnson, MH and Karmiloff-Smith, A (2000a) Electrophysiological correlates of cross-linguistic speech perception in native English speakers. Behavioral Brain Research 111, 13–23.CrossRefGoogle Scholar
Rivera-Gaxiola, M, Johnson, MH, Csibra, G and Karmiloff-Smith, A (2000b) Electrophysiological correlates of category goodness. Behavioral Brain Research 112, 1–11.CrossRefGoogle Scholar
Sharma, A and Dorman, MF (1998) Exploration of the perceptual magnet effect using the mismatch negativity auditory evoked potential. Journal of the Acoustical Society of America 104, 511–517.CrossRefGoogle ScholarPubMed
Streeter, LA (1976) Language perception of 2-month-old infants shows effects of both innate mechanisms and experience. Nature 259, 39–41.CrossRefGoogle ScholarPubMed
Sundara, M, Polka, L and Baum, S (2006) Production of coronal stops by simultaneous bilingual adults. Bilingualism: Language and Cognition 9, 97–114.CrossRefGoogle Scholar
Sundara, M and Polka, L (2008) Discrimination of coronal stops by bilingual adults: The timing and nature of language interaction. Cognition 106(1), 234–258.CrossRefGoogle ScholarPubMed
Sussman, E, Kujala, T, Halmetoja, J, Lyytinen, H, Alku, P and Näätänen, R (2004) Automatic and controlled processing of acoustic and phonetic contrasts. Hearing Research 190, 128–140.CrossRefGoogle ScholarPubMed
Sussman, E and Steinschneider, M (2009) Attention effects on auditory scene analysis in children. Neuropsychologia 47, 771–785.CrossRefGoogle ScholarPubMed
Sutton, S, Braren, M, Zubin, J and John, ER (1965) Evoked-Potential Correlates of Stimulus Uncertainty. Science 150, 1187–1187.CrossRefGoogle ScholarPubMed
Werker, J (2012) Perceptual foundations of bilingual acquisition in infany. Annals of the New York Academy of Sciences 1251, 50–61.CrossRefGoogle Scholar
Williams, L (1977) The perception of stop consonant voicing by Spanish–English bilinguals. Perception & Psychophysics 21, 289–297.CrossRefGoogle Scholar
Williams, L (1979) The modification of speech perception and production in second-language learning. Perception & Psychophysics 26, 95–104.CrossRefGoogle ScholarPubMed
Winkler, I, Kujala, T, Alku, P and Näätänen, R (2003) Language context and phonetic change detection. Cognitive Brain Research 17, 833–844.CrossRefGoogle ScholarPubMed
Winkler, I, Kujala, T, Tiitinen, P, Sivonen, P, Alku, P, Lehtokoski, A, Czigler, I, Csépe, V, Ilmoniemi, RJ and Näätänen, R (1999) Brain responses reveal the learning of foreign language phonemes. Psychophysiology 36, 638–642.CrossRefGoogle ScholarPubMed
Supplementary material: PDF

Wig and García-Sierra supplementary material

Wig and García-Sierra supplementary material

Download Wig and García-Sierra supplementary material(PDF)
PDF 331.6 KB