Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-14T05:12:02.480Z Has data issue: false hasContentIssue false
  • English
  • Français

AGE MATTERS, AND SO MAY RATERS

Rater Differences in the Assessment of Foreign Accents

Published online by Cambridge University Press:  29 October 2014

Becky H. Huang  and
Sun-Ah Jun
Becky H. Huang*
Affiliation:
University of Texas at San Antonio
Sun-Ah Jun
Affiliation:
University of California Los Angeles
*
*Correspondence concerning this article should be addressed to Becky Huang, Department of Bicultural-Bilingual Studies, The University of Texas at San Antonio, One UTSA Circle, San Antonio, TX 78249. E-mail: becky.huang@utsa.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Research on the age of learning effect on second language learners’ foreign accents utilizes human judgments to determine speech production outcomes. Inferences drawn from analyses of these ratings are then used to inform theories. The present study focuses on rater differences in the age of learning effect research. Three groups of raters who differed in their native language background and language experience participated in the study: inexperienced native English speaker (NES) raters, experienced NES raters, and advanced nonnative English speaker (NNES) raters. All raters evaluated 64 speech samples taken from both NESs and NNESs who varied in their age of arrival in the second-language-speaking country. Results from the study showed that experienced NES raters were better than the other two rater groups at distinguishing NESs from NNESs. Although no rater group differences were found in scaling speakers’ foreign accents or in their interrater reliability, inexperienced NES raters were stricter in their ratings than both experienced NES raters and advanced NNES raters. Implications for research on age of learning effects are discussed.

Type
Articles
Information
Studies in Second Language Acquisition , Volume 37 , Issue 4 , December 2015 , pp. 623 - 650
Check for updates
Copyright
Copyright © Cambridge University Press 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Abrahamsson, N., & Hyltenstam, K. (2009). Age of onset and nativelikeness in a second language: Listener perception versus linguistic scrutiny. Language Learning, 59, 249–306.CrossRefGoogle Scholar
Asher, J. J., & García, R. (1969). The optimal age to learn a foreign language. Modern Language Journal, 53, 334–341.CrossRefGoogle Scholar
Audacity Team. (2000). Audacity (Version 1.2.5) [Computer software]. Retrieved fromhttp://audacity.sourceforge.net/Google Scholar
Bachman, L. F., Lynch, B. K., & Mason, M. (1995). Investigating variability in tasks and rater judgements in a performance test of foreign language speaking. Language Testing, 12, 238–257.CrossRefGoogle Scholar
Baker, W., Eddington, D., & Nay, L. (2009). Dialect identification: The effects of region of origin and amount of experience. American Speech, 84, 48–71.CrossRefGoogle Scholar
Barnwell, D. (1989). “Naïve” native speakers and judgments of oral proficiency in Spanish. Language Testing, 6, 152–163.CrossRefGoogle Scholar
Boersma, P., & Weenink, D. (2009). Praat: Doing phonetics by computer (Version 5.1.05) [Computer program]. Retrieved fromhttp://www.fon.hum.uva.nl/praat/Google Scholar
Bongaerts, T. (1999a). De keuze van beoordelaars in onderzoek naar uitspreekvaardigheid in een vreemde taal [Selecting raters for research on L2 pronunciation]. In Gerritsen, M. & Springorum, D. (Eds.), Een bundel bedrijfscommunicatie voor Ger Peerbooms bij gelegenheid van zijn 65e verjaardag (pp. 1–10). Nijmegen, the Netherlands: Nijmegen University Press.Google Scholar
Bongaerts, T. (1999b). Ultimate attainment in foreign language pronunciation: The case of very advanced late foreign language learners. In Birdsong, D. (Ed.), Second language acquisition and the Critical Period Hypothesis (pp. 133–159). Mahwah, NJ: Erlbaum.Google Scholar
Bongaerts, T., Mennen, S., & van der Slik, F. (2000). Authenticity of pronunciation in naturalistic second language acquisition: The case of very advanced late learners of Dutch as a second language. Studia linguistica, 54, 298–308.CrossRefGoogle Scholar
Bongaerts, T., Planken, B., & Schils, E. (1995). Can late starters attain a native accent in a foreign language? A test of the Critical Period Hypothesis. In Singleton, D. & Lengyel, Z. (Eds.), The age factor in second language acquisition (pp. 30–50). Clevedon, UK: Multilingual Matters.CrossRefGoogle Scholar
Bongaerts, T., van Summeren, C., Planken, B., & Schils, E. (1997). Age and ultimate attainment in the pronunciation of a foreign language. Studies in Second Language Acquisition, 19, 447–465.CrossRefGoogle Scholar
Bradlow, A., & Bent, T. (2003). Listener adaptation to foreign-accented English. In Solé, M. J., Recasens, D., & Romero, J. (Eds.), Proceedings of the 15th International Congress of Phonetic Sciences (pp. 1581–1583). Barcelona, Spain: Universitat Autònoma de Barcelona.Google Scholar
Bradlow, A. R., & Bent, T. (2008). Perceptual adaptation to non-native speech. Cognition, 106, 707–729.CrossRefGoogle ScholarPubMed
Brodkey, D. (1972). Dictation as a measure of mutual intelligibility. Language Learning, 22, 203–220.CrossRefGoogle Scholar
Caban, H. L. (2003). Rater group bias in the speaking assessment of four L1 Japanese ESL students. Second Language Studies, 21, 1–43.Google Scholar
Carey, M. D., Mannell, R. H., & Dunn, P. K. (2011). Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews? Language Testing, 28, 201–219.CrossRefGoogle Scholar
Chalhoub-Deville, M. (1995). A contextualized approach to describing oral language proficiency. Language Learning, 45, 251–281.CrossRefGoogle Scholar
Clarke, C. M., & Garrett, M. F. (2004). Rapid adaptation to foreign-accented English. Journal of the Acoustical Society of America, 116, 3647–3658.CrossRefGoogle ScholarPubMed
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar
Cook, V. (1999). Going beyond the native speaker in language teaching. TESOL Quarterly, 33, 185–209.CrossRefGoogle Scholar
Dąbrowska, E. (2010). Naive vs. expert intuitions: An empirical study of acceptability judgments. The Linguistic Review, 27, 1–23.CrossRefGoogle Scholar
Davies, A. (1999). Standard English: Discordant voices. World Englishes, 18, 171–186.CrossRefGoogle Scholar
Davies, A. (2003). The native speaker: Myth and reality. Clevedon, UK: Multilingual Matters.Google Scholar
Derwing, T. M., & Munro, M. J. (2013). The development of L2 oral language skills in two L1 groups: A 7-year study. Language Learning, 63, 163–185.CrossRefGoogle Scholar
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1, 170–177.CrossRefGoogle Scholar
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.CrossRefGoogle Scholar
Faez, F. (2011). Reconceptualizing the native/nonnative speaker dichotomy. Journal of Language, Identity & Education, 10, 231–249.CrossRefGoogle Scholar
Fayer, J. M., & Krasinski, E. (1987). Native and nonnative judgments of intelligibility and irritation. Language Learning, 37, 313–326.CrossRefGoogle Scholar
Flege, J. E. (1988). Factors affecting degree of perceived foreign accent in English sentences. Journal of the Acoustical Society of America, 84, 70–79.CrossRefGoogle ScholarPubMed
Flege, J. E. (1995). Second-language speech learning: Theory, findings, and problems. In Strange, W. (Ed.), Speech perception and linguistic experience: Issues in cross-language research (pp. 229–273). Timonium, MD: York Press.Google Scholar
Flege, J. E., Frieda, E. M., & Nozawa, T. (1997). Amount of native-language (L1) use affects the pronunciation of an L2. Journal of Phonetics, 25, 169–186.CrossRefGoogle Scholar
Flege, J. E., Munro, M. J., & MacKay, I. R. (1995). Factors affecting strength of perceived foreign accent in a second language. Journal of the Acoustical Society of America, 97, 3125–3135.CrossRefGoogle Scholar
Flege, J. E., Yeni-Komshian, G. H., & Liu, S. (1999). Age constraints on second-language acquisition. Journal of Memory and Language, 41, 78–104.CrossRefGoogle Scholar
Galloway, V. B. (1980). Perceptions of the communicative efforts of American students of Spanish. Modern Language Journal, 64, 428–433.CrossRefGoogle Scholar
Gass, S., & Varonis, E. (1984). The effect of familiarity on the comprehensibility of nonnative speech. Language Learning, 34, 65–89.CrossRefGoogle Scholar
Hadden, B. L. (1991). Teacher and nonteacher perceptions of second-language communication. Language Learning, 41, 1–24.CrossRefGoogle Scholar
Hsieh, C.-N. (2011). Rater effects in ITA testing: ESL teachers’ versus American undergraduates’ judgments of accentedness, comprehensibility, and oral proficiency. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 9, 47–74.Google Scholar
Huang, B. H. (2014). The effects of age on second language grammar and speech production. Journal of Psycholinguistic Research, 43, 397–420.CrossRefGoogle ScholarPubMed
Huang, B. H., & Jun, S.-A. (2011). Specifying the age-related effect on the acquisition of second language prosody. Language and Speech, 54, 387–414.CrossRefGoogle Scholar
Isaacs, T., & Thomson, R. I. (2013). Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions. Language Assessment Quarterly, 10, 135–159.CrossRefGoogle Scholar
Kennedy, S., & Trofimovich, P. (2008). Intelligibility, comprehensibility, and accentedness of L2 speech: The role of listener experience and semantic context. Canadian Modern Language Review, 64, 459–489.CrossRefGoogle Scholar
Keselman, H. J., Algina, J., & Kowalchuk, R. K. (2001). The analysis of repeated measures designs: A review. British Journal of Mathematical and Statistical Psychology, 54, 1–20.CrossRefGoogle ScholarPubMed
Kim, Y. H. (2009). An investigation into native and non-native teachers’ judgments of oral English performance: A mixed methods approach. Language Testing, 26, 187–217.CrossRefGoogle Scholar
Kuhl, P. K. (2004). Early language acquisition: Cracking the speech code. Nature Reviews Neuroscience, 5, 831–843.CrossRefGoogle ScholarPubMed
Kunnan, A. J. (Ed.). (2000). Fairness and validation in language assessment: Selected papers from the 19th Language Testing Research Colloquium, Orlando, Florida (Vol. 9). Cambridge, UK: Cambridge University Press.Google Scholar
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.Google ScholarPubMed
Lix, L. M., & Keselman, H. J. (1995). Approximate degrees of freedom tests: A unified perspective on testing for mean equality. Psychological Bulletin, 117, 547–560.CrossRefGoogle Scholar
Long, M. H. (2005). Problems with supposed counter-evidence to the Critical Period Hypothesis. International Review of Applied Linguistics, 43, 287–317.CrossRefGoogle Scholar
MacKay, I. R., Flege, J. E., & Imai, S. (2006). Evaluating the effects of chronological age and sentence duration on degree of perceived foreign accent. Applied Psycholinguistics, 27, 157–183.CrossRefGoogle Scholar
Major, R. C. (2007). Identifying a foreign accent in an unfamiliar language. Studies in Second Language Acquisition, 29, 539–556.CrossRefGoogle Scholar
Markham, D. (1997). Phonetic imitation, accent, and the learner. Lund, Sweden: Lund University Press.Google Scholar
MATLAB (Version 6.5.1) [Computer software]. Natick, MA: MathWorks.Google Scholar
Moyer, A. (1999). Ultimate attainment in L2 phonology. Studies in Second Language Acquisition, 21, 81–108.CrossRefGoogle Scholar
Moyer, A. (2004). Age, accent and experience in second language acquisition. Clevedon, UK: Multilingual Matters.CrossRefGoogle Scholar
Mulder, K., & Hulstijn, J. H. (2011). Linguistic skills of adult native speakers, as a function of age and level of education. Applied Linguistics, 32, 475–494.CrossRefGoogle Scholar
Munro, M. J., Derwing, T. M., & Morton, S. L. (2006). The mutual intelligibility of L2 speech. Studies in Second Language Acquisition, 28, 111–131.CrossRefGoogle Scholar
Munro, M., & Mann, V. (2005). Age of immersion as a predictor of foreign accent. Applied Psycholinguistics, 26, 311–341.CrossRefGoogle Scholar
Navarra, J., Sebastián-Gallés, N., & Soto-Faraco, S. (2005). The perception of second language sounds in early bilinguals: New evidence from an implicit measure. Journal of Experimental Psychology: Human Perception and Performance, 31, 912–918.Google ScholarPubMed
Nikolov, M. (2000). The Critical Period Hypothesis reconsidered: Successful adult learners of Hungarian and English. International Review of Applied Linguistics in Language (IRAL) Teaching, 38, 109–124.Google Scholar
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50, 417–528.CrossRefGoogle Scholar
Oyama, S. (1976). A sensitive period for the acquisition of a nonnative phonological system. Journal of Psycholinguistic Research, 5, 261–283.CrossRefGoogle Scholar
Pallier, C., Bosch, L., & Sebastián-Gallés, N. (1997). A limit on behavioral plasticity in speech perception. Cognition, 64, B9–B17.CrossRefGoogle ScholarPubMed
Piller, I. (2002). Passing for a native speaker: Identity and success in second language learning. Journal of Sociolinguistics, 6, 179–208.CrossRefGoogle Scholar
Piske, T., MacKay, I. R., & Flege, J. E. (2001). Factors affecting degree of foreign accent in an L2: A review. Journal of Phonetics, 29, 191–215.CrossRefGoogle Scholar
Pressley, M., & Afflerbach, P. P. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.Google Scholar
Scovel, T. (2000). A critical review of the critical period research. Annual Review of Applied Linguistics, 20, 213–223.CrossRefGoogle Scholar
Sebastián-Gallés, N., & Soto-Faraco, S. (1999). Online processing of native and non-native phonemic contrasts in early bilinguals. Cognition, 72, 111–123.CrossRefGoogle ScholarPubMed
Thompson, I. (1991). Foreign accents revisited: The English pronunciation of Russian immigrants. Language Learning, 41, 177–204.CrossRefGoogle Scholar
Tomasello, M. (1992). First verbs: A case study of early grammatical development. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
Trofimovich, P., Lightbown, P. M., Halter, R., & Song, H. (2009). Comprehension-based practice: The development of L2 pronunciation in a listening and reading program. Studies in Second Language Acquisition, 31, 609–639.CrossRefGoogle Scholar
Urbaniak, G. C., & Plous, S. (2013). Research randomizer (Version 4.0) [Computer software]. Retrieved fromhttp://www.randomizer.org/Google Scholar
Weinberger, S. (2014). Speech accent archive [Database]. Retrieved fromhttp://accent.gmu.edu.Google Scholar
Winke, P., & Gass, S. (2013). The influence of second language experience and accent familiarity on oral proficiency rating: A qualitative investigation. TESOL Quarterly, 47, 762–789.CrossRefGoogle Scholar
Winke, P., Gass, S., & Myford, C. (2013). Raters’ L2 background as a potential source of bias in rating oral performance. Language Testing, 30, 231–252.CrossRefGoogle Scholar
Xi, X., & Mollaun, P. (2011). Using raters from India to score a large test. Language Learning, 61, 1222–1255.CrossRefGoogle Scholar
Zhang, Y., & Elder, C. (2011). Judgments of oral proficiency by non-native and native English speaking teacher raters: Competing or complementary constructs? Language Testing, 28, 31–50.CrossRefGoogle Scholar