Hostname: page-component-78c5997874-4rdpn Total loading time: 0 Render date: 2024-11-11T12:01:26.692Z Has data issue: false hasContentIssue false
  • English
  • Français

A neural network model of the effects of entrenchment and memory development on grammatical gender learning*

Published online by Cambridge University Press:  30 August 2012

DEREK MONNER ,
KAREN VATZ ,
GIOVANNA MORINI ,
SO-ONE HWANG  and
ROBERT DeKEYSER
DEREK MONNER*
Affiliation:
Department of Computer Science, University of MarylandCollege Park
KAREN VATZ
Affiliation:
Center for Advanced Study of Language, University of MarylandCollege Park
GIOVANNA MORINI
Affiliation:
Department of Hearing and Speech Sciences, University of MarylandCollege Park
SO-ONE HWANG
Affiliation:
Center for Research in Language, University of CaliforniaSan Diego
ROBERT DeKEYSER
Affiliation:
Second Language Acquisition Program, University of MarylandCollege Park
*
Address for correspondence: Derek Monner, A.V. Williams Bldg #3136, University of Maryland, College Park, MD 20742, USAdmonner@cs.umd.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

To investigate potential causes of L2 performance deficits that correlate with age of onset, we use a computational model to explore the individual contributions of L1 entrenchment and aspects of memory development. Since development and L1 entrenchment almost invariably coincide, studying them independently is seldom possible in humans. To avoid this confound, we study neural network models that learn to solve gender assignment and agreement tasks in Spanish and French. We model the learner as a collection of recurrent cell assemblies that subserve working memory and are facilitated by trainable long-term connections. Varying the time-course over which assemblies and connections are added allows us to compare small, growing, child-like networks to fixed-size adult-like ones. Networks undergo variable-length exposure to L1 before L2 onset to control the amount of L1 entrenchment. This model, by allowing us independent control of both variables, lends us a novel glimpse of all sides of their interaction and affords a rare test of the less-is-more hypothesis. Network comparisons suggest that final L2 proficiency declines as L2 onset delays increase relative to L1, implicating an L1 entrenchment effect. However, aspects of memory development during learning play a key role in mitigating these impairments, lending support to less-is-more as a contributor to sensitive periods.

Keywords

age effectsgrammatical genderentrenchmentmemory developmentrecurrent neural networks
Type
Research Article
Information
Bilingualism: Language and Cognition , Volume 16 , Issue 2: Computational Modeling of Bilingualism , April 2013 , pp. 246 - 265
Copyright
Copyright © Cambridge University Press 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Thanks to two anonymous reviewers for helpful comments. This work was supported in part by NSF IGERT award DGE-0801465.

References

Abrahamsson, N., & Hyltenstam, K. (2008). The robustness of aptitude effects in near-native second language acquisition. Studies in Second Language Acquisition, 30 (4), 481–509.Google Scholar
Abrahamsson, N., & Hyltenstam, K. (2009). Age of onset and nativelikeness in a second language: Listener perception versus linguistic scrutiny. Language Learning, 59 (2), 249–306.Google Scholar
Avons, S. E., Wragg, C. A., Cupplesa, W. L., & Lovegrove, W. J. (1998). Measures of phonological short-term memory and their relationship to vocabulary development. Applied Psycholinguistics, 19, 583–601.Google Scholar
Baddeley, A. (2003). Working memory and language: An overview. Journal of Communication Disorders, 36, 189–208.CrossRefGoogle ScholarPubMed
Baddeley, A. D., Gathercole, S. E., & Papagno, C. (1998). The phonological loop as a language learning device. Psychological Review, 105, 158–173.Google Scholar
Baker, W. (2010). Effects of age and experience on the production of English word-final stops by Korean speakers. Bilingualism: Language and Cognition, 13 (3), 263–278.Google Scholar
Bley-Vroman, R. (1988). The fundamental character of foreign language learning. In Rutherford, W. & Sharwood Smith, M. (eds.), Grammar and second language teaching: A book of readings, pp. 19–30. New York: Newbury House.Google Scholar
Brown, G. D. A., & Hulme, C. (1996). Non-word repetition, STM, and age-of-acquisition: A computational model. In Gathercole, S. E. (ed.), Models of short-term memory, pp. 129–148, Hove: Psychology Press.Google Scholar
Carroll, S. E. (1995). The hidden dangers of computer modeling: Remarks on Sokolik and Smithʼs connectionist learning model of French gender. Second Language Research, 11 (3), 193–205.Google Scholar
Chen, L., Shu, H., Liu, Y., Zhao, J., & Li, P. (2007). ERP signatures of subject–verb agreement in L2 learning. Bilingualism: Language and Cognition, 10 (2), 161–174.Google Scholar
Cochran, B. P., McDonald, J. L., & Parault, S. J. (1999). Too smart for their own good: The disadvantage of a superior processing capacity for adult language learners. Journal of Memory and Language, 41, 30–58.Google Scholar
Conway, A. R. A., Cowan, N., & Bunting, M. F. (2001). The cocktail party phenomenon revisited: The importance of WM capacity. Psychonomic Bulletin & Review, 8, 331–335.Google Scholar
Corbett, G. (1991). Gender. New York: Cambridge University Press.Google Scholar
CUMBRE (n.d.). Corpus del Español Contemporáneo de España e Hispanoamérica. Madrid: SGEL.Google Scholar
DeKeyser, R. M. (2000). The robustness of critical period effects in second language acquisition. Studies in Second Language Acquisition, 22 (4), 499–533.Google Scholar
DeKeyser, R. M. (2012). Age effects in second language learning. In Gass, S. & Mackey, A. (eds.), Handbook of second language acquisition, pp. 442–460. London: Routledge.Google Scholar
DeKeyser, R. M., Alfi-Shabtay, I., & Ravid, D. (2010). Cross-linguistic evidence for the nature of age effects in second language acquisition. Applied Psycholinguistics, 31 (3), 413–438.Google Scholar
DeKeyser, R. M., & Larson-Hall, J. (2005). What does the critical period really mean? In Kroll, J. F. & de Groot, A. M. B. (eds.), Handbook of bilingualism: Psycholinguistic approaches, pp. 89–108. Oxford: Oxford University Press.Google Scholar
Duncan, J., Seitz, R. J., Kolodny, J., Bor, D., Herzog, H., & Ahmed, A. (2000). A neural basis for general intelligence. Science, 289 (5478), 457.Google Scholar
Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179–211.Google Scholar
Elman, J. L. (1993). Learning and development in neural networks: The importance of starting small. Cognition, 48, 71–99.Google Scholar
Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11 (1), 19–23.CrossRefGoogle Scholar
Gathercole, S. E. (1999). Cognitive approaches to the development of short-term memory. Trends in Cognitive Sciences, 3 (11), 410–419.Google Scholar
Gathercole, S. E., & Baddeley, A. D. (1993). Working memory and language. Hove: Lawrence Erlbaum.Google Scholar
Gathercole, S. E., & Pickering, S. J. (2000). Assessment of working memory in six- and seven-year old children. Journal of Educational Psychology, 92, 377–390.Google Scholar
Gers, F. A., & Cummins, F. (2000). Learning to forget: Continual prediction with LSTM. Neural Computation, 12 (10), 2451–2471.Google Scholar
Gers, F. A., & Schmidhuber, J. (2001). LSTM recurrent networks learn simple context-free and context-sensitive languages. IEEE Transactions on Neural Networks, 12 (6), 1333–1340.Google Scholar
Goldowsky, B. N., & Newport, E. L. (1993). Modeling the effects of processing limitations on the acquisition of morphology: The less is more hypothesis. In Clark, E. (ed.), Proceedings of the 24th Annual Child Language Research Forum, pp. 124–138. Stanford, CA: Center for the Study of Language and Information (CSLI).Google Scholar
Guillelmon, D., & Grosjean, F. (2001). The gender marking effect in spoken word recognition: The case of bilinguals. Memory & Cognition, 29, 503–511.CrossRefGoogle ScholarPubMed
Guion, S. G., Harada, T., & Clark, J. J. (2004). Early and late Spanish–English bilinguals’ acquisition of English word stress patterns. Bilingualism: Language and Cognition, 7, 207–226.Google Scholar
Hakuta, K., Bialystok, E., & Wiley, E. (2003). Critical evidence: A test of the critical-period hypothesis for second-language acquisition. Psychological Science, 14 (1), 31–38.Google Scholar
Harley, B. (1979). French gender ‘rules’ in the speech of English-dominant, French-dominant and monolingual French-speaking children. Working Papers in Bilingualism, 19, 129–156.Google Scholar
Hernandez, A., & Li, P. (2007). Age of acquisition: Its neural and computational mechanisms. Psychological Bulletin, 133 (4), 638–650.Google Scholar
Hernandez, A., Li, P., & MacWhinney, B. (2005). The emergence of competing modules in bilingualism. Trends in Cognitive Sciences, 9 (5), 220–225.Google Scholar
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9 (8), 1735–1780.Google Scholar
Hudson Kam, C. L., & Newport, E. L. (2005). Regularizing unpredictable variation: The roles of adult and child learners in language formation and change. Language Learning and Development, 1 (2), 151–195.CrossRefGoogle Scholar
Hudson Kam, C. L., & Newport, E. L. (2009). Getting it right by getting it wrong: When learners change languages. Cognitive Psychology, 59, 30–66.CrossRefGoogle Scholar
Hyltenstam, K., & Abrahamsson, N. (2003). Maturational constraints in second language acquisition. In Doughty, C. J. & Long, M. H. (eds.), Handbook of second language acquisition, pp. 539–588. Oxford: Blackwell.Google Scholar
Iverson, P., Kuhl, P. K., Akahane-Yamada, R., Diesch, E., Tohkura, Y., Kettermann, A., & Siebert, C. (2003). A perceptual interference account of acquisition difficulties for non-native phonemes. Cognition, 87, B47–57.Google Scholar
Jia, G., & Aaronson, D. (2003). A longitudinal study of Chinese children and adolescents learning English in the United States. Applied Psycholinguistics, 24 (1), 131–161.Google Scholar
Jia, G., Aaronson, D., & Wu, Y. (2002). Long-term language attainment of bilingual immigrants: Predictive variables and language group differences. Applied Psycholinguistics, 23 (4), 599–621.CrossRefGoogle Scholar
Johnson, J. S., & Newport, E. L. (1989). Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 21, 60–99.CrossRefGoogle ScholarPubMed
Kane, M. J., & Engle, R. W. (2003). Working memory capacity and the control of attention: The contributions of goal neglect, response competition, and task set to Stroop interference. Journal of Experimental Psychology: General, 132, 47–70.Google Scholar
Kareev, Y., Lieberman, I., & Lev, M. (1997). Through a narrow window: Sample size and the perception of correlation. Journal of Experimental Psychology: General, 126 (3), 278–287.Google Scholar
Kersten, A. W., & Earles, J. L. (2001). Less really is more for adults learning a miniature artificial language. Journal of Memory and Language, 44, 250–273.Google Scholar
Lapkin, S., & Swain, M. (1977). The use of English and French cloze tests in a bilingual education program evaluation: Validity and error analysis. Language Learning, 27, 279–314.Google Scholar
Lenneberg, E. H. (1967). Biological foundations of language. New York: Wiley.Google Scholar
Lew-Williams, C., & Fernald, A. (2010). Real-time processing of gender-marked articles by native and non-native Spanish speakers. Journal of Memory and Language, 63, 447–464.Google Scholar
Li, P., Farkas, I., & MacWhinney, B. (2004). Early lexical development in a self-organizing neural network. Neural Networks, 17 (8–9), 1345–1362.Google Scholar
Li, P., Zhao, X., & Mac Whinney, B. (2007). Dynamic self-organization and early lexical development in children. Cognitive Science, 31 (4), 581–612.Google Scholar
Long, M. (1990). Maturational constraints on language development. Studies in Second Language Acquisition, 12 (3), 251–285.Google Scholar
Long, M. (2005). Problems with supposed counter-evidence to the Critical Period Hypothesis. International Review of Applied Linguistics in Language Teaching (IRAL), 43 (4), 287–316.Google Scholar
Lyster, R. (2006). Predictability in French gender attribution: A corpus analysis. Journal of French Language Studies, 16, 69–92.Google Scholar
MacWhinney, B. (2006). Emergent fossilization. In Han, Z. & Odlin, T. (eds.), Studies of fossilization in second language acquisition, pp. 134–156. Clevedon: Multilingual Matters.Google Scholar
MacWhinney, B., Leinbach, J., Taraban, R., & McDonald, J. (1989). Language learning: Cues or rules? Journal of Memory and Language, 28, 255–277.Google Scholar
Matthews, C. A. (1999). Connectionism and French gender attribution: Sokolik and Smith re-visited. Second Language Research, 15, 412–427.Google Scholar
Mayberry, R. I. (1993). First-language acquisition after childhood differs from second-language acquisition: The case of American Sign Language. Journal of Speech and Hearing Research, 36, 51–68.Google Scholar
Mayberry, R. I., & Lock, E. (2003). Age constraints on first versus second language acquisition: Evidence for linguistic plasticity and epigenesis. Brain and Language, 87, 369–383.Google Scholar
Mayberry, R. I., Lock, E., & Kazmi, H. (2002). Linguistic ability and early language exposure. Nature, 417, 38.CrossRefGoogle ScholarPubMed
Metsala, J. L. (1999). Young children's phonological awareness and non-word repetition as a function of vocabulary development. Journal of Educational Psychology, 91, 3–19Google Scholar
Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81–97.Google Scholar
Monner, D., & Reggia, J. A. (2012). A generalized LSTM-like training algorithm for second-order recurrent neural networks. Neural Networks, 25, 77–83.CrossRefGoogle ScholarPubMed
Morford, J. P., & Carlson, M. L. (2011). Sign perception and recognition in non-native signers of ASL. Language Learning and Development, 7, 149–168.Google Scholar
Munakata, Y. (2004). Computational cognitive neuroscience of early memory development. Developmental Review, 24, 133–153.Google Scholar
New, B. (2006). Lexique 3: Une nouvelle base de données lexicales. Actes de la Conférence Traitement Automatique des Langues Naturelles (TALN 2006), Louvain, Belgique. http://www.lexique.org (retrieved March 18, 2009).Google Scholar
Newport, E. L. (1988). Constraints on learning and their role in language acquisition: Studies of the acquisition of American Sign Language. Language Sciences, 10, 147–172.CrossRefGoogle Scholar
Newport, E. L. (1990). Maturational constraints on language learning. Cognitive Science, 14 (1), 11–28.Google Scholar
OʼReilly, R. C., & Frank, M. J. (2006). Making working memory work: A computational model of learning in the prefrontal cortex and nasal ganglia. Neural Computation, 18 (2), 283–328.Google Scholar
Paradis, M. (2009). Declarative and procedural determinants of second languages. Amsterdam: John Benjamins.Google Scholar
Petanjek, Z., Judaš, M., Kostović, I., & Uylings, H. B. (2008). Lifespan alterations of basal dendritic trees of pyramidal neurons in the human prefrontal cortex: A layer-specific pattern. Cerebral Cortex, 18, 915–929.CrossRefGoogle Scholar
Pulvermüller, F., & Schumann, J. H. (1994). Neurobiological mechanisms of language acquisition. Language Learning, 44 (4), 681–734.Google Scholar
Rohde, D. L. T., & Plaut, D. C. (1999). Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72, 67–109.Google Scholar
Sabourin, L., Stowe, L., & de Haan, G. (2006). Transfer effects in learning a second language grammatical gender system. Second Language Research, 22, 1–29.Google Scholar
Scherag, A., Demuth, L., Rösler, F., Neville, H. J., & Röder, B. (2004). The effects of late acquisition of L2 and the consequences of immigration on L1 for semantic and morphosyntactic language aspects. Cognition, 93, B97–108.Google Scholar
Schmid, H. (1994). Probabilistic part-of-speech tagging using decision trees. Proceedings of the International Conference on New Methods in Language Processing, 12, 44–49.Google Scholar
Seidenberg, M. S., & Zevin, J. D. (2006). Connectionist models in developmental cognitive neuroscience: Critical periods and the paradox of success. In Munakata, Y. & Johnson, M. (eds.), Attention & Performance XXI: Processes of Change in Brain and Cognitive Development, pp. 585–612. Oxford: Oxford University Press.Google Scholar
Sokolik, M. E., & Smith, M. E. (1992). Assignment of gender to French nouns in primary and secondary language: A connectionist model. Second Language Research, 8, 39–58.Google Scholar
Surridge, M. E. (1989). Le facteur sémantique dans l'attribution du genre aux inanimés en français. Canadian Journal of Linguistics/Revue Canadienne de Linguistique, 34, 19–44.Google Scholar
Surridge, M. E. (1993). Gender assignment in French: The hierarchy of rules and the chronology of acquisition. International Review of Applied Linguistics in Language Teaching (IRAL), 31, 77–95.Google Scholar
Surridge, M. E. (1995). Le ou la? The gender of French nouns. Philadelphia, PA: Multilingual Matters.Google Scholar
Teschner, R. V., & Russell, W. M. (1984). The gender patterns of Spanish nouns: An inverse dictionary-based analysis. Hispanic Linguistics, 1, 115–132.Google Scholar
Thomas, M. S. C. (2009). Competition as a mechanism for producing sensitive periods in connectionist models of development. In Mayor, J., Ruh, N. & Plunkett, K. (eds.), Progress in Neural Processing 18: Proceedings of the Eleventh Neural Computation and Psychology Workshop, pp. 349–360. Singapore: World Scientific.Google Scholar
Thomas, M. S. C., & Johnson, M. H. (2008). New advances in understanding sensitive periods in brain development. Current Directions in Psychological Science, 17 (1), 1–5.CrossRefGoogle Scholar
Trofimovich, P., & Baker, W. (2006). Learning second language suprasegmentals: Effect of L2 experience on prosody and fluency characteristics of L2 speech. Studies in Second Language Acquisition, 28 (1), 1–30.Google Scholar
Ullman, M. T. (2004). Contributions of memory circuits to language: The declarative/procedural model. Cognition, 92, 231–270.Google Scholar
Uylings, H. B. M. (2006). Development of the human cortex and the concept of “critical” or “sensitive” periods. Language Learning, 56, 59–90.Google Scholar
Vogel, E. K., McCollough, A. W., & Machizawa, M. G. (2005). Neural measures reveal individual differences in controlling access to working memory. Nature, 438, 500–503.Google Scholar
Wikipedia: The free encyclopedia. (2011). FL: Wikimedia Foundation, Inc. http://www.wikipedia.org (retrieved January 15, 2011).Google Scholar
Wilson, M., & Emmorey, K. (1997). A visuospatial “phonological loop” in working memory: Evidence from American Sign Language. Memory and Cognition, 25, 313–320.CrossRefGoogle ScholarPubMed
Xie, X., & Seung, H. S. (2003). Equivalence of backpropagation and contrastive Hebbian learning in a layered network. Neural Computation, 15 (2), 441–454.Google Scholar
Zhao, X., & Li, P. (2010). Bilingual lexical interactions in an unsupervised neural network model. International Journal of Bilingual Education and Bilingualism, 13, 505–524.Google Scholar