Hostname: page-component-78c5997874-g7gxr Total loading time: 0 Render date: 2024-11-15T02:24:11.842Z Has data issue: false hasContentIssue false
  • English
  • Français

ESTABLISHING EVIDENCE OF LEARNING IN EXPERIMENTS EMPLOYING ARTIFICIAL LINGUISTIC SYSTEMS

Published online by Cambridge University Press:  09 February 2017

Phillip Hamrick  and
Rebecca Sachs
Phillip Hamrick*
Affiliation:
Kent State University
Rebecca Sachs
Affiliation:
Virginia International University
*
*Correspondence concerning this article should be addressed to Phillip Hamrick, Language and Cognition Research Laboratory, Department of English, Kent State University, 475 Janik Drive, Kent, Ohio 44242. E-mail: phamric1@kent.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Artificial linguistic systems (ALSs) offer many potential benefits for second language acquisition (SLA) research. Nonetheless, their use in experiments with posttest-only designs can give rise to internal validity problems depending on the baseline that is employed to establish evidence of learning. Researchers in this area often compare experimental groups’ performance against (a) statistical chance, (b) untrained control groups’ performance, and/or (c) trained control groups’ performance. However, each of these methods can involve unwarranted tacit assumptions, limitations, and challenges from a variety of sources (e.g., preexisting perceptual biases, participants’ fabrication of rules, knowledge gained during the test), any of which might produce systematic response patterns that overlap with the linguistic target even in the absence of learning during training. After illustrating these challenges, we offer some brief recommendations regarding how triangulation and more sophisticated statistical approaches may help researchers to draw more appropriate conclusions going forward.

Type
Research Article
Information
Studies in Second Language Acquisition , Volume 40 , Issue 1 , March 2018 , pp. 153 - 169
Copyright
Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We would like to thank Ronald Leow, Alison Mackey, Nick B. Pandža, Kelli Ryan, and the reviewers for their helpful comments on earlier versions of this paper. We are also grateful to Patrick Rebuschat for his valuable guidance on our early forays into implicit learning research, and for collaborations with him that have positively influenced our work in numerous productive ways. All remaining errors are our own. Both authors contributed equally to this paper.

References

REFERENCES

Baayen, R., Davidson, D., & Bates, D. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.CrossRefGoogle Scholar
Cunnings, I. (2012). An overview of mixed-effects statistical models for second language researchers. Second Language Research, 28, 369–382.CrossRefGoogle Scholar
de Graaff, R. (1997). The eXperanto experiment. Studies in Second Language Acquisition, 19, 249–276.CrossRefGoogle Scholar
Dienes, Z., & Altmann, G. T. A. (2003). Measuring learning using an untrained control group: Comment on R. Reber and P. Perruchet. The Quarterly Journal of Experimental Psychology, 56, 117–123.CrossRefGoogle Scholar
Endress, A. D., Carden, S., Versace, E., & Hauser, M. D. (2010). The apes’ edge: Positional learning in chimpanzees and humans. Animal Cognition, 13, 483–495.CrossRefGoogle ScholarPubMed
Ettlinger, M., Morgan-Short, K., Faretta-Stutenberg, M., & Wong, P. C. M. (2015). The relationship between artificial and second language learning. Cognitive Science, 40, 822–847.CrossRefGoogle ScholarPubMed
Faretta-Stutenberg, M., & Morgan-Short, K. (2011). Learning without awareness reconsidered: A replication of Williams (2005). In Granena, G., Koeth, J., Lee-Ellis, S., Lukyanchenko, A., Prieto Botana, G., & Rhoades, E. (Eds.), Selected proceedings of the 2010 Second Language Research Forum: Reconsidering SLA research, dimensions, and directions (pp. 18–28). Somerville, MA: Cascadilla Proceedings Project.Google Scholar
Field, A. (2009). Discovering statistics using SPSS. London, UK: Sage.Google Scholar
Grey, S., Williams, J. N., & Rebuschat, P. (2014). Incidental exposure and L3 learning of morphosyntax. Studies in Second Language Acquisition, 36, 611–645.CrossRefGoogle Scholar
Hama, M., & Leow, R. (2010). Learning without awareness revisited: Extending Williams (2005). Studies in Second Language Acquisition, 32, 465–491.CrossRefGoogle Scholar
Hamrick, P. (2012). Associative chunk learning supports early phases of adult L2 syntactic development. Poster presented at the Second Language Research Forum, Pittsburgh, PA.Google Scholar
Hamrick, P. (2013). Development of conscious knowledge during early incidental learning of L2 syntax (Unpublished doctoral dissertation). Georgetown University, Washington, DC. Retrieved from ProQuest Dissertations and Theses database (3558525).Google Scholar
Hamrick, P. (2014a). A role for chunk formation in statistical learning of second language syntax. Language Learning, 64, 247–278.CrossRefGoogle Scholar
Hamrick, P. (2014b). Recognition memory for novel syntactic structures. Canadian Journal of Experimental Psychology, 68, 2–7.CrossRefGoogle Scholar
Hamrick, P. (2015). Declarative and procedural memory abilities as individual differences in incidental language learning. Learning and Individual Differences, 44, 9–15.CrossRefGoogle Scholar
Hamrick, P., & Rebuschat, P. (2012). How implicit is statistical learning? In Rebuschat, P. & Williams, J. N. (Eds.), Statistical learning and language acquisition. Berlin: Mouton de Gruyter.Google Scholar
Hamrick, P., & Rebuschat, P. (2014). Frequency effects, learning conditions, and the development of implicit and explicit lexical knowledge. In Connor-Linton, J. & Amoroso, L. (Eds.), Measured language: Quantitative approaches to acquisition, assessment, processing and variation (pp. 125–139). Washington, DC: Georgetown University Press.Google Scholar
Knowlton, B. J., & Squire, L. R. (1996). Artificial grammar learning depends on implicit acquisition of both abstract and exemplar-specific information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 169–181.Google ScholarPubMed
MacWhinney, B., Bates, E., & Kliegl, R. (1984). Cue validity and sentence interpretation in English, German, and Italian. Journal of Verbal Learning and Verbal Behavior, 23, 127–150.CrossRefGoogle Scholar
Morgan-Short, K., Steinhauer, K., Sanz, C., & Ullman, M. T. (2012). Explicit and implicit second language training differentially affect the achievement of native-like brain activation patterns. Journal of Cognitive Neuroscience, 24, 933–947.CrossRefGoogle ScholarPubMed
Onnis, L., & Thiessen, E. (2013). Language experience changes subsequent learning. Cognition, 126, 268–284.CrossRefGoogle ScholarPubMed
Onnis, L., Waterfall, H., & Edelman, S. (2008). Learn locally, act globally: Learning language with variation set cues. Cognition, 109, 423–430.CrossRefGoogle ScholarPubMed
Perruchet, P., & Reber, R. (2003). Why untrained control groups provide invalid baselines: A reply to Dienes and Altmann. The Quarterly Journal of Experimental Psychology, 56, 125–130.CrossRefGoogle Scholar
Plonsky, L. (2015). Demographics in SLA: A systematic review of sampling practices in L2 research. Paper presented at the Second Language Research Forum, Atlanta, GA.Google Scholar
Reber, A. (1967). Implicit learning of artificial grammars. Journal of Verbal Learning and Verbal Behavior, 6, 855–863.CrossRefGoogle Scholar
Reber, R., & Perruchet, P. (2003). The use of control groups in artificial grammar learning. The Quarterly Journal of Experimental Psychology, 56, 97–115.CrossRefGoogle ScholarPubMed
Rebuschat, P. (2008). Implicit learning of natural language syntax (Unpublished doctoral dissertation). University of Cambridge, Cambridge, UK.Google Scholar
Rebuschat, P., & Williams, J. N. (2012). Implicit and explicit knowledge in second language acquisition. Applied Psycholinguistics, 33, 829–856.CrossRefGoogle Scholar
Rebuschat, P., Hamrick, P., Riestenberg, K., Sachs, R., & Ziegler, N. (2015). Triangulating measures of awareness: A contribution to the debate on learning without awareness. Studies in Second Language Acquisition, 37, 299–334.CrossRefGoogle Scholar
Rebuschat, P., Hamrick, P., Sachs, R., Riestenberg, K., & Ziegler, N. (2013). Implicit and explicit knowledge of form-meaning connections: Evidence from subjective measures of awareness. In Bergsleithner, J. M., Frota, S. N., & Yoshioka, J. K. (Eds.), Noticing and second language acquisition: Studies in honor of Richard Schmidt (pp. 255–275). Honolulu: University of Hawai’i at Manoa, NFLRC.Google Scholar
Redington, M., & Chater, N. (1996). Transfer in artificial grammar learning: A reevaluation. Journal of Experimental Psychology: General, 125, 123–138.CrossRefGoogle Scholar
Rogers, J., Révész, A., & Rebuschat, P. (2015). Challenges in implicit learning research: Validating a novel artificial language. In Rebuschat, P. (Ed.), Implicit and explicit learning of languages (pp. 275–300). Philadelphia, PA: John Benjamins.CrossRefGoogle Scholar
Sachs, R., & Weger, H. D. (2011). Motivation and effort in the comparison group: A factor in studies of L2 feedback? Paper presented at the Second Language Research Forum, Ames, IA.Google Scholar
Stafford, C. A., Bowden, H. W., & Sanz, C. (2012). Optimizing language instruction: Matters of explicitness, practice, and cue learning. Language Learning, 62, 741–768.CrossRefGoogle Scholar
Tagarelli, K. M., Borges Mota, M., & Rebuschat, P. (2015). Working memory, learning context, and the acquisition of L2 syntax. In Zhisheng, W., Borges Mota, M., & McNeill, A. (Eds.), Working memory in second language acquisition and processing: Theory, research, and commentary. Bristol, UK: Multilingual Matters.Google Scholar
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634.CrossRefGoogle ScholarPubMed
Whitson, J. A., & Galinsky, A. D. (2009). Lacking control increases illusory pattern perception. Science, 322, 115–117.CrossRefGoogle Scholar
Williams, J. N. (2005). Learning without awareness. Studies in Second Language Acquisition, 27, 269–304.CrossRefGoogle Scholar
Williams, J. N., & Kuribara, C. (2008). Comparing a nativist and emergentist approach to the initial stage of SLA: An investigation of Japanese scrambling. Lingua, 118, 522–553.CrossRefGoogle Scholar