Replication research in pedagogical approaches to formulaic sequences: Jones & Haywood (2004) and Alali & Schmitt (2012)

Abstract

Research into the formulaic nature of language has grown in size and scale in the last 20 years or more, much of it based in corpus studies and involving the identification and categorisation of formulas. Research suggests that there are benefits for second and foreign language learners recognising formulaic sequences when listening and reading, and using them in speaking and writing. Very few studies, however, actually focus on formulaic sequences and teaching methodology and materials design in second and foreign language learning. This paper presents a brief background to researching, teaching and learning formulaic sequences, and considers the case for replication research in this area. The third part focuses on two original studies (Jones & Hayward 2004; Alali & Schmitt 2012) and makes suggestions for possible replication studies.

1. Introduction

Interest in formulaic language and second and foreign language learning has grown rapidly in recent years as researchers investigate patterns of lexical items in texts, how these items are stored and then later retrieved from memory (see Wray 2008; Conklin & Schmitt 2012), and how the mind processes them (Siyanova-Chanturia, Conklin & Schmitt 2011; Siyanova-Chanturia, Conklin & van Heuven 2011). Estimates of the amount of formulaic language in English suggest that up to 50% of the language is formulaic in some way (Erman & Warren 2000; Conklin & Schmitt 2012). Early pedagogically oriented ‘calls-to-arms’ for a focus on formulaic sequences in language learning include Nattinger & DeCarrio (1992) and Lewis (1993), but also see Wray's (2013) research timeline for other areas of research in formulaic sequences, including theory, language disorders, first language acquisition, and culture.

Formulaic sequences appear to promise much in return for learning. Possible advantages of knowing formulaic sequences include gains in processing texts, but these gains are likely to be higher for more proficient learners because they have had more exposure to these features of language. Unlike native speakers of English, non-native speakers are more likely to process formulaic sequences one word at a time (Conklin & Schmitt 2012). Other benefits are that language learners who use them while speaking may be judged as more proficient in language use (see Boers et al. 2006; Stengers et al. 2010, 2011), more fluent (Pawley & Syder 1983), and be seen as belonging to groups who use the same language (Wray 2002).

Researchers have identified a gap in the field for pedagogically oriented formulaic language studies. Meunier (2012: 123) writes, ‘. . .though L2 teaching no longer ignores the formulaic nature of language, the exact paths to follow to better teach it are still insufficiently lit.’ Granger (1998) points to a lack of theoretical underpinnings for teaching and learning formulaic sequences, and Coxhead (2008) calls for research to support pedagogical approaches to phraseology in language learning, particularly taking into account learner beliefs. The two studies selected for replication in this paper are Jones & Haywood (2004) and Alali & Schmitt (2012), because they focus on multi-word units and pedagogy.

The next section in this paper looks into the background of research into formulaic sequences in more depth. Section 3 presents the aims, methodologies and findings of the original studies, and includes suggestions for possible approaches and rationales for replication. Section 4 concludes the paper.

2. Background

A variety of multi-word units have been investigated in English with L2 learners in mind. A common area of investigation is two-word collocations such as bottom-up, cost-cutting and cure-all from Crawford Camiciottoli's 2007 study of collocations in business, or Durrant's (2009) initial study of academic collocations in a 25-million-word corpus, comprising five academic disciplines. Frames with slots are another area of study, as in the highly frequent frame the XX of, where the frame remains but the slot word can change, for example, the concept of, the beginning of and the effect of. Academic formulas and phrases (for example, as a result of) have also gained attention, with a range of studies considering the frequency and distribution of various lexical bundles in a variety of mostly written academic corpora (Biber, Conrad & Cortes 2004; Cortes 2004; Hyland 2008; Byrd & Coxhead 2010). These studies show that, just like single word forms, some formulaic sequences are more frequent than others and their use is driven by context.

Idioms also come under the umbrella of multi-word units, for example, as good as gold. See Grant & Bauer (2005) for more on defining idioms and Simpson & Mendis (2003) for an interesting account of idioms in spoken academic English from the Michigan Corpus of Academic Spoken English (MICASE) corpus. Note that a lot of the research on multi-word units sits within the fields of English for academic purposes (EAP) or English for specific purposes (ESP). Such research often results in lists of multi-word units for learning and teaching, including recent work such as Liu's (2012) list of frequency-based ‘multi-word constructions’ derived from a written academic corpus; Ackermann & Chen's (2013) Academic Collocations List; and Simpson-Vlach & Ellis's Academic Formulas List (2010). Despite the growing interest and activity in EAP, L. Flowerdew (2015: 104) points out that ‘there are few EAP-oriented studies that go beyond simple frequency counts and also consider learnability and teachability’ when it comes to phraseology. This gap is evident not only in EAP.

Wray (2000: 466–467) identifies a number of problems when researching formulaic sequences, including: working out the proportion of formulaic language in a language; how we might distinguish between newly formed language and formulaic sequences; and difficulties in finding methods to ‘identify formulaicity in a consistent and principled way’. Formulaic sequences pose a number of challenges for teachers and learners in the classroom. Byrd & Coxhead (2010) discuss some of these difficulties, including, for example; the need for contextual information on multi-word units from published lists; how to work with such lists in a meaningful way in class; and frequent encounters with target items in classroom texts and environments. Simpson & Mendis (2003: 436) note that truncation of idioms (haven't the foggiest as a short form for haven't the foggiest idea) and variation in speaker performances (where speakers introduce some variation into an idiom) can be problematic for second language (L2) learners.

Another area of enquiry considers possible pedagogical approaches for word strings, to identify the most effective ways of learning and teaching these items in language classrooms. Boers et al. (2014) contrasted types of commonly used textbook-style exercises, such as odd-man-out and fill-in-the-blanks, where L2 learners were asked to use whole word strings (holistic) or single words (analytic). The researchers found slightly better learning effects from the holistic exercises over the analytic exercises, as well as possible interference between items. Other examples of this research can be found in a thorough discussion of L2 intervention and experimental studies in Boers & Lindstromberg (2012). These researchers summarise a range of intervention studies including investigating the effect of alliteration and assonance on learning word strings (Boers & Lindstromberg 2005, 2008). Simpson & Mendis (2003) recommend several activities for teaching idioms, including discourse analysis and corpus-based methodologies. These studies tend to show that, at the very least, interventions can raise learners' awareness of formulaic language.

Very few replication studies have been carried out in the field of formulaic sequences, yet replication is important for establishing the generalisability and reliability of the findings. Reasons for this lack of published research could be: difficulties with arriving at a definition of ‘formulaic sequences’; previous research involving learners of different first languages and levels of proficiency; and published studies perhaps not including enough methodological information to make replication a possibility. Let us now look at the two aforementioned studies chosen for this paper, which investigate approaches to learning and teaching formulaic sequences.

3. The original studies and suggested approach to replication

3.1 Jones & Haywood (2004)

This study was published in a volume entitled Formulaic sequences: Acquisition, processing, and use (Schmitt 2004). Unlike all but one other study in this volume, Jones & Haywood's work is pedagogically driven. An early attempt at investigating the teaching and learning of formulaic sequences, this research took place in real EAP classrooms with learners and their teacher/researcher. The classes were not discipline-specific but focused on general EAP, which strongly reflects the continued reality of many EAP classrooms and teaching. The study continues to be cited in literature in the areas of formulaic sequences and vocabulary studies, EAP, and pedagogy (for example, L. Flowerdew 2015), and is seen as some evidence of awareness-raising in language teaching (see Boers & Lindstromberg (2012)).

Jones & Haywood (2004) is an ‘exploratory’ study (p. 269) using an intervention framework for learning and teaching formulaic sequences in an EAP context in a UK university. The study took place over ten weeks. The participants had a variety of first languages and different purposes for attending the EAP class. The 21 participants were split into one group of 10, which received training in formulaic sequences, and a control group of 11. The authors initially surveyed three EAP course books on academic writing for potential target lexical bundles, but in the end decided to select the 80 lexical bundles for their study (including, for example, the relationship between, there were no significant differences and studies have shown that) from Biber et al.'s (1999) research.

Jones & Haywood (2004: 276–278) used a variety of teaching methods to focus on the target formulaic sequences, including activities such as drawing attention to the words through highlighting, encouraging memorisation and the productive use of the sequences in writing; concordancing, reviewing and analysing the sequences in context; and completing gap-fill exercises. Pre- and post-tests were used in this study to measure the participants' production of target items in a C-test (Jones & Haywood 2004: 279). Note that the target items in the pre-test are not the same as the target items in the post-test. Two writing activities for the study were used to assess productive use of the target items. The first writing activity was an essay with support from teachers and input texts, and the second required the learners to write using their own ideas. Three students were observed daily in class and interviewed in week two of the programme by the researcher/teachers for half an hour.

Overall, the researchers found that students in their study became more aware of formulaic sequences, but did not appear to learn the sequences very well or use them in their written output. The researchers posited that the learners had not memorised the target items well, or had concentrated on items they knew well or which were salient in the sequences (p. 289). Clearly, the timeframe of this study also posed difficulties, particularly with the free production aspect. Boers & Lindstromberg (2012: 90) suggest that the exposure and practice in the study might not have been enough to lay down a long-lasting memory trace (p. 89). These authors recommend more intentional learning of formulaic sequences to make them more memorable for learners.

3.1.1 Approach to replication

A replication study would be of interest to teachers and learners, as well as researchers in vocabulary studies and applied linguistics. The original study is a small-scale investigation of a fairly small sample of ten learners. One avenue for an approximate replication would be to keep the ‘real’ context but involve a larger number of classes, thereby increasing the generalisability of the conclusions. In the original study, the teacher was also the researcher. A small change might be to put in place a researcher who is not the classroom teacher, which at the very least would lessen the load on the teacher and create some space between the participants and the researcher. In this scenario, the classroom teachers might require careful training on the pedagogy under investigation, for ecological validity purposes.

Because time was a factor in the original study, perhaps following a group of learners through several EAP courses would allow for greater exposure and building better memory traces for the formulaic sequences, and increase the potential and opportunity for productive use. The original study included pre- and post-testing, but a replication should ensure that the same items are in the pre- and post-tests because doing so would add to the validity and robustness of the study. The C-test-prompted production test could be supplemented by a series of more targeted tests, such as the range of tests used in Alali & Schmitt (2012), including form recall, meaning recall, form recognition and meaning recognition, and adapted for the first language (L1) of the participants where possible, to add robustness to the research design and help tease out the different aspects of form-meaning knowledge which might be gained by the participants. A control group would also add strength to the research design of a replication study.

The Jones & Haywood (2004) study took place at a UK university with participants with different L1s; a similar scenario could be found in other English as a second language (ESL) countries such as Aotearoa/New Zealand. Replication studies could work with similar populations and research sites in ESL contexts to provide consistency between studies. Studies undertaken in English as a foreign language (EFL) environments would also be interesting for replication, as the original findings could be compared with results from learners who share the same L1 and have a similar English language education background, but whose exposure to formulaic sequences in everyday situations and opportunities to practise in English are more limited.

As mentioned above, Jones & Haywood (2004) used a variety of commonly-used classroom-based strategies such as raising awareness with formulaic sequences as their focus. Devising a conceptual replication might involve teasing apart the complex design and simplifying the study. The purpose of simplification would be to investigate and compare what effect a particular classroom activity (rather than a raft of activities), such as concordancing or analysing the sequences in context, has on the retention of the formulaic sequences in students' memories or on the productivity of the sequences in their writing. This approach would require very careful management to ensure that various classes and activities are as comparable as possible. Selecting items from the course readings for the EAP class might also ensure that the learners would meet formulaic sequences in their coursework and learn them.

The Jones & Haywood (2004) study included interviews with three participants in the second week, focusing on the participants' views on their writing. A conceptual replication could keep and expand on this qualitative data, perhaps through increasing the number of interviews with participants, introducing pre- and post-interviews, and extending these interviews to find out more about how the learners approached their learning of formulaic sequences. The interviews could also focus on the factors affecting the participants' lexical decision making and learning, including, for example: aspects such as cultural background; writing topic; knowledge of a vocabulary item; perceived demands of the academic audience and context; demands of the learning task; and individual learning beliefs (see Coxhead 2011 for more on this area). Finally, using think-aloud protocols to capture during-writing data is another potential method to introduce into a conceptual replication to increase our understanding of how participants approach the learning tasks.

3.2 Alali & Schmitt (2012)

Alali & Schmitt (2012) was an intervention study on explicit teaching and revision of single words and idioms. Although it is a relatively recent study, citations are appearing in publications on formulaic language learning, including Bardovi-Harlig, Mossman & Vellenga (2015) who note the explicit approach of Alali & Schmitt (2012) in the presentation of the sequences to the learners in the study (Bardovi-Harlig et al. 2015: 325) and outline the testing regime (p. 327). Alali & Schmitt (2012) is one of very few studies to report on direct teaching methodologies for formulaic sequences. The authors highlight the importance of reviewing for learning vocabulary, and present research from an under-researched area of the world. It is also important to note that the research participants are young female learners of English (12–13 years old), thereby adding to much needed empirical study of vocabulary learning by young learners in foreign or L2 learning.

The participants in this study were 35 Kuwaiti 12–13-year-old female students in an international girls' school. All participants were L1 speakers of Arabic and formed an intact classroom group for the study. The study had four research questions, beginning with whether formulaic sequences and single words could be taught using the same methodologies, and then focusing on types of review (oral vs. written) of the sequences, and types of knowledge developed through the intervention study (recall/recognition). Idioms for the study were selected from The Free Dictionary (http://idioms.thefreedictionary.com/), a resource which is based on the Cambridge International Dictionary of Idioms and the Cambridge Dictionary of American Idioms. The single-word items for the study were taken from unknown words in the idioms. For example, dwell on the past and ivory tower were target idioms, so dwell and ivory were used as single-word items in the study (Alali & Schmitt 2012: 178).

The various tests in this study were translation-based. Alali & Schmitt (2012) adapted Laufer & Goldstein's (2004) Computer Adaptive Test of Size and Strength (CATSS) for their study. Alali & Schmitt's comprehensive test contained four sections: form recall, meaning recall, form recognition and meaning recognition (p. 160). The purpose of the tests was to measure initial learning of the L2 form and meaning, both receptively and productively. The first section of the idiom test, for example, provides the translation of the target item and the test taker provides the L2 idiom (form recall). The second section provides the L2 idiom and the test taker provides the L1 meaning of the idiom (meaning recall). In the third section, the test taker is given the idiom in the L1 and chooses between four options in the L2 (form recognition), while in the final section, the idiom is given in the L2 and the options are in the L1 (meaning recognition). Pre-testing showed the target items were not known by the participants. Post-testing took place at the end of the intervention and delayed testing took place 12 days later. The researchers ensured that the test items were randomised, and the tests were carried out in four steps above, with each section being completed separately.

The intervention study was carried out over 12 one-hour classroom sessions. The first six sessions were used for teaching and the other six were used for post-testing. The first three sessions were used for direct teaching using translation of target single items and the second three sessions were used for direct teaching of idioms. Each teaching segment took ten minutes, with one minute per item, followed by a distractor task. The next segment in the teaching procedure contained three different review activities. In the first class, there was no review time but students were given ten extra minutes for grammar teaching. In the second class, there was a verbal repetition activity in which the students chorally repeated the target items. Oral repetition was used because of the educational and cultural context of the study. Students in the third class completed written reviews of the target words. After the review section, the students were then given distractor tasks, followed by an immediate post-test. A delayed test took place 12 days later (using the same tests). The participants were not told that they would be sitting tests after the treatment; however, the authors point out that the pattern of ‘teach then test’ may well have been noticed by the participants part-way into the study (p. 173).

The study found that single words had better learning gains than idioms for these participants and that recognition of form and meaning rather than recall showed better gains. Further, the written review group scored better on the post-tests than the no review or oral review group. The researchers conclude that the teaching methodology in their study ‘produced a similar pattern of learning for one type of formulaic language, idioms, as for single words’ (Alali & Schmitt 2012: 174) and that repetition, as found in many other vocabulary studies, has a strong effect on learning. The researchers also note that the selection of target single words from idioms was problematic in their study.

3.2.1 Approach to replication

Replicating the study including the range of tests in the original study in the same context using several classes would be a useful starting point. A replication study in the same context with a larger number of participants could increase the robustness and generalisability of the original study. Another obvious suggestion for replication is to include boys as the participants, to compare and contrast gender as a variable as other vocabulary studies have done (see Coxhead, Nation & Sim 2015, for example). Another avenue for replication would be to focus on different populations, for example, speakers of other L1s, such as Mandarin, Vietnamese or Spanish. Different learning contexts might produce interesting comparisons and contrasts to the original study. For example, would speakers of other languages show the same results in terms of recognition and recall as the original study? Could the results of the original study be interpreted in a different way if the order of the review was counterbalanced in a replication study? Older learners or higher proficiency learners might have greater awareness of aspects of language such as idioms, through having more exposure to either idioms in their L1 or more years of experience in language learning and therefore exposure in the L2. Results from older learners might tell us more about the nature of oral recall and vocabulary learning and the effects of age. We might also be able to revisit the results from the original study to consider the effects of fairly strictly controlled direct teaching of vocabulary.

Since the written review was shown to have a better retention of the target language, a conceptual approach to replication could be to compare and contrast learning through different kinds of writing activities. For example, one group could use the gap-fill worksheet from Alali & Schmitt (2012), a second group could complete an information transfer activity, while a third group could try to use the target items in their own writing. In an information-transfer activity, learners change the form of the information in a text into another form, for example, from a piece of text into a table, a mind-map or a chart. This activity tries to encourage deep comprehension through encoding. A comparative study of writing activities would be useful for language teachers, learners, materials designers and researchers, and could perhaps shed light on whether the written review in the original study showed better learning because of the nature of writing as a more engaging learning activity or whether the kind of writing has an impact on learning single words and idioms.

Many teachers around the world use choral repetition in language learning classrooms. Alali & Schmitt's (2012) study provides a useful insight into this technique. The choral repetition element of the study could be investigated in a replication study by including individual as well as group choral repetition to form the basis of a comparative study. Could it be that the effectiveness of repeating a single word or idiom is determined by the manner and size of the group doing the activity? Related to this is the suggestion that classroom-based revision activities include some element of competition between learners (see examples in Part II: Consolidating vocabulary learning in Coxhead 2014). Therefore, introducing a competitive element to choral repetition activities in class might help make the idioms used more memorable, include more of a focus on meaning and form-meaning link, and provide insight into the effectiveness of other kinds of classroom activities which are commonly used by language teachers. In the original study, direct teaching and review activities are used, but with no element of motivation to push the learners' performance, beyond perhaps the potential to do well on the post-tests.

Alali & Schmitt (2012: 172) point out that the mode of testing in the study was written, but the input was oral. While any replication study should keep the range of tests in the original study, including the delayed test used by Alali & Schmitt (2012: 163) in their analysis, an approximate replication study could introduce an oral testing element to match the oral input. This change would add an element of comparison between written input + written testing and oral input + oral testing, and effectively round out the data and testing set in the original study. In that case, there could be new findings on vocabulary learning from oral repetition of idioms and single words. Alali & Schmitt (p. 174) find that the participants scored higher on the recognition than the recall tests, and suggest that teachers need to consider the classroom methodology they use to target particular kinds of lexical knowledge. Therefore, classroom activities which focus on developing understanding of the meaning of idioms and connecting the form and meaning of idioms should be included in a conceptual replication study.

A conceptual replication study could introduce a qualitative aspect, such as interviews or focus groups, to find out more about how the participants approached learning the idioms and their opinions of the teaching and revision techniques. It would be useful to know more about how learners themselves use choral repetition in their learning or what they focus on during these kinds of activities. This kind of replication would provide opportunities for triangulation as well as allowing for an evaluation of the results in another way.

Finally, the idiom focus could be replaced by higher frequency items such as lexical bundles or common collocation patterns such as those identified by Shin & Nation (2008) in spoken language or Liu's (2012) multi-word constructions in academic written language, depending on the focus of the classrooms and learners in the studies. With idioms being less prevalent in academic texts (Liu 2012), perhaps a change of focus from idioms to other word strings might be useful. The purpose of this replication would be to address the needs of particular learners in the research design, depending on the likelihood of exposure to those items in their learning as well as whether these items would be needed for the learner's ongoing language learning needs.

4. Conclusion

This paper has suggested two small-scale, classroom-based studies for replication. These studies look at two different types of formulaic sequences: lexical bundles and idioms. Replication studies would offer further insights into pedagogy and formulaic sequences, and support validation and generalisability. This research would be of interest to ESP/EAP teachers, materials designers, course and curriculum designers, and language learners initially, and to a wider audience of language learning in English as well.