The role of bilingualism in paired-associate and cross-situational word learning

Abstract

In adulthood, novel words are commonly encountered in the context of sequential language learning, and to a lesser extent, when learning a new word in one's native language. Paired-associate (PAL) and cross-situational word learning (CSWL) paradigms have been studied separately, under distinct theoretical umbrellas, limiting the understanding of the mechanisms underlying the learning process in each. We tested 126 monolinguals and 111 bilinguals on PAL and CSWL, manipulating familiarity and measuring verbal working memory. Results revealed highly similar learning performance across groups, both demonstrating better performance in PAL than in CSWL, similar sensitivity to familiarity, and similar reliance on phonological working memory. We observed a trend such that bilinguals outperformed monolinguals in PAL but not in CSWL, but this trend was weak. Findings indicate limited effects of bilingualism on word learning in adulthood and suggest highly similar word learning mechanisms in learners with different linguistic experiences.

1. Introduction

Although the childhood period has been the dominant focus in the literature on word learning, adults also learn novel words every day. In adulthood, word learning can be a conscious and effortful process, as when learning vocabulary lists in a foreign-language classroom, or it can happen implicitly (as during childhood) – for example, when inferring meanings of new words through exposure. While there is an extensive psycholinguistic literature on how words are learned, the examinations into the different ways learning can occur have to date been siloed from each other.

In the laboratory, ostensive learning of novel words where learning is explicit and unambiguous has most frequently been studied using Paired-Associate Learning (PAL) tasks. Implicit learning that is meant to resemble ecologically-valid immersion-based learning in ambiguous settings has been studied using cross-situational word learning (CSWL) tasks. PAL and CSWL tasks have never been compared, limiting the understanding of word learning mechanisms in adulthood. The goals of the present study were two-fold. First, we aimed to contrast PAL and CSWL and examine whether word learning paradigm would affect learning performance in adults. Second, we aimed to examine the extent to which language background (i.e., bilingualism), word familiarity and verbal working memory would affect learning accuracy across paradigms. Ultimately, this approach can yield a more comprehensive framework for studying word learning in adulthood, and can reveal similarities and potential differences in the mechanisms by which words are learned.

1.1. PAL vs. CSWL

In paired-associate word learning, there is no ambiguity as to the association between the word to be learned and its referent (e.g., a translation or an object). In contrast, in cross-situational word learning, more than one word and one object are presented in any given trial. While the outcome of both paradigms is a phonological representation of the novel word stored in long-term memory (e.g., Litt et al., 2019; Vlach & Sandhofer, 2014), the learning process in each setting differs.

The PAL paradigm was developed under the theoretical umbrella of Baddeley's working memory model, where verbal information is encoded into a dedicated system called the phonological loop (Baddeley, 2003; Baddeley & Hitch, 1974). The literature in PAL suggests a word learning advantage for bilinguals compared to monolinguals (Antoniou et al., 2015; Kaushanskaya, 2012; Kaushanskaya & Marian, 2009b; Nair et al., 2016; Papagno & Vallar, 1995; but see Tsuboi & Francis, 2020), possibly due to enhanced phonological working memory (Kaushanskaya, 2012; Kaushanskaya & Marian, 2009b; Papagno & Vallar, 1995). Separately, in monolingual research, phonological working memory has been found to support PAL, with phonological working memory interacting with long-term memory to scaffold learning of lexically familiar, but not unfamiliar, novel words (Ellis & Beaton, 1993; Papagno et al., 1991; Papagno & Vallar, 1992; Service & Craik, 1993).

In contrast, CSWL has developed from statistical learning approaches (e.g., Aslin, 2017; Saffran et al., 1996), and thus the CSWL literature has not devoted the same amount of attention to the role of phonological working memory in the learning process. However, findings examining CSWL in monolinguals and bilinguals suggest that perceiving fine phonological detail affects learning (Escudero et al., 2013, 2016b; Mulak et al., 2019). Additionally, studies involving bilinguals suggest a bilingual advantage over monolinguals in CSWL, but only when phonological competition is present in the stimuli (Benitez et al., 2016; Escudero et al., 2016a; Poepsel & Weiss, 2016). In PAL, one of the loci of this advantage has been hypothesized to be enhanced phonological working memory capacity (Kaushanskaya & Marian, 2009b). In contrast, in statistical learning, this bilingual advantage has been attributed to bilinguals’ enhanced executive functioning skills (e.g., Bartolotti & Marian, 2012), and in CSWL specifically, to the need to resolve competition between two languages (Benitez et al., 2016; see Bogulski et al., 2018 for a review across word learning paradigms).

Taken together, the findings suggest that in PAL, phonological working memory plays a central role in successful learning, and that additionally, it might be the locus of a bilingual advantage in word learning. However, this has not yet been studied in CSWL, although one study found a correlation between auditory attention and CSWL performance in children (Vlach & DeBrock, 2019).

Monolinguals and bilinguals have not yet been tested in a single experiment, comparing PAL and CSWL, and the role of phonological working memory has not been examined across both paradigms. This limits the ability to comprehensively understand how bilingualism might affect novel word learning across learning contexts.

1.2. Bilingualism and PAL

Early evidence for the role of phonological working memory in supporting PAL in monolinguals (Baddeley et al., 1988; Papagno et al., 1991; Papagno & Vallar, 1992; Service, 1992) has prompted the question of whether phonological working memory also supports the acquisition of novel words by bi-/multilingual individuals (e.g., Papagno & Vallar, 1995; Kaushanskaya, 2012; Kaushanskaya & Marian, 2009a, 2009b; Tsuboi & Francis, 2020; Van Hell & Mahn, 1997).

A seminal study by Papagno and Vallar (1995) compared a group of bilinguals with a group of multilinguals (with knowledge of at least three languages) on a PAL paradigm with word-word and word-nonword associations to learn. Participants were additionally tested on two phonological working memory measures: a forward digit-span and a nonword repetition task, and measures of general intelligence, vocabulary knowledge in the native language, and visuo-spatial memory. Multilinguals performed significantly better than bilinguals on both phonological working memory tasks and learned the nonwords significantly better compared to bilinguals. A follow-up principal component analysis showed that both phonological memory tasks and nonword learning loaded on the same factor. This suggested, in addition to the lack of significant differences on the other measures administered, that multilinguals’ higher performance on nonword learning was operating through phonological working memory.

While not directly tested, the role of phonological working memory has been implicated in subsequent studies of paired associate word learning in bilingual and monolingual adults. For example, in a study contrasting novel word learning performance in English-speaking monolinguals, English–Spanish bilinguals and English–Mandarin bilinguals, both bilingual groups outperformed the monolingual group on novel word learning (Kaushanskaya & Marian, 2009b). The bilingual groups had acquired their two languages early in life, showing that this advantage was independent of second language learning strategies that could have been developed in the classroom for late bilinguals (as tested in Papagno & Vallar, 1995; Van Hell & Mahn, 1997). It was suggested that this advantage could stem from more efficient encoding of unfamiliar phonology, increased working memory storage (see also Papagno & Vallar, 1995), or that early exposure created a more tolerant phonological system. However, performance on a phonological memory measure (a digit-span task) in the three groups revealed no differences across monolinguals and both groups of bilinguals (Kaushanskaya & Marian, 2009b). Therefore, bilinguals’ better performance in this study could not be attributed to advantages linked to phonological working memory function.

The question of whether phonological working memory supports word learning in bilinguals was further examined in Kaushanskaya (2012), where monolinguals and bilinguals were tested on a digit-span and nonword repetition tasks (both have been widely recognized as phonological working memory measures) and matched by low-span/high-span across groups. Both lexically familiar and unfamiliar stimuli were included in the PAL task, as previous studies in monolingual research found a robust effect of familiarity on novel word learning (Ellis & Beaton, 1993; Papagno et al., 1991; Papagno & Vallar, 1992; Service & Craik, 1993). Bilinguals outperformed both low- and high-span groups of monolinguals, on both word types. These findings suggest a bilingual advantage in word learning, over and above phonological working memory capacity. Nonetheless, it is possible, among other factors, that the role of phonological working memory was masked because participants were tested through the retrieval of the English translations for the novel words, whose phonology was familiar to all participants.

Considering the variety of ways that bilingualism effects on paired associate word learning have been tested and the inconsistency in results, Tsuboi and Francis (2020) examined within one study the role of bilingualism, language dominance and language proficiency in PAL in English monolinguals, English–Spanish bilinguals with dominance in either English or Spanish, and Japanese–English bilinguals. They tested learning both auditorily and visually, and cued a response through an unfamiliar language for all participants (Japanese for English monolinguals and English–Spanish bilinguals, and Spanish for Japanese–English bilinguals), to ensure word referents were unfamiliar to all participants. Findings showed similar performance across monolinguals and bilinguals, and revealed that higher proficiency in the language through which the novel words are learned led to higher learning performance, independent of bilingual status.

To summarize, findings generally point to a bilingual advantage in novel word learning in PAL (Hirosh & Degani, 2018; Warmington et al., 2019; except in Tsuboi & Francis, 2020), but evidence for the association of this advantage with phonological working memory is inconsistent. The role of bilingualism in CSWL has also been tested, although much more sparsely than in PAL, and without focusing on phonological working memory as a possible locus of bilingual effects.

1.3. Bilingualism and CSWL

Contrary to PAL literature, which has made phonological working memory a central tenet of its posited mechanisms, CSWL emerged from statistical learning approaches. Learning in CSWL is theorized to take place either through associative learning, where information is aggregated over time to infer meaning (e.g., Yu & Smith, 2007), or through hypothesis testing, where evidence is gathered to formulate hypotheses, which then support inference-making on word-referent pairings (e.g., Yurovsky & Frank, 2015).

Only a few studies have examined the effect of bilingualism on CSWL, and generally, they indicate that bilinguals learn better in CSWL paradigms, but only in the presence of competition (Benitez et al., 2016; Poepsel & Weiss, 2016). In both studies, competition was created by pairing a referent (an object) with two words instead of one. In Benitez et al. (2016), the participant pool was divided into speakers of one language, and speakers of multiple languages. One of the two words to learn contained a systematic phonological property making it more distinctive than the other word. Results showed that this distinctiveness helped learning performance only for monolinguals, and bilinguals learned both labels in the two-word pairings at significantly higher levels than monolinguals. These findings suggest that one or more factors linked to dual-language learning may help learning novel words that have competing pairing options, such as a lower phonological bias towards one language, and/or variations in language processing ability.

Another study that examined competition in word-referent pairings tested monolinguals and bilinguals in three conditions varying in ambiguity, with 2 x 2 (two words are presented auditorily while two pictures are displayed visually), 3 x 3, and 4 x 4 presentations (Poepsel & Weiss, 2016). Monolingual and bilinguals’ performance did not differ across conditions. However, when introducing two-to-one mappings (two objects for one word) in a 3 x 3 design, bilinguals outperformed monolinguals in learning more of the two-to-one mappings, and converged faster on primacy (first label paired with an object) and recency (second label paired with the same object) pairings. These findings suggest that the bilingual participants were more flexible in their ability to contemplate two-to-one mappings compared to monolinguals.

While these two studies show that bilinguals may be better able to learn in the presence of competition in the input, the role of phonological working memory was not explored in Poepsel and Weiss (2016). It was only indirectly touched upon in Benitez et al. (2016), where sensitivity to phonological detail was tested by manipulating the phonological distinctiveness of one of the two words mapping onto one object. To target sensitivity to phonological overlap within stimuli, Escudero et al. (2016a) examined how Singaporean English–Mandarin bilinguals might differ from Australian English monolinguals in their processing of fine phonological detail when learning novel words in CSWL. The novel word stimuli conformed to native language phonotactics and were divided into non-minimal pairs, consonant minimal pairs or vowel minimal pairs. Moreover, one of the minimal pairs was formed by a vowel height contrast absent in Singaporean English. It was hypothesized that bilinguals would outperform monolinguals based on previous work suggesting a phonological working memory advantage for bilinguals, and that vowel minimal pairs would be the most difficult to learn for all groups. Moreover, bilinguals were expected to perform worst on the minimal pairs containing the contrast absent from their native language. While both groups learned above chance, bilinguals did outperform monolinguals, even on the contrast that was absent from the bilinguals’ inventory. This study however did not include a measure of phonological working memory, limiting the possibility of examining the source of the observed phonological similarity.

1.4. Cross-linguistic overlap and phonological memory

Models of bilingual language processing consistently show that a bilingual's two languages are activated non-selectively (e.g., Dijkstra & Van Heuven, 2002; Duyck, 2005; Jared & Kroll, 2001). One method to tap into dual-language activation is to examine how cross-linguistic overlap affects processing – for example, using cognates, interlingual homographs or homophones. With interlingual homophones, where two words sound the same but are semantically different, studies have found that depending on task demands, processing can be facilitated or hindered. For example, Liu and Wiener (2020) studied adult native speakers of English (L1), in their second semester of learning Mandarin Chinese (L2). Half of the novel words to be learned in the L2 were homophones with words previously learned in the L2, and half were novel words in the L2 that were not L2 homophones. The words were played auditorily and paired with an image representing the word. Participants were then tested in a four-alternative-forced-choice task to identify the novel words. Accuracy was higher on homophones than non-homophones, suggesting a facilitative effect of phonological and lexical information previously learned on building L2 vocabulary. A facilitative effect of homophones has also been found in priming tasks (Duyck, 2005), but not in tasks that require making a judgement about the lexical or phonological nature of the word, as in category-verification or gating (e.g., Friesen & Jared, 2012; Schulpen et al., 2003; Sperber et al., 1982). These findings suggest that for tasks that require a higher cognitive load on working memory, homophones may create a processing cost.

The role of lexical competition across languages has also been studied, manipulating the level of cross-linguistic overlap (Bartolotti & Marian, 2012; Blumenfeld & Marian, 2007; Ju & Luce, 2004; Kaushanskaya & Marian, 2007; Marian & Spivey, 2003; Mishra & Singh, 2014; Nakayama & Archibald, 2005; Weber & Cutler, 2004). When overlap is partial, competition for lexical selection arises, slowing down language processing. However, in language learning, bilinguals tend to resolve this competition quicker and with higher accuracy rates than monolinguals (Bartolotti & Marian, 2012).

In monolingual and bilingual research, when words are manipulated such that their phonotactics conform to the known language, either for familiar words or unfamiliar (nonwords), lexical familiarity has been found to benefit learning (Ellis & Beaton, 1993; Majerus et al., 2004; Papagno et al., 1991; Papagno & Vallar, 1992). This is also true when familiar words are nonwords conforming to the known language phonotactics, and the unfamiliar words contain phonemes absent from the known language (Kaushanskaya & Marian, 2008; Service & Craik, 1993). This lexical and phonological familiarity effect on learning is hypothesized to emerge from the involvement of known phonological representations stored in long-term memory in phonological working memory.

In CSWL, only two studies manipulated phonological overlap, and did so within novel stimuli rather than in terms of overlap between known words and novel words (Escudero et al., 2013, 2016b). In these studies (Escudero et al., 2013, 2016b), learning was worst on vowel minimal pairs. This suggests that in CSWL, the efficiency of the phonological loop in encoding fine phonological detail likely affects learning performance. Therefore, it is possible that when learning familiar words, a facilitation effect might be found, as in PAL. However, the CSWL paradigm is more ambiguous than PAL, due to presentation of at least two words and two referents at the same time in teaching trials. This could increase working memory load, negatively impacting word learning performance (Mulak et al., 2019; Vlach & Sandhofer, 2014; Yu & Smith, 2007), and making the CSWL task especially sensitive to word familiarity manipulations.

1.5. Summary and Current Study

Taken together, findings in PAL suggest a bilingual advantage in novel word learning, but the evidence for phonological working memory as the mechanism supporting this advantage is sparse and mixed. In CSWL, a bilingual advantage was also found, but only in the presence of competition in the stimuli, and while phonological working memory may be a factor in this observation, it has not been directly tested yet. While familiarity was found to facilitate PAL, it has been shown to be less important in bilingual PAL (Kaushanskaya, 2012; Kaushanskaya & Marian, 2009b), and manipulations of phonology have been drastically different in PAL vs. CSWL studies (Escudero et al., 2013, 2016b), making it difficult to identify the loci of such effects and to compare them across paradigms and participants with different language learning histories. The two paradigms have never been compared to each other when testing both monolinguals and bilinguals, and yet, ambiguity at learning suggests that CSWL is more challenging than PAL (Mulak et al., 2019; Vlach & Sandhofer, 2014; Yu & Smith, 2007), indicating that word learning performance could be worse in CSWL compared to PAL.

In the present study, we examined the degree to which word learning paradigm (PAL or CSWL), word familiarity (familiar words – homophones, and unfamiliar words – nonwords), bilingualism, and phonological working memory might predict word learning performance. We hypothesized that across groups, word learning performance would be higher in PAL versus CSWL, and on familiar versus unfamiliar words. Additionally, we predicted that bilinguals would learn more words than monolinguals across conditions and word types, and that this effect would be associated with phonological working memory. Moreover, we predicted that bilinguals’ experience with resolving cross-language ambiguity might give them an advantage in learning homophones compared to monolinguals. However, if the role of phonological working memory in word learning is similar for monolinguals and bilinguals, we should observe that higher scores on phonological working memory across groups are positively associated with word learning accuracy, particularly on CSWL, and that unfamiliar words are learned at a higher rate in participants with higher phonological working memory scores.

2. Method

2.1. Participants

We recruited 136 monolinguals on the online platform Prolific (Palan & Schitter, 2018), and 130 bilinguals, recruited both on Prolific (n = 88), and via a register of participants who had given consent to be recontacted for invitation to participate in new studies (n = 42). We derived the initial sample size of n = 136 from a power analysis conducted using the “modelPower” function in the lmSupport package (Curtin, 2018) in R Studio (v. 4.0.0; R Core Team, 2020). A meta-analysis of second language word learning from spoken input found a large effect of vocabulary gains (g = 1.05) (de Vos et al., 2018), but this figure conflates studies with different populations and testing procedures and does not include all the variables included in the present study. Therefore, we chose a more conservative, medium effect size of ηp ² = .06.

On Prolific, monolingual participants were pre-screened using the following filters: U.S. nationality, location in the U.S., age between 18 and 40, no other language than English, no language disorders and no hearing difficulties. The same filters were applied for bilinguals, and additionally, English had to be the native language, with knowledge of one additional language, Spanish. Bilinguals who were recontacted from previous studies were similarly screened (screener questions are available in the OSF repository for this article, at https://osf.io/k45z3/). One bilingual had to be excluded for indicating an age over 40.

Based on information provided in the Language Experience and Proficiency Questionnaire (LEAP-Q, Marian et al., 2007), where information about languages known, exposure to each language and proficiency levels is provided, 6 participants had to be excluded in the monolingual group due to exposure to another language more than 5% of the time. In the bilingual group, 11 participants had to be excluded due to exposure to a third language more than 5% of the time.

Regarding bilinguals’ second language (L2), 93 indicated it was Spanish, 9 English, 1 Spanish and Mandarin, and 8 did not reply. We excluded two participants who indicated French and Tagalog as L2. Additionally, we excluded one participant who was a native speaker of Spanish with English as L2. For those who did not reply, all had exposure to English and Spanish only, except for the Spanish–Mandarin bilingual who had equal exposure to Mandarin and Spanish (5% each). For the 9 bilinguals who indicated English as their L2, all indicated exposure to English from birth and exposure to Spanish from birth or 1 year old (2), or in a range from 2 to 6 years old (7). Bilingual participants’ characteristics are available in Table 1. The data suggests that the bilinguals learned Spanish on average during childhood, and that they learned in similar proportions by living in a Spanish-speaking country, living with Spanish-speaking family members, and learning in the classroom. We note however that the median for this latter measure is higher, at 4 years, compared to 0.5 and 0.5 years for the other two learning environment measures, suggesting a skew in the data towards more L2 learning in the classroom. The data on L2 proficiency suggests that participants had average to high proficiency in Spanish.

Table 1. Bilingual participants’ characteristics (n = 111)

^a Age of acquisition for Spanish, including participants who indicated an L2 other than Spanish.

^b Number of years and months spent in each environment. Data for months was converted to years.

^c Self-rated proficiency on a scale from 0 (none) to 10 (perfect).

*p < .05. **p < .01. ***p < .001.

The study was approved by the Institutional Review Board of the University of Wisconsin – Madison. All participants were compensated at a rate of $10 an hour, either through Prolific, or with a gift card, according to the source of recruitment. Participants began the study by providing informed consent to participate and indicating whether they consented to audio recordings (8 individuals in the monolingual sample and 7 in the bilingual sample did not consent to audio recordings, but were still included in data analyses). Participant characteristics are summarized in Table 2.

Table 2. Participant characteristics on samples after exclusions

^a Females (F), Males (M), Non-binary (N-B), no reply (NR).

^b Coded 1 through 8, from “Less than High School” to “Ph.D./M.D./J.D.”

^c Score normalized to 100.

^d For the variable of gender, Pearson's Chi-squared test was run instead as the data is categorical.

^e As the variances were unequal across groups for these variables, we used Welch's t-test.

*p < .05. **p < .01. ***p < .001.

In the final sample, monolinguals (M_mono = 29.98, SD = 6.44) were significantly older than bilinguals (M_bi = 25.81, SD = 5.86), and there was a significant gender difference – the bilingual group was majority female, while the monolingual group had a more balanced gender divide. Monolinguals and bilinguals did not significantly differ in the number of years of education received, nor on their English vocabulary score, which is unsurprising because all participants were native English speakers. Moreover, while the groups did not significantly differ on the backward digit-span measure, they did differ on the English nonword repetition task: bilinguals (M_bi = 61.94, SD = 15.76) scored ten percentage points above the monolinguals (M_mono = 51.33, SD = 20.38).

2.2. Materials

Stimuli

Participants learned a total of twelve novel words, of which six were unfamiliar nonwords conforming to English phonotactics (e.g., tosem, posek) and six were familiar words, i.e., were English homophones (e.g., cooker, alike). The unfamiliar words were chosen from the database of Gupta et al. (2004) and had first syllable stress for half of the stimuli, or second syllable stress for the other half. We matched the six familiar words with the unfamiliar words on stress pattern and biphone frequencies using the Clearpond database (Marian et al., 2012).

We chose twelve pictures from the Novel Object and Unusual Name (NOUN) database (Horst & Hout, 2016) to match the novel words. The objects were chosen such that they had average saliency. To limit bias in the pairings, two lists were created, where the objects that were paired with the familiar words in one list were paired with the unfamiliar words in the other. Participants were assigned randomly to a list. Stimuli are presented in Appendix C.

Vocabulary measures

Monolingual participants completed a vocabulary test of their English ability (Woodcock-Johnson III Picture Vocabulary Test - Tests of Achievement, Mather & Woodcock, 2001), and bilingual participants completed the same test, and its equivalent in Spanish (Vocabulario sobre Dibujos - Batería III Woodcock-Muñoz: Pruebas de aprovechamiento, Muñoz-Sandoval et al., 2005).

Both Picture Vocabulary tests were adapted to an online format, and shortened such that the start point was that of typical adults as per manual guidelines. Six pictures were displayed on the screen, at four levels of increasing difficulty. Participants were asked to record themselves saying the name of each picture (instructions are available in Appendix A5 for the English version and B3 for the Spanish version). Recordings were scored for accuracy and 10% of the data was double scored for inter-rater reliability. Research assistants were instructed to give one point per correct answer, or otherwise a zero. They were provided with a list of acceptable answers in both languages. On the Woodcock Johnson in English, 45 cases (8.82% of the data) from the monolingual participants and 42 cases (8.61% of the data) from the bilingual participants had to be removed due to poor or absent audio. On the Woodcock Johnson in Spanish, 69 cases had to be removed due to similar audio issues, or answering in English (14.02% of the data). We calculated an intraclass correlation coefficient using two-way random effects and a single-rater unit. Results on the test in English showed good agreement (ICC =.83, p < .001), and excellent agreement in Spanish (ICC = .99, p < .001). Final scores per participant are normalized out of 100. In our analyses, we retained the score in English as it was common to both groups of participants. The ranges of performance on the English version of this test (M = 53.32, SD = 18.79, Med: 54.17, Range [0:100]) indicated that the measure captured variability in participants’ English language skills.

Phonological working memory measures

All participants completed two working memory tasks: a nonword repetition task (English and Spanish versions: Lado, 2017) and a backward digit-span task (van den Noort et al., 2006; Wechsler, 1997). Monolinguals completed the tasks in English only, while bilinguals completed both tasks in English and Spanish. For the nonword repetition task, the original recordings in English and Spanish from Lado (2017) were used, but only stimuli up to five syllables in length were used, as piloting revealed floor effects beyond this threshold. Stimuli were normalized at 70 dB. Participants had to listen to the nonword pairs and repeat them immediately after (instructions for the English version are available in Appendix A4 and for the Spanish version in Appendix B2). Participants’ productions were scored for accuracy, and 10% of the data was double-scored to gain a measure of inter-rater reliability. Research assistants were instructed to score the productions as correct (1) or incorrect (0), without giving partial points for correct syllables. For the monolingual data, the intraclass correlation coefficient showed good agreement (ICC = .86, p < .001). For the bilinguals, agreement was excellent on the English version of the task (ICC = .92, p < .001). A portion of the data had to be excluded in each group due to poor audio, or no recording: for monolinguals, 110 cases were removed (9.86% of the data) and for bilinguals, 69 cases were removed (5.55% of the data).

The backward digit-span included 16 trials, with two trials per level of difficulty. Trials started at 2-digits length and ended at 9-digits length. The stimuli in English and Spanish were recorded by a simultaneous English–Spanish bilingual speaker and normalized at 70 dB. Participants had to listen to the digit list carefully and repeat the digits backwards (instructions for the English version are available in Appendix A3 and for the Spanish version in Appendix B1). Participants were given one practice trial before beginning the task. Only once the audio finished playing, the box to enter the response appeared, to limit the possibility of writing down numbers as they were being spoken. There was no time-limit on trials, and the Return key had to be pressed to move forward. The task was set to be automatically scored, and scores were normalized out of 100. Similarly to the vocabulary scores, only English scores were included in analyses as they were common to both groups. Because the data for the Spanish versions of the tasks were not the focus of this study, they are not described further.

The nonword repetition task and backward digit-span tasks positively correlated (r = .29, p < .0001). However, only the backward digit-span was used in models as it had no missing data.

Nonverbal IQ measure

Participants completed a measure of their nonverbal intelligence using the Visual Matrices of the Kaufman Brief Intelligence Test (KBIT-2) (Kaufman & Kaufman, 2004). For bilinguals, instructions were available both in English and Spanish. Participants were asked to choose the picture that best completed the relationship or the rule in a set of pictures or patterns (instructions for the monolingual version are available in Appendix A6 and for the bilingual version in Appendix B4). Each trial was limited to 30 seconds, and feedback was provided on the first three trials (following the test manual). A green tick mark was shown for correct answers, and a red cross was shown for incorrect ones. However, participants did not have an opportunity to self-correct on these trials, and difficulty level did not drop with incorrect answers. This task was automatically scored and individual scores were normalized out of 100.

Paired-associate word learning

The PAL experiment started with a teaching phase and ended with a testing phase. In the teaching phase, participants were exposed to each novel word-object pairing three times. Exposures were divided into three blocks. Within-block presentation and block order was randomized between participants.

The task began with instructions, where participants were told that they would be taught the names for several new objects (instructions are available in Appendix A1-i). Once participants were ready to begin, a black cross on a white background appeared at the center of the screen for 1000 ms. Then, the first novel object appeared on the screen and its associated name began to be spoken with a 400 ms delay. The object stayed on screen for 3400 ms. This cycle repeated 36 times. After the first block of 12 presentations, an attention check was inserted. Participants were told this was to check their engagement with the task and were asked to click “next”.

In the testing cycle, each word was tested three times. Presentation was divided into three blocks, with presentation randomized within and between participants. Testing was a four-alternative-forced-choice task, such that participants saw four pictures on the screen while one word was auditorily played, and they had to choose the picture they thought corresponded to the word spoken (instructions are available in Appendix A1-ii). Before the first set of pictures was presented, a fixation cross appeared for 1500 ms. The word was spoken with a 700 ms delay. Presentation of the four pictures was pseudorandomized such that no set contained repeating images within a trial, and pairings were pseudorandomized such that two words within the four options never appeared together more than six times. Each object appeared 10–14 times in all four zones of the screen. A picture could be clicked only after the target word was spoken. Responses were scored as 1 or 0 depending on whether they matched the correct answer.

Cross-situational word learning

This task began with instructions which did not reveal that it was a word learning task (instructions are available in Appendix A2-i). Once participants began the experiment, a black fixation cross appeared for 1000 ms. Then, two objects were displayed on screen and two words were auditorily presented. Each pairing was presented three times, in three blocks, with randomized order within and across blocks. The first word was spoken with a 400 ms delay after both objects appeared on screen, and for a duration of 1300 ms. The second object was named between 1700 ms and 3400 ms, to keep time per trial the same as in PAL. The same attention check as in PAL was presented after the first block for all participants. Presentation of the objects and their naming was counterbalanced left/right. Half of the words had four occurrences where the first word played was paired with the left-side object and the second word played was paired with the right-side object. On the other two occurrences, the first word played was paired with the object on the right side and the second word played, with the object on the left side. For the other six words, on four occurrences the first word played was paired with the right-side object and the second word played, with the left-side object. On the other two occurrences, the first word played was paired with the left-side object and the second word played was paired with the right-side object. This pairing was equally divided between word categories: familiar (homophone) or unfamiliar (nonword).

The testing phase was the same as in PAL, except that instructions did not explicitly indicate that auditorily presented words had to be paired with objects (instructions are available in Appendix A2-ii). The pseudorandomized order ensured that pairs seen as teaching were not systematically reproduced at test.

2.3. Procedure

Participants took the experiment on Gorilla Experiment Builder (Anwyl-Irvine et al., 2020). They first completed the consent form and ticked a box to express their consent to audio recordings. For those who did not consent, they were automatically directed to a version of the experiment that did not contain the tasks that required audio recording (that is, without the Woodcock Johnson test(s) and nonword repetition test(s)). Before beginning the experiment, a sound check was included to ensure participants’ volume was at an appropriate level, and that auto play worked. In both versions, participants first completed the word learning task and were randomly assigned, in equal ratios, to the PAL experiment, list A or B, or the CSWL experiment, list A or B. Bilinguals who consented to audio recordings were randomly assigned to complete the working memory tasks and vocabulary test either all in Spanish first, or English first, counterbalanced. Because the digits on the backward digit-span were the same in both tasks, the English and the Spanish versions were placed furthest apart, such that participants did a backward digit-span task, then a nonword repetition task, then a Woodcock Johnson vocabulary test, and repeated that sequence in opposite order, with both vocabulary tests being back-to-back, in the other language. Bilinguals who did not consent to audio recordings were randomly assigned to take the backward digit-span either in English or Spanish first. Then, all participants completed the KBIT-2 and the LEAP-Q. For monolinguals, the experiment took about 25 minutes, and for bilinguals, 35 minutes.

2.4. Analyses

We screened participant data on an attention check inserted in the teaching phase. Participants who took significantly longer to provide an answer at the attention check (response required to click on the button “Next”) as per reaction time plot visualizations were removed (4 monolinguals and 4 bilinguals).

Trials from the testing phase on which reaction times were three standard deviations above a participant's mean, or below 150 ms were also removed, to avoid introducing bias either due to forgetting after a time lapse, or clicking automatically. This respectively removed 83 cases (1.83% of the data) and 85 cases (1.91% of the data) for monolinguals, and 84 cases (2.10% of the data) and 96 cases (2.45% of the data) for bilinguals. This corresponds to 1.96% of the monolingual and bilingual data combined for trials on which reaction times were above three standard deviations above a participant's mean, and 2.16% of the overall dataset removed for trials on which reaction times were under 150 ms.

We constructed logistic mixed effects models in R Studio, version 4.0.0 (lme4 package, Bates et al., 2015) to examine the role of condition, word type, group, and verbal working memory in predicting the likelihood of word learning accuracy. Because our ad-hoc predictions were that the four variables would differentially affect learning accuracy, we examined both their main effects and interactions. Item-level dichotomous data from the testing blocks on word learning accuracy was used as the dependent variable. We tested model assumptions with the DHARMa package, and they were satisfied (Hartig, 2022).

Because Pearson t-tests revealed that groups significantly differed on age and on the KBIT-2, and a chi-square test showed the groups significantly differed on gender, preliminary analyses included these variables as covariates. Only the KBIT-2 score variable improved model fit, therefore only this covariate was retained in our models.

We included Condition (-0.5, 0.5), Group (-0.5, 0.5), Word Type (-0.5, 0.5) and backward digit-span centered around each participant's mean and their interaction as fixed effects, and the KBIT-2 score centered around each participant's mean as a covariate. Word type varied within participant (participants saw both familiar and unfamiliar words), and group, condition, backward digit-span and the KBIT-2 varied within item (an item is “seen” by individuals in different groups, condition and with different working memory scores). Convergence and singularity issues emerged after reducing both the by-subjects structure to a random intercept, and the by-item random effects structure following Brauer and Curtin, 2018: removing random effects for covariates which do not interact with key predictors, removing lower-order random effects terms, and covariances among random effects and the random intercepts. We next simplified the by-item random effects structure to only a random intercept and reintegrated a slope for word type in the by-subjects structure. This model converged and singularity issues were resolved.

3. Results

Models were run on 237 participants (8184 observations). Both monolinguals and bilinguals learned above chance (set at 25% due to the 4-alternative-forced choice task at test) in PAL (M_mono = 92%, SD_mono = 27%; Range: 39%-100%; t(2121) = 112.13, p < .0001; M_bi = 94%, SD_bi = 24%; Range: 22%-100%; t(1797) = 121.23, p < .0001) and in CSWL (M_mono = 71%, SD_mono = 45%; Range: 17%-100%; t(2245) = 48.26, p < .0001; M_bi = 69%, SD_bi = 46%; Range: 8%-100%; t(2017) = 42.77, p < .0001). Mean learning proportions by group, condition and word type are summarized in Table 3.

Table 3. Average accuracy per group, condition and word type

The logistic mixed-effects model looking at the effects of Condition, Group, Word Type, backward digit-span and their interaction, and controlling for KBIT-2 score showed no significant main effect of Group (b = -0.01, SE = 0.24, z = -0.04, OR = 0.99, p = .97). There was a significant main effect of Condition such that participants in the PAL condition learned novel words more accurately than in the CSWL condition (b = -2.29, SE = 0.24, z = -9.58, OR = 0.10, p < .0001). Results also revealed a significant main effect of Word Type such that familiar words were learned more accurately than unfamiliar words (b = 0.56, SE = 0.20, z = 2.86, OR = 1.75, p < .001). The main effect of the backward digit-span was not significant (b = 0.01, SE = 0.01, z = 1.36, OR = 1.01, p = .17).

There was a trend for an interaction between Group and Condition (b = -0.88, SE = 0.47, z = -1.86, OR = 0.41, p = .064), such that the effect of Condition on word learning decreased by a factor of 0.41 for bilinguals compared to monolinguals. We followed up on this interaction and found no effect of Group on word learning in PAL, or in CSWL, indicating that the interaction was driven by the difference in the direction of the effects, rather than different degrees of significance. See Figure 1 to view graphed results of the Group x Condition interaction.

Figure 1. Word learning probability as a function of Group and Condition, with standard error bars.

The interaction between Condition and Word Type was significant (b = -0.42, SE = 0.19, z = -2.25, OR = 0.66, p < .05), such that the effect of Word Type on the likelihood of learning novel words decreased by a factor of 0.66 in CSWL compared to PAL, indicating that the familiarity effect was stronger in PAL. There was also a trend for an interaction between Condition and backward digit-span (b = -0.02, SE = 0.01, z = -1.79, OR = 0.98, p = .074), such that the effect of the backward digit-span on word learning decreased by a factor of 0.98 in CSWL compared to PAL. We followed up on this interaction and found no effect of the backward digit-span on CSWL, but a trending effect of the backward digit-span on PAL, such that for every one unit increase in the backward digit-span score, its effect on word learning increased by a factor of 1.02 (b = 0.02, SE = 0.01, z = 1.96, OR = 1.02, p < .05).

None of the other two-way, three-way or four-way interactions were significant. See Table 4 for the full model results.

Table 4. Word learning accuracy by Group, Condition, Word Type and backward digit-span, controlling for KBIT-2 score

Note. Model: Accuracy ~ Group * Condition * Word Type * Digit Span + KBIT + (1+Word Type| Participant) + (1|Target Word).

*p < .05. **p < .01. ***p < .001.

As bilinguals scored significantly higher on the nonword repetition task compared to monolinguals, we ran an exploratory model following the same structure as the model reported in Analyses, with the nonword repetition task scores used in place of the backward digit-span scores. All findings had the same level of significance, except for the Condition by Word Type interaction which became significant at the .01 level instead of .05 (b = -0.54, SE = 0.21, z = -2.62, OR = 0.58, p < 0.01).

We ran an equivalence test using Bayesian logistic regression to confirm there was no difference between monolingual and bilingual groups. As there is little previous work on this topic and findings are mixed, we used the default priors for the intercept and all effects. We used Stan in R (Goodrich et al., 2023) to estimate the posterior distribution. We ran four chains with 2,000 iterations each, 1000 warm-up draws and 1000 sampling draws, yielding 4000 total draws from the posterior distribution.

The most likely value for the posterior for the main effect of group was 0: bilinguals and monolinguals were similarly accurate in learning novel words across conditions, word types and at average levels of backward digit-span and KBIT-2 scores, 95% credible interval = [-0.48, 0.48], Bayes Factor = 0.053. Given that the credible interval included 0, we are confident that 0 is a credible value for the main effect of group. Additionally, the Bayes Factor for the main effect of group suggests we have strong evidence for the null hypothesis.

4. Discussion

Previous work has examined the role of bilingualism in PAL, but much less so in CSWL. Moreover, the PAL and the CSWL paradigms have emerged from and been studied under different theoretical umbrellas, which has limited the possibility of gaining a comprehensive overview of word learning mechanisms across paradigms. Research in PAL with monolingual participants has shown that familiarity supports word learning, and that phonological working memory facilitates learning in PAL (e.g., Baddeley et al., 1998). Therefore, this study examined whether bilingualism affects performance in PAL and CSWL, and whether bilingualism interacts with familiarity across paradigms. In addition, the role of phonological working memory in predicting novel word learning across conditions, bilingual status and word familiarity was examined.

We found that word learning was more successful in PAL compared to CSWL for both monolinguals and bilinguals, suggesting that PAL is a less complex word learning paradigm than CSWL, a pattern which can be attributed to its lack of ambiguity (Mulak et al., 2019). While the group effect was not significant in either paradigm, the direction of monolingual and bilinguals’ word learning performance switched between PAL and CSWL. In PAL, bilinguals tended to learn more words than monolinguals, but this effect was reversed in CSWL; this trend was very weak, however.

Therefore, we did not find an overall bilingual advantage for novel word learning, although it has generally been found in PAL (e.g., Bogulski et al., 2018, Exp. 1; Kaushanskaya & Marian, 2009a, 2009b; Kaushanskaya, 2012; Van Hell & Mahn, 1997; Warmington et al., 2019), and also observed in CSWL (Escudero et al., 2016a). However, a lack of group differences on word learning performance in PAL has been observed by Tsuboi and Francis (2020), and in CSWL by Escudero et al. (2016b) and Mulak et al. (2019). In these two CSWL studies, while the samples included both monolinguals and bilinguals, dividing these participants into groups and including group in the models did not improve model fit.

One possible reason for the discrepant findings between our study and those that have observed group differences in word learning performance is that different analysis strategies may yield distinct results. For instance, many of the previous studies that have found a bilingual advantage on novel word learning did not account for the non-independence in the data that emerges from participants providing multiple data points in the testing phase, which introduces error, or randomness, in the data. Failing to account for random effects leads to an overly high Type I error rate (Brauer & Curtin, 2018), which is why we used mixed-effects modelling in our study to address this issue. We note however that while our findings are in line with Tsuboi and Francis (2020), their analyses did not include random effects, warranting replication of our findings. Another possible reason is that the difficulty level of the word learning task (over and above the type of word learning paradigm) may modulate group differences in word learning performance, such that for example, bilingual advantages would be more likely on more taxing tasks (in line with Friesen et al., 2015). This explanation does not quite fit with our data, where a trend for better bilingual performance was observed for the easier PAL task. However, it does provide a fruitful avenue for future work that would implement task-difficulty manipulations across different word learning paradigms to identify the possible task-difficulty threshold where possible bilingual effects on learning might be observed.

Word learning was more successful on familiar compared to unfamiliar words and there was no interaction between group and word type, in line with previous studies (Kaushanskaya, 2012; Kaushanskaya et al., 2013), suggesting that the effect of word familiarity was the same across groups. Our findings confirm that word familiarity supports word learning, over and above word learning paradigm or language learning history. They further suggest that monolinguals and bilinguals do not differ in how they process familiar and unfamiliar words, at least at similar levels of phonological working memory capacity. Moreover, bilinguals’ increased experience with cross-linguistic homophones (e.g., Escudero et al., 2016a) did not translate into a specific homophone learning advantage over monolinguals. We also note that in Tsuboi and Francis (2020) while a general bilingual advantage in PAL was not found, there was a learning advantage associated with higher proficiency in the language through which the novel words were learned. In our study, all novel words (familiar and unfamiliar) conformed to English phonotactics, and groups did not significantly differ on their English vocabulary scores. If novel words were manipulated such that the unfamiliar words were novel in terms of phonotactics, we may have found group differences. That is, if experience with the phonology of two languages leads to a more agile phonological loop, we might see a larger gap in learning performance between learning performance on familiar and unfamiliar words in monolinguals compared to bilinguals.

The main effects of learning condition and word type compounded in an interaction such that the gap between successful learning of familiar versus unfamiliar words widened in CSWL. In other words, learning of unfamiliar words in CSWL was the most difficult learning situation in our experiment. This may be explained by the fact that the information load was twice as high in CSWL compared to PAL, which might have affected performance when compounded with unfamiliar words. This finding was independent of phonological working memory, suggesting that while unfamiliar words may have been more difficult to learn, they may not have been so to such an extent that they required greater reliance on phonological memory to be encoded. Indeed, even the unfamiliar words conformed to English phonotactics. Research on processing unfamiliar words in the PAL literature involving monolingual participants suggests that familiar phonology is processed through activation of long-term memory representations of known phonology, supporting phonological working memory (e.g., Ellis & Beaton, 1993; Majerus et al., 2004; Papagno et al., 1991; Papagno & Vallar, 1992; Service & Craik, 1993). However, unfamiliar phonology cannot be supported by long-term memory and as a result, relies to a larger extent on phonological working memory capacity. To further test this hypothesis, unfamiliar words could be constructed such that they do not conform to any languages known by monolinguals and bilinguals tested.

The effect of phonological working memory was stronger for PAL than for CSWL, although it is important to reiterate that this was only a marginal trend in the analyses. There was no evidence that our measure of phonological working memory was differentially predictive of word learning performance in bilinguals and monolinguals. The absence of a group effect in the current study is independent of phonological working memory as groups did not significantly differ on the backward digit-span measure, and distribution of the data was normally spread in the sample containing both monolinguals and bilinguals (M = 60.45, SD = 18.95, Med: 62.5, Range [0:100]). We note however that the backward digit-span task significantly correlated with the KBIT-2 score (r = .22, p <.001), which was included as a covariate in the model. Its inclusion likely obscured the effect of the backward digit-span, which is significant at the .05 level when the KBIT score is not included. The bilinguals did score significantly higher on the English nonword repetition task compared to monolinguals, suggesting a bilingual advantage on that task (in line with several previous studies, e.g., Kaushanskaya, 2012; Papagno & Vallar, 1995; Warmington et al., 2019). Findings from an exploratory model using the nonword repetition task scores in English instead of the backward digit-span task scores revealed highly similar results, pointing to a similar association of verbal working memory and word learning accuracy across monolinguals and bilinguals, independent of the verbal working memory task used. It must be noted that nonword repetition data from 31 participants across groups (17 monolinguals, 14 bilinguals) were missing or inaudible, therefore, in future studies, better quality nonword repetition data would need to be collected to further examine its association with word learning across groups and word types.

We observed a trend such that the backward digit-span was associated only with PAL and not CSWL, and that association was positive, such that as scores on the backward digit-span task increased, PAL performance increased. While this effect is opposite to our prediction that phonological working memory might associate to a higher extent with CSWL due to its ambiguity compared to PAL, our CSWL task may not have been ambiguous enough to let that effect emerge (Mulak et al., 2019). It would be interesting to examine this hypothesis by increasing the ambiguity at learning in CSWL with a 3 x 3 or 4 x 4 design instead of 2 x 2.

In line with numerous previous studies (Atkins & Baddeley, 1998; Baddeley et al., 1988, 1998, 2017; Gupta, 2003; Papagno et al., 1991; Papagno & Vallar, 1992, 1995; Service, 1992; Speciale et al., 2004), we found an association between phonological working memory and PAL. Compared to CSWL, it is possible that because the PAL task is explicitly a word learning task, participants may have been in “learning mode”, and solicited the phonological loop to a higher extent through rehearsal to learn the novel word-object pairings. To test this hypothesis further, a manipulation of articulatory suppression could be introduced in both paradigms to examine the role of rehearsal in word learning in PAL and CSWL.

5. Conclusion

This study is the first to directly compare PAL and CSWL paradigms across monolinguals and bilinguals. Findings contribute to the understanding of the mechanisms underlying each paradigm. Previous work has suggested a bilingual advantage in word learning both in PAL and CSWL, potentially taking place through phonological working memory. Our findings suggest no difference on word learning performance across monolinguals and bilinguals in either paradigm and on either familiar or unfamiliar words, at similar levels of phonological working memory. We did observe a trend for bilinguals to outperform monolinguals on PAL but not on CSWL, and this finding is consistent with other studies indicating a bilingual advantage on PAL-type tasks (Kaushanskaya, 2012; Kaushanskaya & Marian, 2009a, 2009b), and a much more constrained advantage on CSWL tasks (Escudero et al., 2016a). However, in contrast to previous PAL studies, our findings only indicated a weak trend towards this effect, suggesting that prior reports may have been biased by less appropriate statistics.

We additionally found that word learning is easier in PAL compared to CSWL, and on familiar versus unfamiliar words. This suggests that the role of phonological working memory in supporting the learning of words with varying levels of familiarity is the same across paradigms and groups, at least based on how phonological working memory, word familiarity, and word learning were tested in our study. Further research is warranted to test bilinguals across the lifespan, controlling for both linguistic and cognitive variables that could affect learning performance within bilinguals. Moreover, task design could be further manipulated in terms of word familiarity and task complexity, to examine whether these results hold on a continuum of phonological working memory, and working memory demands.

In conclusion, for the first time, this study tested monolinguals and bilinguals on comparable PAL and CSWL tasks. While a trend for better bilingual performance was observed for PAL but not for CSWL, statistically speaking, this effect was negligible, and the findings largely suggest that word learning was accomplished similarly by the two groups, independent of paradigm and novel word familiarity. Furthermore, while working memory may be more involved in PAL than in CSWL, bilinguals and monolinguals did not differ in their reliance on phonological working memory for the accomplishment of either word-learning task. Together, the findings suggest a great degree of overlap in word learning patterns and the mechanisms that support them for bilingual and monolingual adults. The fact that our data were collected remotely from a relatively large sample of bilinguals and monolinguals strengthens these conclusions and indicates the need for larger-scale approaches to examining possible effects of bilingualism on language performance.