Forty-two years of computer-assisted language learning research: A scientometric study of hotspot research and trending issues

Abstract

For years, computer-assisted language learning (CALL) has thrived as an interdisciplinary subfield, linking applied linguistics and educational technology. Despite its significance and a number of syntheses, CALL research has not yet undergone a comprehensive scientometric synthesis. This study synthesizes CALL research over a period of 42 years by employing a scientometric analysis of sources and document co-citation analyses. Scopus was used to retrieve original articles with a timespan limit from 1980 to 2021. Our records identified 4,631 articles representing CALL-based research, which were published in 63 peer-reviewed journals and collectively contained 186,589 references. The findings indicate that CALL research is supported by robust theoretical frameworks, grounded in socio-cultural and second language acquisition theories. Our research findings have revealed several significant clusters of interest within the realm of CALL, with a pronounced focus on writing among CALL scholars. Additionally, the study identified emerging research areas such as mobile-assisted language learning, synchronous computer-mediated communication, and data-driven learning in CALL literature. Notably, “CALL-core” journals exhibited high productivity, with Language Learning & Technology, Computer Assisted Language Learning, and Computers & Education standing out as top-ranked journals in terms of the Hirsch index (h-index) and co-citation. Suggestions for future research are outlined in the conclusion.

1. Introduction

Computer-assisted language learning (CALL) publications have witnessed exponential growth over the past four decades, commencing with the launch of the first “CALL-core” journal, ¹ CALICO Journal, in 1983. Subsequently, ReCALL in 1989 and Computer Assisted Language Learning in 1990 emerged as platforms for disseminating CALL research, joined by the online journal Language Learning & Technology (LL&T) in 1997, catering to the growing imperative to integrate technology into pedagogy. Owing to its intersection with fields such as educational technology, applied linguistics, and TESOL, CALL publications were not confined to CALL-exclusive journals, even preceding their establishment. Multiple venues for publications probing technology in education (e.g. Computers & Education, British Journal of Educational Technology, Educational Technology Research & Development) and language learning (e.g. Foreign Language Annals, The Modern Language Journal, Language Learning) have featured CALL-related research (Stickler & Shi, 2016). At present, CALL is experiencing a discernible increase in publication, manifested through the creation of CALL journals in non-English-speaking territories, such as the International Journal of Computer-Assisted Language Learning and Teaching and the Journal of China Computer-Assisted Language Learning. This proliferation mirrors the significant evolution of CALL theory and technology from the early 1980s to the 21st century. To shed light on that evolution, Lim and Aryadoust (2022) documented an impressive upward trajectory in CALL publications, escalating from merely two articles in 1977 to 171 in 2017, followed by a remarkable surge to 261 articles in 2019. This trend is projected to persist, exemplifying Bax’s (2003) “normalization” within the CALL field. With the escalating corpus of CALL research articles, an imperative need has arisen to consolidate this extensive literature, not solely from CALL-core journal publications but also including non-CALL journals, employing a scientometric analysis methodology. This approach furnishes a thorough and nuanced comprehension of research trends, offering language instructors, pedagogues, and practitioners insights into previously explored hotspot issues, technology applications, targeted language skills, and research contexts. Such knowledge guides them towards effective language teaching and the identification of untapped areas warranting further exploration.

Indeed, a multitude of systematic review articles and research syntheses center on CALL in its entirety, striving to uncover prevailing research themes (Choubsaz, Jalilifar & Boulton, 2023), underlying theoretical frameworks (Hubbard & Levy, 2016), technological devices (Shadiev & Yang, 2020), and research methodologies (Choubsaz et al., 2023; Debski, 2003). A separate strand of research synthesizes specific facets of CALL, including mobile-assisted language learning (MALL) (Duman, Orhon & Gedik, 2015), blended language learning (Li, 2022), and multimedia glosses (Mohsen & Balakumar, 2011). However, the majority of these research syntheses, whether employing an analytical stance or a traditional systematic review format to encapsulate study findings, exclusively target CALL-core journals, overlooking contributions from non-CALL journals. Additionally, despite the burgeoning growth of the CALL field, there is an observable dearth of comprehensive synthesis utilizing the scientometric analysis method, defined as “quantitatively analyzing patterns in scientific literature to understand emerging trends and the knowledge structure of a research field” (Chen, Hu, Liu & Tseng, 2012: 593). In our study, we endeavor to provide a detailed examination of CALL’s trending research issues, hotspot areas, document co-citation analysis, and source co-citation analysis. By leveraging a scientometric methodology, we scrutinize CALL studies over 42 years, encompassing both CALL and non-CALL publication venues. Synthesizing CALL research should provide valuable insights for researchers through highlighting key trends, technologies, and theories over four decades, additionally pinpointing unexplored areas for future study. Furthermore, educators may leverage the insights from this synthesis to refine pedagogical approaches, thereby optimizing student engagement and aligning instructional practices with contemporary advancements in CALL.

2. Literature review

2.1 Theoretical foundations

Over the years, CALL has experienced substantial development in cutting-edge technologies, requiring corresponding advancements in CALL theoretical foundations. Although Hubbard (2021) argues that much CALL research seems to be atheoretical, with some studies relying on borrowed theories from other domains, such as second language acquisition (SLA) or general education, Hubbard and Levy (2016) posit that theories represented in CALL research can take various forms: “absent, borrowed singly, assembled in an ensemble, instantiated, adapted, synthesized, and even created” (p. 28). Chun (2016) points out that CALL research has been theoretically grounded in SLA theories, specifically the interactionist approach (IA) (Long, 1996). This approach motivates researchers to explore interactions between learners and computers, enhancing the negotiation of meaning. Chun (2016) presents a model based on Bax’s (2003) framework, categorizing CALL’s stages into four periods, each influenced by a dominant theory and accompanied by specific technological devices: structural CALL (1970s–1980s), using mainframe computers; communicative CALL (1980s–1990s), represented by PCs; integrative CALL (2000s), incorporating multimedia and the internet; and ecological CALL (2010s), involving mobile and wearable devices.

Hubbard and Levy (2016) also assert that SLA theories, particularly IA, remain central to much CALL research. IA inspires research on synchronous computer-mediated communication (SCMC), where learners engage in email exchanges, and cases where written input is corrected through automated writing evaluation (AWE) software. Additionally, Chun (2016) identifies two other influential theories in CALL research. First is Vygotsky’s socio-cultural theory, emphasizing the role of social interaction in learners’ language development. Technology enables second language (L2) learners to interact with native speakers and others through various channels like social media or virtual attendance in classes with learners from different parts of the world, such as massive open online courses (MOOCs). Second is constructivist theory, suggesting that language and culture significantly impact learning. Learners gain valuable experiences and enhance their language skills through involvement in social interactions (Chun, 2016). Despite the prevalence of underlying theoretical frameworks guiding much CALL research, many research syntheses, reviewed in section 2.2, fail to pinpoint the extent to which CALL research to date has been rooted in theory.

2.2 Previous CALL research syntheses

Many research syntheses have been proposed to analyze studies investigating the intersection of technology with language learning and teaching across different timespans. The primary aim of these syntheses is to understand how technology is integrated to enhance pedagogy and to examine the treatment of CALL both technologically and methodologically. Our focus here is to review studies that use an analytical approach to synthesize CALL research, aligning with our study objectives. An example of this type of research synthesis is Debski’s (2003) analysis of a corpus of 91 CALL studies conducted between 1980 and 2000. Debski aimed to unravel the technological devices employed and methodological designs adopted in CALL research during that period. The results of the analysis revealed that CALL research was steadily advancing in terms of theoretical foundations and methodological designs. According to Debski, CALL research was most prominently grounded in theories related to SLA, L2 pedagogy, and natural language processing (NLP). Concerning methodological designs, most CALL research adopted an experimental design, and the experiments usually lasted for less than six months. Language testing was predominantly used as an instrument for data collection.

Another attempt was made by Felix (2005), who reviewed 52 CALL studies from 2000 to 2004, aiming to evaluate the effectiveness of CALL research. The review primarily examined design, technology usage, setting, and language skills, revealing an increasing number of experimental and quasi-experimental designs compared to traditional methods. Furthermore, there was a growing emphasis on investigating learning processes alongside outcomes. However, certain issues persisted, such as poor research descriptions, misleading titles that claimed definitive “effectiveness,” and a lack of thorough literature reviews. Felix called for focused research syntheses on key variables to advance understanding of how CALL impacts processes and outcomes. Golonka, Bowles, Frank, Richardson and Freynik’s (2014) analysis of 350 CALL studies concluded that despite abundant publications on technology for L2 learning, robust evidence of direct impact on learning is not well supported. In a broader synthesis covering 11 years (2006–2016), Gillespie (2020) reviewed CALL research from three CALL-core journals: ReCALL, CALICO Journal, and Computer Assisted Language Learning. His analysis of 777 articles revealed 13 main topics: writing, computer-mediated communication (CMC), vocabulary, speaking, corpora, NLP, design, teacher education, reading, listening, Web 2.0, grammar, and feedback. International participation was growing dramatically, especially from Asia. Nonetheless, a detailed analysis could not be fully established as articles published in LL&T and non-core CALL journals, along with CALL studies from other disciplines, were not included in Gillespie’s coverage.

In a more expansive synthesis, subsequent studies were undertaken by Choubsaz et al. (2023) to analyze highly cited articles published in four major CALL journals. This resulted in a corpus of 2,397 articles, from which 426 documents were selected to explore prevalent research issues, methodological designs, and theoretical foundations shaping CALL research articles. The findings revealed that empirical studies utilizing eclectic or quantitative methods were predominant, with an emerging trend in mixed methods. While socio-cultural theory and IT were commonly referenced, it was noted that 20% of the studies lacked a theoretical framework. The research also highlighted that CMC, writing, and vocabulary were the most prevalent research topics. Although employing robust synthesis techniques supported by NVivo, we acknowledge the imperative for a more exhaustive and intricately detailed analysis that encompasses articles published in journals pertaining to educational technology and applied linguistics. In pursuit of this goal, a broader and more intricate synthesis of research derived from non-CALL journals would seem essential, utilizing a scientometric approach. This approach, adopted in the present study, is anticipated to provide a comprehensive and in-depth understanding of the broader landscape surrounding CALL research, incorporating insights from diverse fields and disciplines.

2.3 Scientometrics

Scientometric analysis has been employed extensively across various disciplines to identify and highlight the most prevalent and trending issues within specific fields of study (Chen, 2006). Scientometrics refers to “ways of measuring research quality and impact, understanding the processes of citations, mapping scientific fields and the use of indicators in research policy and management” (Mingers & Leydesdorff, 2015: 1). It is vital to differentiate between scientometrics and bibliometric studies, each with its own unique emphases and methodologies. Bibliometrics primarily focuses on the patterns of authorship, the quantitative assessment of journal and research institution influence, and the content analysis of words found within titles, abstracts, or keywords. Scientometrics, on the other hand, engages in citation analysis to evaluate the progression and direction of scientific research endeavors (Brookes, 1990; Qiu, Zhao, Yang & Dong, 2017; Zakaria & Aryadoust, 2023). Despite their differences in focus, scientometrics and bibliometrics can be interconnected, enabling their methods to be used together. This combination facilitates a thorough examination of academic landscapes, merging insights into authors’ contributions, the significance of publications, and the dynamics of research content. It thus contributes to a more complete understanding of scholarly progression (Brookes, 1990; Qiu et al., 2017).

In the field of scientometrics, document co-citation analysis (DCA) plays a key role, referring to instances where two works are cited together within other scholarly publications (Small, 1980), reflecting a strong thematic relatedness (Zhu & Aryadoust, 2023). DCA examines the data within academic documents to understand the complex connections among different papers (Chen, 2003, 2006). The main goal of DCA is to reveal the complex web of relationships between these documents, identifying significant cases where publications have received recognition through frequent citation (Chen, 2003, 2006). This technique is valuable for exploring the base of scholarly influence and recognition in a particular field. Aryadoust, Zakaria, Lim and Chen (2020) argue that DCA seems similar to factor analysis, which is used to reduce data obtained from surveys and tests. The former is concerned with grouping clusters based on their correlational patterns using data mining methods, the latter with identifying factors using factor analysis methods (Aryadoust et al., 2020). Scientometric analysis is used in different disciplines, such as computer science (Katuk, Ku-Mahamud, Zakaria & Jabbar, 2020), education (Mohsen & Alangari, 2023), and translation studies (Zhu & Aryadoust, 2023); however, it is not well established in language learning and teaching.

There have been limited endeavors to employ scientometric studies for uncovering scholars’ research interests in the realm of applied linguistics, both in its broader scope (Zakaria & Aryadoust, 2023), and within specific domains, such as comprehension subskills (Aryadoust, 2020), language assessment (Aryadoust et al., 2020), machine translation and language learning (Mohsen, Althebi & Albahooth, 2023), as well as CALL (Lim & Aryadoust, 2022). Of direct relevance to our study’s objectives, Lim and Aryadoust (2022) conducted a scientometric examination of CALL literature spanning 1978 to 2019, focusing on 11 CALL and non-CALL journals indexed in Scopus. Their analysis unveiled 37 distinct clusters and their contributions to learning outcomes. The most prominent cluster centered around research on advanced learners, particularly emphasizing SCMC and negotiated interaction. Furthermore, the study indicated strong support for SCMC, digital game-based language learning, multimedia glosses, and negotiated interaction within the CALL literature, while blogs, wikis, and podcasts remained underexplored topics. Another notable effort utilizing topic modeling and bibliometric analysis is from Chen, Zou, Xie and Su (2021), who synthesized developmental research trends in CALL during the period 1995–2019. Their investigation encompassed 1,295 articles from journals indexed in the Web of Science Core Collection (WoSCC). The outcomes revealed a trajectory wherein technologies evolved from initially limited topics like multimedia-enhanced learning (1995–1999) to a range of technologies such as CMC, automatic speech recognition, and MALL in the intermediate stage (2000–2009), culminating in a diverse array of applications and tools including wikis, virtual reality/virtual world (VR/VW), digital game-based language learning, and digital multimodal composing (DMC) as prominent features in the final stage (2010–2019). Employing a nonparametric trend test, they discerned the prevalence of newer studies focusing on DMC, mobiles, wikis, CMC, SCMC, VW, and VR.

Lim and Aryadoust (2022) and Chen et al. (2021) have made significant strides in advancing scientometric and bibliometric studies within CALL. However, there remains an unexplored landscape within CALL literature that their research did not include. To bridge this gap, we conducted a comprehensive investigation across a wide array of journals publishing CALL research. This exploration will involve strategically selecting pertinent keywords to retrieve CALL-focused articles and utilizing expansive databases that index CALL literature. This contrasts with Lim and Aryadoust’s (2022) analysis, which was confined to just 11 journals, leaving numerous journals in applied linguistics, educational technology, and multidisciplinary domains unexplored. Additionally, Chen et al. (2021) limited their data to WoSCC-indexed journals, bypassing key CALL-core journals like CALICO Journal and CALL-EJ, absent from WoSCC. Our undertaking seeks to meticulously depict the CALL research landscape by encompassing an extensive range of journals. Our primary objective is twofold: first, to identify prevalent research themes from 1980 to 2021, with a particular emphasis on the last five years (2017–2021) due to the surge in CALL publications during this period, as noted by Lim and Aryadoust (2022). Second, we aim to unearth additional metrics, including authors’ keywords, co-occurrences, journals’ production, and co-citations, thereby illuminating the most influential contributors and journals within the realm of CALL research. This approach will not only enrich our understanding of CALL’s evolving landscape but also provide valuable insights for CALL researchers seeking to navigate the intricacies of this dynamic field. We address the following research questions:

1. 1. What are the most trending research issues in the CALL field during the general period of 1980–2021, and more specifically in the last five years (2017–2021)?

2. 2. What are the most influential co-cited sources and articles in CALL research during the period spanning 1980–2021?

3. Methodology

3.1 Data source

Use of Scopus for data retrieval was driven by several key considerations: (1) Scopus offers an extensive compilation of reliable publications across diverse disciplines, encompassing peer-reviewed journals that span humanities, social sciences, computer sciences, and multidisciplinary research – a spectrum well suited to CALL studies. (2) The indexing of CALL-core journals in Scopus from their inaugural issues ensures a comprehensive representation of research articles within the scope of our study. (3) The research information system (RIS) and comma-separated values (CSV) file formats extracted from Scopus facilitate seamless integration with scientometric software, such as CiteSpace and VOSviewer. It is noteworthy that we did not incorporate data from WoSCC due to the delayed indexing of essential CALL-core journals, including LL&T, ReCALL, and Computer Assisted Language Learning; additionally, CALICO Journal remains absent from the SSCI. To ensure data quality, our sources comprised 63 peer-reviewed journals, excluding 11 journals that were discontinued from Scopus due to publication concerns or being classified as predatory (supplementary material A1). Our data set is confined to articles only, with other document types like reviews, letters to editors, books chapters, conference proceedings, and corrections excluded from our study.

3.2 Search terms

CALL articles usually appear in four types of journals:

1. 1. CALL-focused journals are categorized into two primary subgroups: (a) CALL-core journals and (b) non-CALL-core journals (supplementary material A2).

2. 2. Educational technology journals whose scope is to publish studies that integrate technology into education. Sixteen journals were recorded that publish CALL research (supplementary material A3).

3. 3. Applied linguistics and SLA journals whose scope is to investigate issues related to language learning and teaching.

4. 4. Multidisciplinary and social science journals that have a broad scope and address a wide spectrum of social science topics, including issues pertinent to CALL research.

In our pursuit, we meticulously extracted all article documents within category 1 (N = 2,291). Subsequently, we executed targeted key term searches in educational technology journals (category 2), aligning with the terms employed by Lim and Aryadoust (2022), albeit with minor adaptations (supplementary material B1). This attempt resulted in the identification of 2,236 articles. We conducted further key term searches to capture articles falling within categories 3 and 4. These searches honed in on educational technology, language learning, literacy, and teaching-related terms, as adapted from Chen et al. (2021) (supplementary material B2). In particular, the search in categories 3 and 4 was confined to subject areas falling under social sciences, arts, and humanities, resulting in the discovery of 8,977 articles. It is important to note that the search was not restricted by a specific time frame, excluding 2022 articles. The search process was concluded on August 10, 2022.

3.3 Data filtering

The data underwent a thorough screening review conducted by a panel of five researchers comprising three master’s students specializing in the field, a doctoral candidate, and a professor, all with expertise in CALL. This assessment aimed to ensure the alignment of retrieved articles with our predetermined criteria for identifying CALL research, as defined by Levy (1997), which emphasizes the computer’s application in language teaching and learning. We scrutinized article abstracts to determine if they primarily investigated technology as a primary focus for enhancing language learning and teaching. Notably, articles employing technological tools solely for monitoring student learning processes, such as eye-tracking software and keylogging programs, were excluded.

Several articles lacked abstracts or author information, or were editorial pieces erroneously classified as original articles; these were excluded during the data refinement phase, as were retracted documents. Although our initial intention was to exclude review articles from our search, some review and meta-analysis studies appeared in the search results, misclassified as original articles. To address this, we employed specific terms such as scoping review, systematic review, meta-analysis, meta-synthesis, research synthesis, and research agenda in the titles and/or abstracts of retrieved documents. Matching results were subsequently excluded.

Ambiguous articles that did not definitively fit the CALL-based research criteria were identified, flagged, and resolved through in-depth discussions during online meetings, culminating in a consensus. For heightened reliability, peer review was conducted on files for categories 2, 3, and 4 to ensure article adequacy: interrater reliability was .86, .91, and .88, respectively.

Our comprehensive examination yielded a total of 4,631 records, representing CALL-based articles published across 63 peer-reviewed journals, featuring 186,589 references, with a validity rate of 98.0%. This exhaustive scrutiny spanned the period from 1980 to 2021, resulting in a robust and well-rounded data set for our analysis.

3.4 Checking inconsistency in spelling

The data underwent an initial assessment in CiteSpace to identify and rectify misspelled author names, study titles, publication sources, and issue and volume numbers. Data derived from CiteSpace were organized by author names and then transferred into an Excel sheet. Discrepancies in author names, publication sources, and study titles were thoroughly screened and rectified. Corrections were applied to both RIS and CSV files to ensure the uniformity of the data set, recognizing that differences in spellings could lead to distinct entities (see supplementary materials C1, C2, C3).

Additionally, we scrutinized authors’ keyword inconsistencies and acronyms by importing the CSV files into VOSviewer, conducting word co-occurrence analysis based on authors’ keywords. Subsequent to this analysis, data were extracted from VOSviewer into a CSV file, highlighting instances where similar keywords were being counted as separate occurrences. For instance, variations like CALL, computer-assisted language learning, computer assisted language learning, and computer assisted language learning (call) were harmonized under the unified term computer assisted language learning. Near-synonymous terms such as learning and acquisition, flipped classroom and flipped learning were consolidated. Singular and plural forms of words like learner/learners, collocation/collocations, and wiki/wikis were combined (supplementary material C4).

The final refined data were converted into XLSX format to facilitate subsequent analysis. The sources of publication underwent refinement in both the publication title and the references section for each study. This comprehensive review was conducted by cross-referencing the initial data extracted from CiteSpace (supplementary material C2). For example, the journal Computer Assisted Language Learning was referenced under various names such as CALL, CALL Journal, Computer Assisted Language Learning, Computer Assisted Language Learning Journal, and Computer-Assisted Language Learning.

3.5 Data analysis

We employed CiteSpace for cluster identification and VOSviewer for keyword co-occurrence and co-citation analysis. We used Excel to calculate publication source h-indexes and CiteSpace for structural and temporal metrics, including modularity Q, silhouette scores, and centrality. Our study focused on identifying hotspots in CALL research, utilizing timeline views, citation bursts, and analyzing productivity and co-citation frequencies of authors and journals (see supplementary material D for further explanation).

4. Results

4.1 Timeline view

Due to the significant surge in publications over the last five years (2017–2021), resulting in a substantial 1,892 articles (40.74% of the total), and in alignment with a recent scientometric study that particularly emphasized the latest time frame of published articles (Solmi et al., 2022), we adopted a division in the DCA process. This division entails two distinct timespan sets: the first from 1980 to 2021 and the second focusing solely on the last five years (2017–2021).

4.1.1 1980–2021 period

Initially, the analysis time frame was set from 1980 to 2021. However, CiteSpace calculated the nearest time for co-citation and omitted the empty intervals, thereby altering the previously established time frame to 1987–2021. CiteSpace settings were thus modified to the following parameters, with the intent to maximize the number of possible results: link retaining factor = −1, look-back years = −1. CiteSpace identified 154 clusters containing 1,447 nodes and 11,826 links for CALL research, with a modularity Q of 0.4585 and an average silhouette of 0.7807. We limited the visualization to major clusters, resulting in 10 clusters that encapsulated the trending research and hotspot issues over the 42-year span of CALL research. These are summarized in Figure 1, with detailed information represented in Table 1 and supplementary material E1.

Figure 1. Co-cited references network for the period 1980–2021 generated by CiteSpace 6.1 R3

Table 1. Clusters information (1980–2021 time frame)

Note. LLR = language learning research; SCMC = synchronous computer-mediated communication; CALL = computer-assisted language learning.

^a Label has been overwritten using “user-defined cluster label” feature in CiteSpace.

4.1.2 2017–2021 period

To highlight recent work, the time frame was specifically set to 2017–2021. The CiteSpace settings were restored to their default values (link retaining factor = 3, look-back years = 8) to ensure the returned results were representative of the last decade. This configuration resulted in a network comprising 27 clusters. The visualization in CiteSpace was confined to the 12 major clusters, as depicted in Figure 2. This network consists of 394 nodes and 1,462 links, with a modularity Q of 0.6751 and an average silhouette of 0.8629. The information regarding these 12 major clusters within the 2017–2021 time frame is concisely summarized in Table 2 and supplementary material E2.

Figure 2. Co-cited references network for the period 2017–2021 generated by CiteSpace 6.1 R3

Table 2. Clusters information (2017–2021 time frame)

Note. LLR = language learning research.

Figures 3 and 4 illustrate the timeline view of the clusters for the two specified time frames (1980–2021 and 2017–2021), categorizing them by burstness and centrality. Within this representation, the nodes of the clusters are dispersed across a timeline, demarcating the commencement and conclusion of each cluster. Red tree rings are indicative of burstness; outer purple rings encircle nodes that exhibit higher betweenness centrality.

Figure 3. Timeline view (by burstness and centrality) for the period 1980–2021 generated by CiteSpace 6.1 R3

Figure 4. Timeline view (by burstness and centrality) for the period 2017–2021 generated by CiteSpace 6.1 R3

CiteSpace was employed to execute burst detection, a method to identify publications receiving heightened attention in CALL. Figures 5 and 6, extracted from CiteSpace, display the top 10 references characterized by the most robust citation bursts within the two defined time frames.

Figure 5. Top 10 references with the strongest citation bursts (1980–2021 time frame)

Figure 6. Top 10 references with the strongest citation bursts (2017–2021 time frame)

In supplementary materials G1 and G2, we highlight references with the highest centrality and sigma scores for the respective time frames (1987–2021 and 2017–2021). Specifically, we focus on Levy’s (1997) book Computer-Assisted Language Learning: Context and Conceptualization, a reference marked by the second-highest betweenness centrality score (0.10) and the largest sigma score (3.45). Furthermore, Figure 5 reveals that Levy’s work was ranked as the fifth-strongest citation in terms of burstness (13.02) and had the third-longest span of burstness, extending from 1999 to 2009.

In the analysis of the 2017–2021 time frame (supplementary material G2), we focus on the last decade by adjusting CiteSpace settings, as previously described. In particular, the work of Reinders and Wattana (2014) on readiness to communicate during digital game play emerged prominently. This reference is characterized by the highest sigma score (3.45) and one of the top centrality scores (0.14), underlining its seminal contribution and influence at the intersection of technology and language learning research. Furthermore, as depicted in Figure 6, it ranks as the fifth most impactful reference in terms of burstness, with a concentrated burst lifespan from 2018 to 2019.

In a subsequent phase of analysis, we employed VOSviewer to examine authors’ keyword co-occurrences (see Figure 7), scholarly production trends, and journal co-citation frequencies within this study (see Figure 8). Additionally, we computed h-index values for the sources of publication using Microsoft Excel.

Figure 7. Authors’ keywords co-occurrences’ mapping

Figure 8. Mapping of the co-citation frequency of sources of publication

The findings from the VOSviewer analysis indicate that the most frequently used author keyword is computer assisted language learning, occurring 342 times (Table 3). This underscores the central theme of the field under examination. Following closely, computer-mediated communication ranks as the second most utilized keyword with 188 occurrences. This term encompasses various technological innovations and online platforms dedicated to language learning and has gained prominence over time (Hubbard & Levy, 2016). Among the top 10 recurrent author keywords, writing, a fundamental language skill, appears in sixth position with 90 instances, highlighting the significant focus on the development of writing skills. Additionally, MALL occupies the seventh position, emphasizing its emergence as a research hotspot in the past decade: it represents the second-largest cluster in the first period and the largest cluster in the second, demonstrating a particular emphasis on the application of mobile devices for language learning.

Table 3. Top 10 authors’ keywords co-occurrences

The outcomes of the co-citation and production analysis pertaining to the sources of publication manifest the clear predominance of CALL-core journals. Four of the top five most productive journals align with the CALL-core classification (Table 4), whereas only three positions within the ranks of the most frequently co-cited journals are reserved for CALL-core journals (supplementary material F). LL&T emerges as the most productive and concurrently the most highly co-cited journal, amassing 18,045 citations, a co-citation frequency of 5,400, and an h-index of 75. Computer Assisted Language Learning, with 15,493 citations, a co-citation frequency of 5,389, and an h-index of 57, occupies the second rank in both production and co-citation. Despite Computers & Education not being classified as a CALL-core journal and possessing the fewest documents among the top five (258 documents), it attains the third rank in both productivity and co-citation, as evidenced by 10,585 citations, a co-citation frequency of 3,971, and an h-index of 52. Additional CALL-core journals within the top five regarding production include ReCALL and CALICO Journal, with 9,356 and 7,456 citations, and h-indexes of 51 and 41, respectively. Conversely, the rankings of the fourth and fifth most frequently co-cited journals deviate slightly from those associated with the most productive journals. CALICO Journal and The Modern Language Journal constitute the fourth and fifth most highly co-cited journals, with 3,457 and 3,433 occurrences, respectively (see supplementary material F). Despite The Modern Language Journal not being a CALL-core journal and its focus on applied linguistics, it has secured a position among the top five highly co-cited journals. Likewise, System was ranked as the sixth most productive journal in terms of citation frequency and number of published documents (see Table 4). A more nuanced discussion of these results will be provided in the Discussion section.

Table 4. Top 12 most productive journals with h-index ranked by number of citations

5. Discussion

5.1 Trending research issues

We have successfully identified 23 major clusters corresponding to two distinct timespan periods: (1) 1980–2021 and (2) 2017–2021. These clusters serve as indicators of the most prominent research areas at the intersection of technology and language learning. In particular, one of the most extensively explored topics in CALL research is the development of writing skills, which resonates with the findings of Choubsaz et al. (2023) and Gillespie (2020), underscoring writing as the most frequently investigated skill in CALL literature. This thematic focus was dominant across the two periods (details are given in supplementary material H). This emphasis on writing-related issues is also manifested in the prevalence of relevant keywords used by authors. Notably, articles addressing technology-assisted L2 writing have garnered substantial citations, as evident from burstness analysis, thereby accentuating the profound impact of the writing theme on the CALL research landscape. This prevailing trend highlights scholars’ great interest in leveraging state-of-the-art technologies, including the prominent AWE software – a distinct trending cluster (cluster #8) – to enhance students’ writing accuracy and fluency. A plausible explanation for the prominence of writing within the research themes, as suggested by one reviewer, is the accessibility of text production for both learners and researchers. In contrast, the investigation of speaking skills presents greater complexity, requiring additional transcription of oral production to text: such processes can create barriers to research and analysis. On the other hand, research pertaining to reading and listening skills tends to be more inferential in nature. The underlying challenge is that these skills are internal cognitive processes, often opaque to direct observation and measurement. Consequently, understanding the precise degree to which reading and listening occur within a given context becomes a more elusive task, relying on inference rather than direct assessment.

Within writing clusters, another significant research theme gaining prominence in CALL research is “collaborative writing.” Cutting-edge technological devices have provided learners with the capability to engage in collaborative writing, vocabulary sharing, brainstorming of ideas, and refining revisions, thereby facilitating real-time interaction and simultaneous contributions among learners (Zenouzagh, 2022). This emergent trend is also grounded in robust theoretical frameworks, encompassing socio-cultural theory, the noticing hypothesis, and IA. The enduring influence of these foundational theories is seen through citations of seminal works by pioneering scholars such as Vygotsky and Cole (1978), Ellis (2003), and Long (1996). This scholarly interest reflects the evolving exploration of collaborative writing, underpinned by these theoretical foundations. It is worth mentioning that the emphasis on writing as a hotspot in research was not identified in prior bibliometric analysis conducted by Chen et al. (2021) or a scientometric study carried out by Lim and Aryadoust (2022). This could potentially be attributed to the restricted data set used by Chen et al. (2021), which relied on WoSCC data – a database encompassing fewer journals compared to Scopus. Moreover, even though Lim and Aryadoust (2022) utilized the Scopus database, their study’s scope was confined to just 11 sources of publication within the domains of CALL and educational technology.

Another prominent issue observed in CALL research is MALL, which aligns with the increasing utilization of smartphones as predominant technological learning tools over the past decade (Chun, 2016). Our study reveals that leveraging mobile devices to facilitate language learning constitutes a prominent area of focus within CALL research. MALL facilitates personalized and convenient language learning experiences, ensuring equitable access for all learners, thereby augmenting motivation and overall satisfaction (Nami, 2020). This emphasis on MALL is particularly notable during the second period (2017–2021), where it emerges as a predominant theme within cluster #0, ranking as the major cluster during that time frame. It also surfaces in the earlier period as cluster #1, labelled cell phone. Significantly, Stockwell’s (2010) seminal work on MALL exhibits the second most substantial citation burst within CALL literature during the 2017–2021 period (as depicted in Figure 6). The fervent interest in MALL research can be attributed to the mobility and portability offered by such devices, effectively transforming language learning from a controlled setting into a ubiquitous endeavor. This transformation empowers language learners to access learning materials anywhere and at any time (Stockwell, 2016). Our findings corroborate the insights of Chen et al. (2021), indicating that MALL held significant influence in CALL research during the period of 2000–2009. This prominence can be attributed to the advent of disruptive technology, particularly smartphones, which emerged as a transformative force during the early years of the 21st century.

SCMC emerges as a significant focal point as cluster #3. An associated cluster that aligns with SCMC is discerned in the 2017–2021 period, labeled using wikis, which ranks as cluster #7. This observation underscores the pivotal role of computer-mediated interaction in scaffolded language learning and the facilitation of negotiation of meaning. Interestingly, this using wikis cluster has also been identified in previous scientometric and bibliometric analyses (Chen et al., 2021; Lim & Aryadoust, 2022). However, Lim and Aryadoust (2022) indicated that wikis constituted one of the less explored research domains within CALL literature. Conversely, other studies have recognized wikis as a trending research issue within CALL literature (Chun, 2016; Hubbard & Levy, 2016). The robust theoretical foundations of SCMC are anchored in Long’s (1996) IA, a pivotal framework guiding CALL research. Furthermore, another notable cluster encompassing both time periods is data-driven learning, occupying clusters #5 and #10. This cluster emphasizes a learner-centered approach, wherein the learner assumes the role of the main investigator, the teacher serves as the facilitator of the language learning process, and computer software functions as the informant (Friginal, 2018). Additional clusters that emerged during the 2017–2021 period include second life, learning motivation, and vocabulary learning. These findings collectively shed light on the evolving landscape of research topics within CALL literature.

We have explored several research clusters that were identified either as the least researched issues or as having never been investigated in the previous CALL syntheses. In particular, intercultural competence, denoted as the fifth-largest cluster in CALL during the first time frame, explores the intersection of culture and CALL, a concept termed “computer-assisted Languaculture learning” by Chun (2016) (details in supplementary material H). Interestingly, Gillespie (2020) identified culture as one of the least explored areas in CALL research. The divergence in findings may be attributed to our study’s broader and more extensive data set. Our study also highlights the emergence of computer-assisted pronunciation training (CAPT) as a significant research cluster. CAPT has received substantial attention in educational technology journals, which explains its absence in previous research syntheses such as Gillespie (2020) and Choubsaz et al. (2023), which relied on CALL-core journals.

In the second period, several new clusters surfaced not previously identified in CALL syntheses. Among these, the flipped classroom approach, particularly in the realm of L2 writing characterized by complexity and accuracy, emerged as the third-largest research cluster, garnering significant attention due to highly cited references and articles emphasizing its efficacy in improving L2 writing skills. This contrasts with Choubsaz et al. (2023), who identified the flipped classroom as one of the least explored topics in CALL research. Again, this discrepancy is attributed to Choubsaz et al.’s narrower data set, relying exclusively on CALL-core journals. Another significant issue identified in the second period is immersive technology, termed second life by CiteSpace, ranking as the fourth-largest cluster. This cluster predominantly explores the use of 3D VR in L2 learning and teaching (details in supplementary material H).

5.2 Publication metrics

The findings of the present study unveil the considerable influence exerted by certain highly cited references, particularly those boasting elevated centrality scores, on the CALL research community. In particular, a significant majority of these cited works stem from books. Illustrative examples of these highly referenced publications encompass Vygotsky and Cole’s seminal socio-cultural theory (1978), Levy’s book Computer-Assisted Language Learning: Context and Conceptualization (1997), Krashen’s influential input hypothesis (1982), and Nation’s pivotal contribution to L2 vocabulary learning (2001) – detailed in supplementary material G1. Scholars in CALL thus appear to favor reference books when citing theoretical models and conceptual frameworks. This preference likely arises from the comprehensive scope that books offer, a quality that often surpasses that of journal articles, particularly in the realms of social sciences and humanities (Kousha, Thelwall & Rezaie, 2011). Coinciding with our study findings, Zhu and Aryadoust (2023) corroborate the tendency of highly cited references within the domain of translation to emanate from books. This trend is explained by authors’ proclivity to reference theories within their respective fields, a proclivity that is notably fulfilled by books. An additional factor contributing to the prevalence of book references is their robust representation within Scopus, our and Zhu and Aryadoust’s (2023) principal data source. Consequently, this enhanced representation within Scopus may potentially augment the frequency of citations directed towards book references.

In terms of journal production and co-citation, LL&T and Computer Assisted Language Learning, both core CALL journals, emerged as the leading contributors in citations and h-indexes. Notably, Computers & Education, a significant journal in educational technology, ranked third in citations and held an impressive h-index of 52. This is noteworthy, as the journal primarily focuses on technology in education rather than solely on CALL topics. It suggests a clear preference among CALL authors for high-impact-factor journals, aligning with Smith and Lafford’s (2009) findings. Our study also identified five major educational technology journals and two applied linguistics journals, System and Foreign Language Annals, among the top influential journals for the CALL research community. Additionally, three distinct clusters in co-cited sources of publication were revealed, representing CALL, educational technology, and applied linguistics. This intricate mapping illustrates that CALL research thrives at the intersection of applied linguistics and educational technology.

6. Conclusion

This study presents a comprehensive scientometric analysis of CALL research spanning over four decades. Our primary objectives were to identify evolving research trends over time and to provide quantitative insights into publication venues using structural and temporal metrics, focusing particularly on the period 2017–2021, marked by a significant surge in publications. The findings of this analysis reveal that a multitude of research clusters related to L2 writing have consistently dominated the CALL literature during both time frames. These clusters signify a sustained exploration of various technological interventions aimed at enhancing students’ writing skills. This emphasizes the need for CALL researchers to not only delve into advanced technologies for improving writing but also extend their investigations to other language skills. In addition to the predominant L2 writing clusters, this study identifies several other prominent clusters within CALL research, including MALL, SCMC, data-driven learning, and Second Life. These clusters represent substantial concentrations of inquiry within the CALL field at present. Despite their potential significance, emerging technological advancements such as artificial intelligence (AI) applications appear to be underrepresented in existing CALL literature. This identified gap in research underscores an intriguing inconsistency and signals potential avenues for future academic exploration and growth. The integration of cutting-edge technologies, including AI, holds promise for expanding and enhancing the field of CALL.

Furthermore, our examination of publication metrics in this study offers valuable guidance to CALL researchers seeking influential publication outlets. It provides a direction for selecting impactful journals for disseminating their CALL research effectively. This information can aid researchers in making informed decisions about where to publish their work, thus contributing to the ongoing development and dissemination of knowledge within the CALL community.

7. Limitations and suggestions for future research

The current study, while extensive, may still not fully represent the breadth of CALL literature. It relies on the Scopus database, potentially excluding studies indexed elsewhere or published in languages other than English falling outside our key terms. Future research could address these limitations by diversifying data sources and including a broader range of document types. Another limitation is that quality new journals need time to be covered by Scopus, such as JALT CALL journal, which was indexed in 2015 while its first issue was published in 2005, resulting in a loss of many articles to be covered by Scopus. Other journals’ articles were also excluded from Scopus; for example, CALICO Journal articles published in 1997 were not indexed. While citation metrics may indicate good quality of the sources of publication, there might be bias in citations as some of the cited work could be criticized rather than credited (Xu et al., in press). Another bias for citations is that authors tend to cite journals with high metrics or might self-cite their own work, or some editors might increase citations of journals by augmenting editorial documents. We limited the search to 15 educational technology journals, which could bias against other journals in the field of technology and education of which we are not aware.

In our study, we attempted to cover key terms representing CALL research but must acknowledge limitations. Among others, some terms that may resonate within the field, such as technology-enhanced language learning (TELL), may have remained absent from our designated key terms. Although we targeted articles containing the phrase language learning in educational technology journals where technology was a component and included searches with the term technology within the category of applied linguistics, it is conceivable that numerous articles remained undetected because of the absence of specific terminology such as TELL. The omission of emerging technological terms also constrains our study’s comprehensiveness. These limitations present opportunities for future research to expand upon our findings. Additionally, our identified clusters did not include trending themes like AI and immersive technology. This gap is not a dismissal of their relevance but highlights an unexplored area in the literature. CALL researchers are encouraged to explore these technologies, recognizing their potential to advance language learning and teaching. These findings offer a reflection on the field’s current state and a guide for continued exploration in CALL.

Although the primary objective of this study was to identify major research areas explored within the CALL community, future syntheses in CALL could examine specific features such as research designs employed in the field. This would enable the identification of robust methodologies and suggest ways to enhance scientific rigor, thereby contributing to the generalizability of research findings. Another emerging trend worth investigating is the incorporation of advanced technologies like AI, machine learning, and NLP. These technologies hold significant potential for enhancing L2 learning and warrant thorough empirical scrutiny. Additionally, another promising avenue for future research could involve a comparative analysis of trending research topics as covered by core CALL journals and those in the broader field of educational technology.