Digital multimodal composing in L2 classrooms: A research agenda

Abstract

Research on digital multimodal composing (DMC) in second language (L2) classrooms has proliferated considerably in recent years, to a large extent in response to the changing digital and multimodal communication landscape. This article offers a research agenda on DMC in L2 classrooms. We begin with a theoretically oriented overview of DMC scholarship. We then examine seven research themes for future research inquiry, from which we draw seven research tasks. The seven themes are: (1) the effectiveness of DMC for L2 writing development; (2) DMC task design; (3) L2 teacher education/training for implementing DMC; (4) feedback practice for DMC; (5) DMC assessment; (6) collaborative DMC as a translanguaging space; and (7) the deployment of DMC for critical digital literacies. Throughout the article, we refer to interdisciplinary scholarship and methods from multimodality, L2 writing, composition studies, new literacy studies, language teacher education, and computer-assisted language learning. The seven research tasks represent what we see as the essential next steps for understanding DMC, which is a young domain that has great potential to advance L2 language and literacy education in the digital age.

1. Introduction

Digital multimodal composing (DMC) has emerged as a prominent teaching and learning practice in second language (L2) classrooms over the past two decades. The emergence of DMC is a response to the paradigmatic shift in language and literacy education in L2 classrooms, where writing has been reconceptualised as a process of multimodal design that is frequently mediated by digital technologies (Belcher, 2017; Lankshear & Knobel, 2006; New London Group, 1996). DMC as a pedagogical approach has been defined as “a textual practice that involves the use of digital tools to produce texts by combining multiple semiotic modes that include, but are not limited to, image, word, and soundtrack” (Jiang, 2017, p. 413). Examples of DMC include various forms of video documentaries, presentation slides, podcasting, infographics, webpages, or digital stories, among others (Hafner, 2014; Jiang et al., 2024; Smith et al., 2022). Thus, DMC can be seen as an emerging umbrella term that covers the use of a wide range of new forms of representation in pedagogical contexts, including such practices as digital storytelling and digital writing. The body of DMC research has been steadily growing across L2 contexts since the pioneering use of digital multimodal storytelling for underprivileged L2 learners by Hull and Nelson (2005) and Nelson (2006).

DMC practices in L2 classrooms have received increasing scholarly attention, especially over the last decade or so. In 2013, a symposium in TESOL Quarterly highlighted the notion of digital composition in a second or foreign language as one way to reconceptualise reading and writing in the digital age (Hafner et al., 2013). Since then, a number of special issues in prominent journals in the field have addressed issues of digital multimodal forms of expression in English language teaching and learning. These include, among others: special issues of TESOL Quarterly on multimodality (Early et al., 2015) and digital literacies (Darvin & Hafner, 2022); special issues of the Journal of Second Language Writing on L2 writing in the age of computer-mediated communication (Li & Storch, 2017) and multimodal composing (Yi et al., 2020); and a thought-provoking debate on multimodal composing in the disciplinary dialogues section of the Journal of Second Language Writing (Xu & Matsuda, 2017). These efforts will continue with a special issue of Computers and Composition on DMC in the era of artificial intelligence (Gilje et al., forthcoming). Over the years, research on DMC has evolved into a dynamic domain of inquiry that straddles multiple areas, including L2 writing, composition studies, new literacy studies, language teacher education, and computer-assisted language learning.

Owing to continued developments in digital technologies and evolving literacy expectations in L2 classrooms, we foresee that the domain of DMC research will continue to flourish. To move the domain forward, the purpose of this article is to present a forward-looking research agenda by detailing seven research themes and related tasks that we consider to be the most pressing for L2 researchers and practitioners. The development of the seven research tasks in this article is informed not only by our engagement with DMC research and practice but also by previous reviews of DMC scholarship, including Li and Akoto's (2021) systematic review of existing research foci (i.e. L2 students' composing process, perceptions, and benefits), the systematic review of DMC studies in secondary classrooms by Smith et al. (2020), the systematic review of DMC studies in post-secondary contexts by Zhang et al. (2023), and Lim and Kessler's (2022) review of prior L2 studies on multimodal writing from a second language acquisition (SLA) perspective, as well as their recent research timeline article (Lim & Kessler, 2023). Different from previous reviews, we followed a two-step procedure in developing the seven research themes. First, we focused on what remains controversial in the field by synthesising the inconclusive and contradictory research findings in extant literature. As the effectiveness of DMC for L2 writing development emerges from this synthesis as an oft-disputed theme, we reinforce the pressing need for more research in this area. Second, we categorise the remaining research areas identified as in need of further exploration into six additional themes, thereby extending Lim and Kessler's (2022) work in three main ways. First, Lim and Kessler's (2022) review suggested four areas for future research (i.e. language development, multimodal writing process, multimodal text analysis, and student needs/practices) but was informed only by literature published prior to 2020. We include the most recent publications (up to 2023) and highlight the under-researched but much needed issues of pedagogic design, teacher education, feedback, assessment, collaboration, translanguaging, and critical digital literacies. Second, in addition to the cognitive perspective proposed by Lim and Kessler (2022), we add sociocultural and critical perspectives as theoretical foundations for further research on DMC. Third, we highlight the need for more research on DMC to examine the development of critical digital literacies to resist implicit forms of digital manipulation and power embedded in new digital platforms/structures and to reconstruct a more equitable and inclusive online space for literacy learning. In the next section, before presenting the research tasks in detail, we first outline the trajectories of the conceptual underpinnings of DMC scholarship.

2. Conceptual trajectories for researching DMC

To understand what theoretical perspectives may inform further research on DMC, an overview of the epistemological trajectories surrounding DMC is necessary. To date, DMC research has been informed by four major theoretical approaches, including the multimodal, sociocultural, cognitive, and critical approaches to language and literacy education.

From its inception, DMC has been introduced into L2 classrooms as an activity for language learning and teaching by applied linguists (e.g. Nelson, 2006; Shin & Cimasko, 2008), who are often aligned with a social semiotic perspective on multimodality as an approach to communication, seeing language as just one among many semiotic resources that are required to make meaning (Kress, 2010). DMC is seen as a process that culminates in the creation of a multimodal ensemble incorporating linguistic, visual, aural, gestural, and spatial modes, among others. To understand the literate process of multimodal meaning-making within and across modes, theoretical constructs such as poly-semiotics (the spontaneous combination of two or more semiotic systems), semiotic remediation (the various ways that semiotic performances/materials are re-represented and rewoven across modes/media), visual metadiscourse (the visual cues and markers that connect various visual elements to meet the visual needs of audiences), and remixing (the combination of cultural artefacts into new creative blends) have been adopted by various L2 scholars (Hafner, 2015; Knobel & Lankshear, 2007; Li et al., 2023; Prior & Hengst, 2010; Toohey & Dagenais, 2015). A systemic functional approach to multimodal discourse analysis (SF-MDA), focusing on the grammars of semiotic resources to understand meanings arising from semiotic choices and combinations (Jewitt et al., 2016; Unsworth, 2006), has also been adopted to guide the analysis of multimodal choices, intermodal relations, and meaning constructions during the process of DMC (Shin et al., 2020).

DMC research has also been informed by sociocultural approaches to language and literacy education. Such approaches focus on examining the social, cultural, and historical contexts of DMC, which is often conceptualised as a site for socially embedded and situated literacy practices (Lantolf & Thorne, 2006; Street, 1984; Vygotsky, 1978). Along with the New Literacy Studies tradition, relevant literacy frameworks for DMC research include multimodal literacies (Jewitt & Kress, 2003), new literacies (Gee, 2010; Lankshear & Knobel, 2006), digital literacies (Jones & Hafner, 2021; Weninger, 2023), and multiliteracies (New London Group, 1996). These frameworks collectively refute the autonomous model of literacy as decontextualised reading and writing and highlight the sociocultural contexts and material worlds of digital multimodal composers. A sociocultural approach theorises DMC as goal-directed social practices that are infused with composers' presentation of self and identity development (Lim & Toh, 2020). Such an approach contributes to DMC research by enabling researchers to examine the role of DMC in, for instance, creating spaces for identity expression, developing multiliteracies, and bridging the gap between in-school academic learning and students' vernacular literacy practices and lived experiences.

More recently, there has been a growing body of research on DMC undergirded by cognitive perspectives on SLA, including the noticing hypothesis (Schmidt, 1990) and the cognitive process of writing theory. The noticing hypothesis posits that awareness of a gap between input and output is necessary for SLA to occur, and DMC researchers (e.g. Dzekoe, 2017) have used this hypothesis to show that DMC can help L2 student writers to notice linguistic problems in their writing. The cognitive process theory takes writing as “a set of distinctive thinking processes which writers orchestrate or organize” (Flower & Hayes, 1981, p. 366) and has been applied to exploring the cognitive process of DMC (Tan, 2023). The DMC research underpinned by these cognitive perspectives mostly theorises DMC as a cognitive process involving a range of functional skills. The cognitive perspective allows DMC researchers to examine in detail the cognitive process and composing behaviours involved in DMC, as well as the relationship of these behaviours with L2 language and literacy development.

A small number of recent DMC studies have also been informed by critically oriented theories of literacy and identity, including critical media literacy theory (Kellner & Share, 2005) and identity investment theories (Norton, 2013). These critical perspectives focus on issues of power and ideology, analysing the politics of representation of gender, race, class, and sexuality with the purpose of exploring how power, media, and information are linked (Kellner & Share, 2005). Such a perspective has informed studies examining L2 students' representation of their critical understanding of the power and politics of English through DMC (Huang, 2012, 2019). The sociological concept of investment (Darvin & Norton, 2015) has been used to explore how often-marginalised learner identities can be empowered through DMC in L2 classrooms (Jiang et al., 2020; Kendrick et al., 2022). With the critical perspective, researchers (e.g. Cummins et al., 2015) have started to use DMC to engage minoritised L2 learners with identity text production for transformative literacy learning experiences.

The abovementioned theoretical advances and conceptual trajectories in DMC research indicate that the field is becoming ripe for more empirical and theoretically informed research in L2 classrooms. In the following sections, we illustrate how future research on seven key themes in DMC scholarship can be addressed with reference to seven research tasks, one per theme.

3. Research themes and research tasks

This section presents seven research themes, each of which gives rise to a research task that could contribute to the body of research on DMC in L2 classrooms. For each theme, we first provide an overview of relevant research, thus setting out the need for future research, before suggesting specific strategies that researchers can employ to accomplish each task.

3.1 Research theme: Effectiveness of DMC for L2 writing development

Over the years, substantial evidence has been reported in support of the value of DMC for many dimensions of L2 learning, including the development of multiliteracies and learner motivation, agency, autonomy, and identities (Hafner & Miller, 2011; Jiang & Luk, 2016; Kendrick et al., 2022; Lim et al., 2022a). Despite this evidence, the integration of DMC into L2 pedagogy is not without controversy. One prominent concern relates to the effectiveness of DMC in fostering L2 writing development and the pertinent debate about the instructional use of DMC. On the one hand, some scholars have argued that the use of DMC in L2 classrooms provides an effective way to address multimodal forms of expression that have gained prominence in the digital age (Belcher, 2017; Hafner, 2014; Yi, 2017). On the other hand, some L2 writing researchers and practitioners have expressed concerns, with one opining that “the dazzling multimodal discourse has lured students' attention away from the language in a text, shifting their conscious monitoring (in Krashenian sense (1981)) to other modes of representation” (Qu, 2017, p. 93). Such a critique echoes the conceptualisation of DMC as a distraction (Manchón, 2017) from “traditional” writing and questions the use of DMC as an asset-based pedagogy in L2 classrooms. To address these concerns, there is a need for further research on the effectiveness of DMC for L2 writing development. Such research would provide empirical evidence on the use of DMC to foster writing development, which is a key goal in L2 classrooms (Lim & Kessler, 2022).

However, compared with the affective and sociocultural benefits (e.g. motivation, identities) of DMC, research on the linguistic benefits of DMC remains limited (Kim et al., 2023). Studies of DMC seemed to have often drawn on qualitative methodologies to focus on students' reported perspectives and observed experiences (e.g. Hafner & Miller, 2011; Pham & Li, 2022), which did not intend to explore whether and how students develop their linguistic writing skills through DMC. Meanwhile, the results obtained from the limited research on the impact of DMC on L2 writing outcomes remain inconclusive. On the one hand, some studies have reported a positive impact of DMC on the writing performance of L2 students. For example, Vandommele et al. (2017) investigated the effects of a DMC intervention in in-school and out-of-school leisure settings on the development of writing skills among L2 Dutch beginner learners. They noted that compared with the non-intervention group, the in-school group showed growth in complexity, text length, content, and communicative effectiveness and the out-of-school group revealed greater gains in syntactic complexity, lexical diversity, and text length. A similar positive impact of DMC on the writing performances of L2 student writers has also been reported in the U.S. and Chinese contexts (Dzekoe, 2017; Tan, 2023; Xu, 2021). On the other hand, studies in the Korean context have reported no significant differences in accurate clause rate, content and language alignment, or amount of reflection in writing outcomes between DMC and traditional writing (Cho & Kim, 2021; Kim & Belcher, 2020). Mixed perceptions regarding the benefits of DMC for improving writing skills were also reported in Kim and Belcher's (2020) study involving 18 university students. However, in a recent quasi-experimental study of 41 Korean university English as a Foreign Language (EFL) students, in which the researchers compared the writing development of students doing traditional writing tasks with another group doing DMC, Kim et al. (2023) reported greater gains in content and language for the DMC-integrated writing group. The contradictory findings may result from the use of different writing genres and writing quality measurements in the different studies. In addition, the inconclusive results are further compounded by limited sample sizes and small effect sizes. Given that L2 writing teachers are more likely to incorporate DMC into their language teaching classrooms if there is clear evidence of its benefits for traditional, language-focused writing development, more research on the linguistic benefits of DMC is warranted. This leads us to the first research task, as follows.

Research task 1

Explore the linguistic benefits of DMC by examining the effectiveness of DMC in promoting L2 writing development.

To undertake this research task, cognitive theories of SLA can be used to conceptualise how and why DMC may contribute to L2 writing development. For instance, the noticing hypothesis can be used to illuminate how the integration of visuals into DMC can help L2 student writers to notice the linguistic and rhetorical issues in their writing (Dzekoe, 2017). The theory of learning transfer (DePalma & Ringer, 2011) offers a comprehensive framework for DMC researchers to analyse the linguistic features of writing performance with DMC and examine how L2 student writers transfer learning and knowledge across print-based writing and various forms of DMC (Kang, 2022). Cognitive process theory (Flower & Hayes, 1981) also provides a framework to compare the cognitive processes underlying DMC and traditional monomodal writing (Tan, 2023). Researchers can draw upon such theoretical frameworks to explicate the cognitive mechanisms underlying DMC and the impact of DMC on L2 writing development.

In terms of research methods, a quasi-experimental design with pre- and post-tests would be helpful to investigate the impact of DMC on L2 writing performance and development. Kim et al.'s (2023) quasi-experimental study comparing the effects of traditional monomodal writing activities and DMC activities in English for Academic Purposes (EAP) classrooms offers a good example of how the linguistic benefits of DMC can be documented in a systematic way. In this study, students were assigned to one of two conditions: the first using traditional, monomodal writing instruction and the second using DMC-integrated writing instruction. The researchers first compared linguistic performance in outputs of traditional writing and DMC assignments, using measures of text length, complexity, and accuracy. They used timed TOEFL-style pre-tests and post-tests to evaluate language development in the two groups based on content, organisation, and language scores using an analytic rubric. As this line of research is at present limited by the relatively small sample sizes drawn from a single L2 context, replication studies with larger samples across a range of contexts are warranted. Quasi-experimental studies focusing on a wider range of L2 writing development measures are also needed to counteract the limited effect sizes in existing studies.

3.2 Research theme: DMC task design

Tasks are foundational and continuously evolving in L2 classrooms (Swales & Feak, 2023). Despite the increasing implementation of DMC in L2 classrooms, research on DMC task design is in its infancy (Li & Zhang, 2023). Here, “task design” refers to both process and product: the processes that learners are asked to navigate as well as the products that they are asked to design. To date, very little is known about what elements need to be included in a DMC task for it to be successful or whether different kinds of L2 learners (e.g. different age groups, different L2 proficiency levels) respond to elements of DMC task design (e.g. collaborative versus individual work) in different ways. There is also a need for further research on what forms of pedagogical scaffolds have been adopted in practice and how the use of these scaffolds can contribute to efficacious DMC task designs and outcomes in L2 classrooms. DMC involves very complex and fluid processes, the effects of which are mediated by a range of elements, including task scaffolds (e.g. pre-task storyboarding, grouping), learner training, composition tools, and L2 learners’ multimodal preferences. Therefore, to customise DMC task designs for different learning needs, it is necessary to identify the possible permutations of DMC tasks and test these permutations with different types of learners. The extent to which generative artificial intelligence (e.g. ChatGPT) becomes a part of or is incorporated into DMC tasks has also become an interesting question for DMC researchers and practitioners. One important long-term goal of this line of inquiry is to develop theoretical heuristics and models that can explain differences in DMC task designs and scaffolding frameworks across contexts as well as any foundational commonalities. This leads us to the second research task, as follows.

Research task 2

Explore the dimensions of DMC task design and how different dimensions affect learning outcomes.

To identify valid, evidence-based task design options for DMC, it will be useful to first conceptualise DMC task and its design. With reference to previous work on DMC tasks (Hafner, 2013; Kim et al., 2022), the concept of a DMC task can be defined as a task in which L2 learners construct meanings by incorporating various modes of communication. The task outcome can include a variety of genres, from digital video scientific documentaries to infographics. DMC tasks, often conceptualised in the literature as projects, tend to be complex, comprising a range of sub-tasks at various levels that often require a lengthy period of time to complete. Accordingly, DMC task design can be taken as the design, scaffolding, and implementation of DMC activities throughout the pre-task, during-task, and post-task stages.

Throughout the three major task stages, important dimensions to consider in DMC task design can be conceptualised with reference to the work of Grapin (2019), Jiang et al. (2022a), Liang and Lim (2021), Lim and Polio (2020), and Pacheco et al. (2021), among others. Based on these works, we suggest that DMC task design entails a number of dimensions, including genres (e.g. video documentary or photo story), processes (e.g. collaborative or individual work), purposes (e.g. creative, disciplinary, critical), scaffolds (e.g. pre-task storyboarding), choice of linguistic modes (e.g. written or spoken), other semiotic modes (e.g. visual, soundtrack), composing tools (e.g. WeVideo, ChatGPT), and composing topics (e.g. curricular topics or student-initiated topics). Among these dimensions, scaffolds may be the most essential and complex. According to Pacheco et al.'s (2021) systematic review, scaffolds for DMC tasks fall into seven major categories: collaboration, direct instruction, exemplar texts, translanguaging, discussion, encouragement, and questioning. These categories and dimensions offer a starting point and a workable checklist for future explorations into DMC task design in different L2 classrooms.

In terms of research methods, a sequential mixed-methods design can be usefully adopted. First, large-scale surveys can be conducted to understand existing practices of DMC task design, as observed in authentic L2 classrooms. The survey items can be designed with reference to the conceptual framework of DMC task dimensions described above. For instance, L2 teachers can be surveyed on whether and how they have integrated generative artificial intelligence technologies such as ChatGPT into DMC task design, and if so, at what stages, for what purposes, and with what impact on new forms of digital genres in L2 classrooms (Hafner et al., 2023). Second, longitudinal single-case studies or studies with multiple-case designs can be conducted with purposively selected innovative teachers to search for trends and commonalities in DMC task design across a variety of L2 classrooms. Data can be collected through in-depth interviews with teachers, lesson observations, think-aloud protocols as teachers plan and design DMC tasks, and post-lesson reflections. Based on within- and cross-case analyses, empirically grounded frameworks for DMC task design in L2 classrooms can be synthesised and developed for larger-scale intervention purposes. Third, evaluation studies testing the impact and learning outcomes of various dimensions of DMC task design can be conducted through quasi-experimental designs with pre- and post-tests. The perceptions of participating teachers and students regarding the efficacy of various permutations of DMC task designs can also be examined as additional data to supplement the evaluation of how different dimensions influence learning outcomes.

3.3 Research theme: L2 teacher education

To implement DMC in L2 classrooms, teachers need to feel competent and empowered, leading some researchers to suggest that the success of DMC pedagogy rests on effective teacher preparation (Jiang & Ren, 2021; Li, 2020). However, L2 teachers have often reported a perception of being under-prepared in this respect (Mills & Exley, 2014; Tan & McWilliam, 2009; Yi & Angay-Crowder, 2016). In contrast with the many studies that have been conducted on the use of DMC for student learning, relatively few studies have attended to the issue of DMC from the teachers' perspective. Indeed, teacher expertise in general remains almost uncharted terrain in research on L2 writing (Hirvela & Belcher, 2022; Swales & Feak, 2023). Studies on teacher preparation for DMC have examined how pre-service teachers can benefit from DMC assignments that are integrated into their teacher training programmes (Li, 2020), but they have not considered in-service teachers and how engaging with DMC might help such teachers to learn, change, and develop the competences needed to implement DMC in L2 classrooms. With respect to the implementation of DMC in the L2 classroom, teachers need to develop an appreciation of the need and rationale for DMC instruction, sound multimodal communicative competence and semiotic awareness, and sound DMC pedagogical content knowledge. As such, to better prepare L2 teachers to use DMC effectively as an instructional strategy, it is necessary to understand how they develop the relevant expertise, in a number of dimensions. These dimensions include teachers' perceptions of DMC, of their own multimodal communicative competence compared with their actual competence as observed in classroom practice, and of their ability to implement DMC compared with their actual ability as observed in classroom practice. This leads us to research task 3, as follows.

Research task 3

Explore the dimensions of L2 teachers' expertise in developing multimodal L2 competence and relevant pedagogical content knowledge by engaging with DMC.

A challenge in undertaking research task 3 is to conceptualise what expertise in using DMC entails for L2 teachers. To grasp the target expertise/competences required for L2 teachers to learn to use DMC, one should first consider expertise “as something in motion”, rather than as “a fixed or finished product” (Hirvela & Belcher, 2022, p. 5). Following this process-oriented conceptualisation, DMC researchers can apply the adaptive expertise model (Hatano & Inagaki, 1986; Lee & Yuan, 2021), which distinguishes routine expertise from adaptive expertise, to explore whether and how L2 teachers “engage in an ongoing exploration and expansion of what they know and thus make adjustments in how they teach” (Hirvela & Belcher, 2022, p. 4) through DMC. More importantly, to theorise what comprises expertise in using DMC, researchers can draw on theories of multimodality (Kress, 2010) and multimodal literacies (Jewitt & Kress, 2003), which can explain how particular semiotic resources are used to achieve rhetorical effects (Hafner, 2015; Yi et al., 2020). As such, knowledge of the semiotic principles of design, multimodal pedagogy, semiotic awareness, mindsets, attitudes, and emotions (Tour, 2015) can be helpful in unpacking specific dimensions of L2 teachers' expertise in fostering multimodal L2 competences through DMC.

Among the many options for undertaking research task 3, we recommend action research by L2 teacher educators, who can integrate DMC into their teacher education programmes and then examine how various dimensions of L2 teachers' expertise in fostering multimodal L2 competences can be expanded and developed. In the initial stage, large-scale surveys can be conducted to explore L2 teachers' perceptions of the value of DMC, understanding of multimodal L2 competence, and perceived ability to foster such competence. The survey results can better inform the design of teacher education programmes through DMC for L2 teachers. Various cycles of action research with planning, acting, observing, and reflecting stages (Burns, 1999) can then be implemented to explore different dimensions of L2 teachers' development of expertise in using DMC. To uncover the nuanced process of how L2 teachers respond to, adapt, and develop expertise regarding DMC use, case studies can also be constructed to focus on one or multiple teachers. Data can be collected through classroom observations, multiple in-depth interviews, reflections on teaching, and teaching materials prepared by teachers. The findings can be used to illuminate how DMC can be better utilised to promote teacher learning and engender the desired multimodal L2 competence and DMC pedagogical content knowledge. Studies that have been conducted on teacher engagement with DMC (Jiang et al., 2021; Lim et al., 2022b) and on teacher strategies to cope with the challenges of using DMC (Jiang et al., 2022b; Lim & Unsworth, 2023) offer a good starting point for further research along this line. Li's (2020) research on incorporating two DMC projects (i.e., a mid-term multimodal reflection assignment and a multimodal instructional material design task) in two online graduate courses to develop preservice language teachers' knowledge and skills of digital multimodal pedagogy presents a successful example of preparing pre-/in-service teachers to cross the gap between traditional literacy and multiliteracies. It thus offers a good model for future DMC researchers. Altogether the long-term goal is to provide valuable context-based insights into the essential expertise and competences in relation to DMC use to establish important learning outcomes of L2 teacher education programmes.

3.4 Research theme: Feedback practice on DMC

Feedback is an essential element of using DMC in L2 pedagogies but scant attention has been paid to the feedback practices associated with DMC. This contrasts markedly with the voluminous body of research on L2 writing feedback. Despite the increasing use of DMC in L2 classrooms and the fact that DMC expands the traditional written focus to include other modes of expression (e.g. visual, aural) in text construction, studies suggest that many L2 practitioners still focus primarily on language and replicate the feedback practices found in traditional, “monomodal” L2 writing assignments (Elola & Oskoz, 2022; Jiang et al., 2022b). In other words, there appears to be a predominant focus in feedback provision on the linguistic aspect of DMC, with non-linguistic modes sidelined or ignored. According to Oskoz and Elola (2020), the reasons for this solely linguistic focus for feedback on DMC include a lack of guidelines for L2 teachers to provide DMC feedback, a lack of curricular objectives that take DMC as learning outcomes, and a lack of familiarity with DMC.

Given that digital multimodal texts “are here to stay and will increasingly populate diverse academic contexts” (Elola & Oskoz, 2022, p. 591), the role of L2 teachers as feedback providers should be “radically changing while, at the same time, becoming more critical” (Caws & Heift, 2016, p. 133). Elola and Oskoz (2022) argued that to adapt feedback for DMC, a traditional focus on language should be maintained and expanded into a more “multifaceted, comprehensive, and modern feedback method” (p. 577). In search of such a method, Elola and Oskoz (2022) conceptualised a list of components and guiding questions to be considered for offering feedback on DMC by L2 instructors. Along this line of inquiry and to build a solid empirical basis for recommended DMC feedback practices, we suggest that there is a need for in-depth inquiries into the practices and perceptions of L2 teachers, particularly those with innovative experiences of incorporating different forms of DMC tasks in their teaching. Important questions that can be asked include the following: To what extent do teachers engage with multimodal communicative competence in their feedback on DMC? If they provide sensitive and effective feedback on students' use of multiple modes, what kinds of conceptual tools do they make use of? These questions lead us to research task 4, as follows.

Research task 4

Explore L2 teachers' feedback practices with respect to DMC by examining whether and how they engage with multimodal communicative competence in addition to providing linguistic feedback.

To examine whether and how L2 teachers have expanded their feedback practices on DMC beyond the traditional focus on global (i.e. content, structure, organisation) and local issues (e.g. grammar, lexis) of language use, researchers may first find it useful to conceptualise multimodal communicative competence by drawing on theories of multimodality grounded in social semiotics (Kress, 2010; Oskoz & Elola, 2020), systemic functional multimodal discourse analysis (Jewitt et al., 2016), and multiliteracies (New London Group, 1996). Social semiotic theories can help define multimodal communicative competence as the ability to arrange “available meaning-making resources into a multimodal whole, making authorial decisions appropriately for specific audiences and purposes” (Shin et al., 2020, p. 2). In addition, scholars of multimodality have drawn attention to the way that different modes (linguistic, visual, gestural, etc.) provide different affordances for meaning-making and interact with each other in a range of ways. Multimodal communicative competence entails an awareness of such affordances and interactions and of how these vary within and across modes and media through processes of transformation (the actions that reorder meanings within one mode) and transduction (the reorganisation of meanings across modes). By targeting such a multimodal communicative competence, L2 teachers can give more specific feedback on students' communication within and across modes, attending not only to linguistic issues but also to the way that L2 learners shift meanings across modes when necessary.

Existing rubrics and frameworks to evaluate DMC can serve as starting points to explore L2 teachers' DMC feedback practices. For instance, Hafner and Ho's (2020) process-oriented model can be used to examine whether and how L2 teachers offer feedback across the planning, editing, reflecting, and sharing stages of DMC. The rubric, devised by Jiang et al. (2022a) for a genre-based model, offers additional conceptual tools to understand how feedback on global (e.g. audience, purpose, rhetorical structure) and local (e.g. quality of base unit, layout) issues of DMC can be combined. Elola and Oskoz's (2022) comprehensive digital story rubric is also helpful in conceptualising how feedback may have been offered with respect to traditional linguistic issues, audience expectations, choice of semiotic resources, and authorship, among other dimensions. As well as what teachers are providing feedback on, it is also important to consider when and how they are doing so to gain an understanding of the entire feedback process.

To achieve a comprehensive understanding of DMC feedback in practice, we foresee the value of large-scale cross-sectional studies of L2 teachers' feedback practices across a wider range of classroom contexts. To ascertain the focus of feedback, such studies can examine teachers' feedback (e.g. on planning documents, drafts, and final DMC products) with reference to the criteria described above. For determining teachers' rationale, students' expectations of teachers, and the impact of feedback on student practices, surveys and interviews will be needed. The principles of grounded theory (Creswell, 2007) can be applied to yield a comprehensive understanding of L2 teachers' concerns and practices regarding DMC feedback in context. To dig deeper and observe processes over a period of time, case studies of L2 teachers with unique feedback practices can be conducted, making use of classroom observations and interviews. Furthermore, a phenomenological approach can be adopted to explore L2 teachers' experiences and struggles, if any, in expanding the feedback focus from linguistic to multimodal communicative competence in DMC. Undertaking research task 4 can lead to the design of a more coherent model/framework for DMC feedback.

3.5 Research theme: DMC assessment

Another important issue that remains under-researched is the question of how to assess DMC. Compared with the widely accepted L2 writing assessment rubrics that are made available by major language testing organisations, such as TOEFL and IELTS, there have been only a few tentative models and guidelines for assessing DMC (e.g. Elola & Oskoz, 2022; Hafner & Ho, 2020; Jiang et al., 2022a). Some of the existing DMC assessment models, such as the genre-based model (Jiang et al., 2022a), also tend to be limited to the products of one genre of DMC (i.e. video essays), with just a few categories (i.e. purpose, base unit, layout, navigation, rhetoric) included in the rubrics. It remains unclear whether these rubrics would work for other genres of DMC and whether additional categories would be needed. Given that DMC covers a variety of genres and purposes that differ significantly from those of traditional L2 writing tasks, there is a need for a suite of DMC assessment options that can be flexibly adjusted for different circumstances and contexts (Zhang et al., 2023). Furthermore, given that a DMC task normally involves multiple stages, it is clearly not sufficient to focus assessment on products alone. Research suggests that a process-oriented model of assessment would be beneficial, and this raises issues of when and what to assess (Hafner & Ho, 2020). It also remains unclear what assessment methods would promote self-assessment and peer assessment among student composers. Finally, the literature has tended to portray L2 teachers as mostly lacking preparedness and resistant to assessing DMC, with rather limited documentation of the assessment practices of those innovative teachers who are integrating various forms of DMC into their classrooms. Focusing on the experience of these teachers could provide a fruitful avenue for developing and evaluating innovative DMC assessment methods. To address these research gaps, we suggest research task 5, as follows.

Research task 5

Develop and evaluate innovative methods to assess DMC processes and products and underlying multimodal communicative competence.

To undertake research task 5, two levels of conceptualisation may be needed. First, DMC assessment researchers need to conceptualise the what in assessing DMC processes and products. As noted above, theories of multimodality (Jewitt et al., 2016; Kress, 2010) and multiliteracies (New London Group, 1996) can serve to theorise multimodal communicative competence and the related notion of semiotic awareness (Towndrow et al., 2013). This assessment orientation would promote a strong version of DMC that considers both linguistic and non-linguistic modes of expression to be equally important to learning and communication in L2 classrooms (Lim & Polio, 2020). Given the wide range of curricular objectives in L2 classrooms, a weak version of DMC can also be embraced in DMC assessment, with non-linguistic modes serving supporting roles for L2 development (Grapin, 2019). DMC assessment can be taken as a continuum with the strong and weak versions at each pole to allow for the flexibility to account for possible differences in L2 teachers' curricular objectives when integrating DMC. In addition, the construct of visual metadiscourse (such as information value, framing, connective elements, graphic elements, and fonts) (D'Angelo, 2016; Li et al., 2023) can be drawn upon to evaluate students’ multimodal communicative competence in drawing upon various metadiscourse resources to engage and inform the audiences of their multimodal compositions.

The second level of conceptualisation concerns the how of DMC assessment. To account for a range of potential purposes or principles of DMC assessment, researchers may find it useful to refer to the three major perspectives on classroom assessment practices: assessment of learning (AoL), assessment for learning (AfL), and assessment as learning (AaL) (Earl, 2003). The purpose of AoL is to measure learning outcomes for summative and administrative reasons. AfL carries a strong formative orientation, and its priority is to promote student learning (Williams, 2009). AaL extends the role of formative assessment for learning but highlights students as the critical connector between learning and assessment, encouraging students to exert self-regulation over their learning processes and stressing the importance of students becoming their own evaluators. As such, peer assessment and self-assessment of DMC among students can be taken as important aspects of AaL. These three approaches can be drawn upon to determine the purposes and principles of DMC assessment across contexts and to evaluate DMC assessment methods. Important questions that can be asked include the following: In what ways can AoL and AfL be used as strategies to assess DMC? What methods are needed to integrate AaL into DMC assessment?

In searching for innovative methods of DMC assessment, researchers can consider three specific strategies: the top-down approach, the bottom-up communal approach, and the situated approach. The top-down approach involves developing innovative assessment methods with reference to theories of multimodality. As one example, systemic functional multimodal discourse analysis (SF-MDA) (Jewitt et al., 2016) could be helpful in assessing how various modes are deployed to achieve ideational, textual, and interpersonal functions in DMC products. The bottom-up communal approach involves students in co-creating assessment methods and rubrics to evaluate DMC (Adsanatham, 2012). Advocates for this approach hold that assessing DMC is not only a concern for instructors but also a crucial skill for students to develop as successful writers and composers. Finally, a situated approach would focus on the practices of innovative L2 teachers in situated contexts and explore how they assess the processes and products of DMC. Such an approach to research design has been proven in earlier studies (e.g. Hafner & Ho, 2020; Jiang et al., 2022a) to help in offering contextualised evidence for the development of practice-based and teacher-friendly assessment models for DMC.

Armed with the above conceptualisations, DMC researchers can make use of collaborative action research and work with teachers to integrate the three major classroom assessment perspectives (i.e. AoL, AfL, AaL) into L2 instruction. The top-down approach can be combined with the bottom-up communal approach when developing innovative methods to integrate, for instance, AoL, AfL, or AaL, into the assessment of DMC processes and products. To develop a suite of assessment options that can work for L2 teachers across a range of contexts, large-scale surveys of L2 teachers' assessment practices can be conducted with the purpose of identifying best practices, problems, and needs regarding DMC assessment. As a follow-up, L2 teachers who are willing to develop and evaluate innovative methods of assessment for DMC can be identified and case studies in context can be conducted. Relevant data include observations of assessment processes, semi-structured interviews, and stimulated recall sessions in which teachers share how they have assessed particular DMC products (see, e.g., Hafner & Ho, 2020). Based on within- and cross-case analyses, best practices and practice-based principles and methods of assessing DMC processes and products can be synthesised for further large-scale intervention purposes.

3.6 Research theme: Collaborative DMC as a translanguaging space

With the notion of writing reconceptualised as multimodal composing, traditional collaborative writing research (Storch, 2011) needs to be expanded to include collaborative DMC, elsewhere referred to as multimodal collaborative writing (Li & Zhang, 2023). There has been growing attention to collaborative DMC in recent years, but the focus has been mostly on how students have distributed tasks during DMC, the benefits of DMC for L2 learning motivation, collaborative partnerships, and differences between collaborative and individual DMC (Hafner & Miller, 2019; Smith, 2019). In other words, prior studies have mainly explored the benefits and patterns of collaborative DMC from the cognitive/SLA and sociocultural perspectives, paying scant attention to critical aspects of DMC. Alongside the continuation of such research, investigations are also needed into such critical issues as who collaborates with whom, what and how they learn from each other, and what impact collaboration has on learning opportunities and participation structures in L2 classrooms, particularly for those who are often labelled as deficient in traditional writing. Translanguaging, considered “an all-encompassing term for diverse multilingual and multimodal practices” (Li, 2018, p. 9), offers an important lens because prior studies (Ho, 2022; Mendoza et al., 2023) show that translanguaging can create a space for students of diverse backgrounds to adapt to, value, respect, and learn from each other's plural, multilingual, and multimodal repertoires. However, constructing a translanguaging space in L2 classrooms can be challenging. To tackle this challenge, we recommend conducting more research into collaborative DMC as a translanguaging space, which is a social space where multilingual and multimodal repertories interact, collaborate, and co-create new meanings (Li, 2018). In making this suggestion, we do not deny the need for more research on collaborative DMC and on DMC as a translanguaging space as separate topics but highlight the importance of bridging the two as one prominent expansion of traditional research on collaborative writing in the current digital and multimodal environment.

Given that a translanguaging space is not “a space where different identities, values, and practices simply co-exist, but combine together to generate new identities, values, and practices” (Li, 2011, p. 1223), it is important to explore how collaborative DMC can contribute to new identity generation and development and what that might mean for L2 learners, particularly those who are marginalised owing to their socioeconomic status. This question is important because, in view of the increasing linguistic and cultural diversity among students, students' access to DMC can be uneven and their participation in collaborative DMC cannot be taken for granted. Previous research (e.g. Smith, 2019) has shown that inequalities in digital competences and multimodal life experiences can constrain digitally and linguistically disadvantaged students to the extent that they contribute little more than an assistive role during collaborative DMC. Although studies (e.g. Cummins et al., 2015) have revealed the potential of composing identity texts to validate the home literacies of young immigrants and empowering their classroom participation, little attention has been paid to how they collaborate with mainstream students during the process of multimodal composing and the pertinent implications for education justice (Cope & Kalantzis, 2023). In the translanguaging space constructed by collaborative DMC, it is therefore important to ask who collaborates with whom and why, who benefits the most, and how varied collaboration practices shape different learning opportunities (Smith et al., 2022). To empower disadvantaged and marginalised students to go beyond assistive roles and to understand how students use multimodal resources as they collaborate/interact with each other, more research on collaborative DMC as a translanguaging space is warranted. This leads us to research task 6, as follows.

Research task 6

Evaluate collaborative DMC as a translanguaging space by examining how students of different backgrounds learn from each other within such a space.

To carry out this research task, DMC researchers can conceptualise collaborative DMC through the lens of translanguaging and co-learning as developed by Li (2014, 2018). Li (2014) discussed the importance in multilingual L2 classrooms of co-learning, defined as “a process in which several agents simultaneously try to adapt to one another's behaviours so as to produce desirable global outcomes that would be shared by the contributing agents” (p. 169). Li (2018) further highlighted that for co-learning to occur, it is important for teachers and students to “go between and beyond socially constructed language and educational systems, structures, and practices to engage diverse multiple meaning-making systems and subjectivities” (p. 24). Researchers can also consider theories of translanguaging and trans-semiotising (He & Lin, 2023; Lin, 2019) to analyse how multimodal resources contribute to the creation of texts and contexts for the co-learning of language and literacy in L2 classrooms.

Design-based research can be conducted to develop designs of collaborative DMC that effectively create a translanguaging space in which all students' linguistic and cultural repertoires are valued as meaningful resources for co-learning. University–school partnerships can also be set up for collaborative action research between DMC researchers and school instructors regarding the adoption and implementation of collaborative DMC as a translanguaging pedagogy for more equitable co-learning among diverse learners. Prasad and Lory's (2020) research with elementary schools in the U.S. on collaborative multilingual bookmaking offers an excellent reference for future researchers seeking to enact translanguaging pedagogy through DMC across a wider range of contexts in support of diverse learners' equitable participation in collaborative DMC. To evaluate collaborative processes in DMC, researchers can draw on a case study approach to provide an in-depth analysis of students' collaboration and co-learning and the changes, if any, that occur during the DMC process. Existing approaches to collaborative writing offer useful frameworks that can be modified to examine the interactions of pairs and groups of students. For example, the conventional language-related episodes (LREs) framework (Swain & Lapkin, 1995, 1998) can be expanded to DMC-related episodes (Kim et al., 2022) to understand how learners work together to negotiate multimodal meanings, all the while constructing a translanguaging space for collaboration among diverse learners. By identifying such DMC-related episodes, researchers can examine how students utilise diverse languages, modes, and other resources when collaborating with others. Where student participants include socio-economically disadvantaged groups, such as those with immigrant/refugee backgrounds, it would be particularly illuminating if their choices of composing tools, languages, modes, resources, and partners were recorded and if their collaboration patterns were traced across various DMC projects over time. Such information would help to illuminate the power of collaborative DMC as a translanguaging space to empower diverse groups of L2 learners.

3.7 Research theme: Deployment of DMC for critical digital literacies

For the final research theme, we turn to the teaching and learning of critical digital literacies (Darvin, 2017; Mirra & Garcia, 2020; Weninger, 2023), which is another prominent yet under-represented area of socio-political research in digitally mediated language education that has evolved dynamically over the past decade. Critical digital literacies go beyond skills of digital technology use to include practices that enable learners to examine how power operates across various contexts of technology use and challenge those power mechanisms through agentive actions and online participation (Darvin & Hafner, 2022). In the post–COVID-19 pandemic era, increased use of technology has become a new normal in language and literacy education. At the same time, there are ongoing crises of disinformation, online misconduct, and new invisible forms of social exclusion and marginalisation (Cope & Kalantzis, 2023; Dooly & Darvin, 2022). Such crises lay bare problems in students' critical digital literacies with not only discerning the veracity of what they read and write online but also creating counter-narratives of marginalised communities who are often misrepresented in social media (Lee, 2023; Mirra & Garcia, 2020). To tackle these problems, scholars have highlighted critical digital literacies as an essential goal in L2 classrooms. Across the globe, critical digital literacies are high on the education and policy agenda and are often regarded as “an essential attribute of a discerning citizenry and a competitive workforce” (Weninger, 2023, p. 1).

Research has shown that DMC provides valuable opportunities for students to engage with social justice–oriented digital activism and civic participation, which are core to critical digital literacies development (e.g. Amgott, 2018; Jiang, 2021, 2023; Mirra & Garcia, 2020). For instance, in his study of the composing of photo journals and video narratives by multilingual youths during a digital innovation project, Jiang (2023) revealed three new forms of digital civic literacy practices in their DMC activities. These critically oriented literacy actions were disclosing the unequal access of members of ethnic minorities to recess space during the pandemic, devising creative solutions to increase the public acceptance of pandemic control–related apps, and popularising specialist knowledge for healthy quarantine. Despite the illuminating findings, this line of inquiry remains limited and needs to be further validated across a wider range of L2 classrooms. Although a growing number of DMC projects have been implemented in L2 contexts, prior studies have mostly treated DMC as a form of creative, or disciplinary, expression that allows students to exert agency for individual identity expression and display (Lim & Polio, 2020); very few have engaged students in the critical exploration of social issues and critical digital literacies. To address this research gap, we propose research task 7, as follows.

Research task 7

Explore the influence of DMC on the learning and development of critical digital literacies through classroom-based interventions.

To undertake research task 7, DMC researchers can first conceptualise the notion of critical digital literacies as sets of power-laden social practices rather than merely decontextualised competences/skills (Darvin & Hafner, 2022; Jones & Hafner, 2021; Weninger, 2023). Drawing on a social practice orientation, both digital literacy and critical literacy can be integrated into DMC to facilitate students' agentive actions for change and improvement through authoring digital multimodal texts that engage with issues of power and so serve to develop critical digital literacies (Pangrazio, 2016). Accordingly, the multifaceted construct of critical digital literacies can be operationalised as, for instance, “skills and practices that lead to the creation of digital texts through which students explore personal identity, interrogate the world, and enact civic participation for the betterment of communities and social equity” (Jiang & Gu, 2022, p. 1030). With this operationalised understanding, the role of DMC in promoting the learning and development of critical digital literacies can be specified through equity-oriented actions of creating/authoring digital multimodal texts. Furthermore, considering increased online surveillance and the power of online influencers, it is imperative to investigate how DMC can help empower learners to resist and, if possible, transform manipulative and exploitative practices in online discourses and spaces (Darvin, 2023).

Classroom-based interventions on using DMC for critical digital literacies can include longitudinal collaborations between researchers and L2 instructors for the planning and implementation of DMC projects, following the collaborative action research paradigm (Burns, 1999). Useful data can be drawn from semi-structured interviews, teacher and student journals, classroom observations and recordings, and documents pertaining to literacy practices and actions around student-authored multimodal compositions. Telling cases (Yin, 2009) can be constructed to analyse whether and how multilingual students develop critical digital literacies over time. Multimodal discourse analyses can be conducted upon student-authored multimodal compositions and the relevant literacy activities. Such analyses can contribute to an understanding of the working mechanisms of DMC in promoting critical digital literacies development and help reveal the inhibiting and facilitative factors for such development. The compelling cases of L2 students' enactment of civic participation and digital activism through DMC presented by Jiang (2021, 2023) offer an initial example. Darvin and Zhang's (2023) discussion of how the production of a YouTube video explaining lexical gaps between Cantonese and English helped Cantonese first language (L1) learners assert their identities as legitimate multilingual speakers and resist colonial ways of knowing offers another point of reference for further research endeavours along this line of inquiry.

A scaled-up possibility for research task 7 may involve multi-site interventions and longitudinal tracing of critical digital literacies development across a series of DMC projects. Multi-site research can complement the understandings gleaned from single-site studies. A team of researchers and schoolteachers from a range of classroom contexts could be involved for a wider impact on the critical use of digital tools and multimodality for social justice–oriented literacy actions. A possible focus could be implementing the same set of DMC projects and materials in different schools. Such research could have far-reaching educational implications and can explain how the same DMC project is interpreted, appropriated, and adapted depending on the local situation (e.g. school policies, teacher identity, learner backgrounds). Multi-site studies can also reveal how student and teacher agency in using DMC for critical digital literacies is distributed and evolves across time and space. Collectively, these research works can help sustain and amplify DMC as an important site of the transformative use of literacy for social goals.

4. Conclusion

As a relatively young domain in L2 language and literacy research, DMC has attracted considerable interest over the past two decades. Given that digital and multimodal literacy practices remain an important element of the contemporary communicative landscape, DMC is likely to continue to attract attention from researchers and practitioners. In this article, we show how DMC can be conceptualised from multimodal, cognitive, sociocultural, and critical perspectives. After highlighting various, and sometimes disputed, perspectives on DMC use that have recently emerged in the field, we propose seven research tasks in relation to writing development, task design, teacher education, feedback, assessment, translanguaging, and critical digital literacies. The seven research tasks highlighted in this article are by no means exhaustive of all the possible directions for further DMC research in the field of L2 language and literacy education. However, these research tasks provide an outline of what we see as the critical and essential next steps for developing DMC research into a robust and theoretically grounded domain of inquiry in applied linguistics and L2 education. We hope the research agenda presented here will inspire others to embark on DMC projects, thus moving this promising area forward and helping to frame the terrain and trajectory of DMC research in increasingly diverse L2 classrooms.