Crossing a Virtual Divide: Wargaming as a Remote Teaching Tool

Abstract

In Fall 2020, political science instructors at the Massachusetts Institute of Technology (MIT) and the Naval Postgraduate School (NPS) partnered to conduct a virtual-learning wargame centered on Taiwan. This article presents its design and execution along with the results from pre- and post-game surveys and interviews with the participants that were conducted to measure the achievement of its learning objectives. The game conduct and empirical results demonstrate two main findings. First, wargames are effective tools of active learning that aid in classroom instruction and grab the attention of students—even over Zoom—in a way that traditional methods of instruction do not. Second, wargames can bridge gaps between different fields. The MIT–NPS wargame tackled the civil–military divide by bringing together military officers at NPS and academics from MIT. These results show that wargaming holds promise as a bridge-building tool of instruction that can engage students, scholars, and practitioners in achieving positive learning outcomes.

In Fall 2020, political science instructors at both the Massachusetts Institute of Technology (MIT) in Cambridge, Massachusetts, and the Naval Postgraduate School (NPS) in Monterey, California, faced a thorny set of challenges common to all political science instructors confronted by the ongoing COVID-19 pandemic. First, how should teaching proceed with the normal business of learning in abnormal times? Second, how best can instructors bridge the divide between students from different backgrounds in different places? Third, how can instructors foster an active-learning experience when the phrase of the day was “Zoom fatigue”?

To address these questions, teachers at MIT and NPS partnered to conduct a virtual crisis simulation, or wargame. This article presents the design and execution of that wargame, along with the results from pre- and post-game surveys and interviews with the participants that were conducted to measure the achievement of its learning objectives through self-reported measures.

Our experience during the game and empirical results demonstrate two main findings. First, wargaming is an effective tool of active learning that aids in classroom instruction, especially in a remote setting. The immersion and intensity of wargames grab the attention of students—even over Zoom—in a way that traditional methods of instruction do not. Second, wargames can bridge gaps between different fields. Our wargame tackled numerous divides by bringing together officers at NPS and academics from MIT from a variety of demographic backgrounds, academic specialties, and military ranks. Our results show that wargaming is a valuable addition to any instructor’s toolkit that can be applied online and across disciplines.

WHAT IS WARGAMING?

Wargames are immersive simulations used for two broad purposes: analysis and pedagogy (Goldblum, Reddie, and Reinhardt 2019; Lin-Greenberg, Pauly, and Schneider 2021). Historically, wargames of both types have been played across the US defense establishment and academia (Pauly 2018; Perla and McGrady 2011, 113; Schechter, Schneider, and Shaffer 2021, 514). Both analytical and pedagogical games have several common features. Peter Perla, an expert in wargaming, defines a wargame as “a model involving people making decisions in a synthetic environment of competition or conflict, in which they see the effects of their decisions on that environment and then get to react to those changes” (Perla 2022, 199).

Central to Perla’s definition is that wargames change over time in response to input by the players, creating the possibility for feedback loops. This renders wargaming suitable for the study of complex problems (e.g., armed conflict) because it can model scenarios with multiple decision makers, conflicting streams of information, and competing objectives (Schechter, Schneider, and Shaffer 2021, 1). Wargames are “games” because the simulation is propelled forward by players taking actions, or moves, within the game (Brewer and Shubik 1979, 8). This allows players to experience “how” a decision is made and the “challenges and solutions” generated by the crises and their actions (Goldblum, Reddie, and Reinhardt 2019).

Analytical games are undertaken to reach generalizable findings and can be used, for example, to develop military doctrine (Development, Concepts and Doctrine Centre 2017, 5–6). The Prussian Army developed the first analytical wargames, called Kriegsspiel, in the nineteenth century and gained acclaim after that nation’s subsequent military victories (Sabin 2012, 31). In the United States, the canonical example of analytical wargaming was the effort in the 1920s and 1930s by the US Navy to develop War Plan Orange, a military approach for a potential war with Imperial Japan (Perla and McGrady 2011). In a postwar lecture at the Naval War College, Admiral Chester W. Nimitz famously stated: “The war with Japan had been reenacted in the game rooms at the War College by so many people, and in so many different ways, that nothing that happened during the war was a surprise—absolutely nothing except the Kamikaze tactics towards the end of the war” (Snyder 1989, 51).

Experimental games, a subset of analytical games, are more like traditional political science experiments using randomization to maximize internal validity. Recent experimental games have been run by scholars to test hypotheses about nuclear use and cyber exploits (Schechter, Schneider, and Shaffer 2021); the escalation risk of drone warfare (Lin-Greenberg 2019); and agricultural-disease management (Clark et al. 2020).

In contrast, pedagogical games are focused on the player experience and do not attempt to produce generalizable findings. The goal of pedagogical wargames is to capitalize on the immersive nature of the games that engages students to improve learning outcomes (Alme and Hvidsten 2022; Fielder 2022). Having a role in the game as a player forces students to participate critically and think about complex scenarios with a goal beyond simple memorization (Schechter, Schneider, and Shaffer 2021, 6). One of the first uses of gaming in education was to teach business policy, shifting away from the traditional case-study approach (Wolfe and Guth 1975). More recent academic literature focuses on the importance of measuring the utility of pedagogical gaming in a wide array of fields, from history (Loban 2021) to cybersecurity (Haggman 2019).

Wargames are quintessentially active-learning experiences, leading to far better learning outcomes than more-static teaching methods, such as lectures (Asal 2005, 359–66; Asal and Blake 2006; Brynen 2010; Hensley 1993; Kelle 2008; Newmann and Twigg 2000; Pallister 2015; Sabin 2012; Smith and Boyer 1996; Starkey and Blake 2001) Active learning has been shown to be significantly more effective than lecture-style learning for material retention and course performance—so much so that authors of experimental studies on active learning have argued that the control group should have been “stopped for benefit” due to the clearly superior efficacy of active learning (Freeman et al. 2014).

THE LEARNING OBJECTIVES

Our game expanded on recent pedagogical games; that is, the immersion and flexibility that wargaming provides allowed us to tailor a game suited to the needs of MIT and NPS students. NPS students were enrolled in an East Asian security course and MIT students were enrolled in a class on academic gaming; for both groups of students, the wargame was part of the course syllabus. The overlapping pedagogical interests of the instructors led to the following three main learning objectives:

• • First, we aimed to deepen players’ understanding of major international relations concepts, particularly credibility, alliance politics, and crisis management. These specific concepts flowed from the syllabus objectives of the NPS course. Given the subject matter of that course, the game’s objective was to teach about how these core concepts affected relations between the People’s Republic of China (PRC) and the United States in the context of an East Asian crisis scenario. NPS students prepared for the game by reviewing primary-source and academic secondary-source literature on the historical context of the PRC–Taiwan dispute and recent tensions between the United States and the PRC. MIT students prepared for the game by studying other gaming efforts to model political crises.

• • Second, we wanted the game to bridge the divide between civilian and military players while also teaching about the complications that civil–military relations present to both civilian diplomatic efforts and military-planning activities during a crisis. This learning objective was crucial considering the mixed civil–military backgrounds of our participants.

• • Third, we designed the game to introduce wargaming to students as an active-learning experience. For active-duty military officers at NPS, familiarity with wargaming is a given due to its prominence within the US Department of Defense for both pedagogical and analytical purposes. For MIT students, the game provided an opportunity to learn about the construction of pedagogical games, which then could inform their own efforts to build games as instructors. Given the difficulties of online education during the pandemic, a key goal of the game was to keep both groups of students actively learning.

GAME DESIGN

The design of the MIT–NPS wargame flowed directly from our learning objectives, allowing us to operationalize them. To fulfill our first learning objective, we chose to situate the game within a Taiwan crisis scenario. This choice allowed us to engage with and learn about crisis escalation, credibility, and alliance management in the context of US–PRC relations. Moreover, the choice of Taiwan provided a scenario with a real chance of military action but one that also involved civilian decision making as much as military planning.

We chose to feature four country teams: Taiwan, the PRC, the United States, and Japan. Including teams beyond only the United States and the PRC forced the players to engage with a more confusing mix of dyadic relationships, all of which carried the possibility for misperception and tension.

Before the game, MIT and NPS students were assigned brief background reading on the current situation in Taiwan and assigned to teams, but they were not told the specific crisis trigger until the game began. During the game, teams concurrently made decisions for each turn, and each turn took approximately one and a half hours of real time. Teams made decisions through group deliberations in a Zoom room. Although no player was assigned as the leader, players were assigned roles (e.g., US Secretary of Defense) that guided their role in decision making. For the first part of each turn, the team met together to discuss overall move goals, then broke into civilian and military sides to plan their moves. The whole team reconvened for the final portion of the turn to finalize their move. Splitting into civilian and military sides simulated a civil–military relations dynamic. Via email, teams submitted their move orders as documents to the adjudicators, who then decided between rounds how the game progressed based on the teams’ moves as well as actions of non-player entities. Move orders included, for example, moving military assets to assist in humanitarian aid and dispatching nuclear experts to assess radiation levels. The adjudicators—or “White Cell” in wargaming terminology—functioned as referees of the game and determined how the moves of different teams interacted and the effectiveness of specific actions.

Because the goal of the game was educational rather than analytical, adjudication decisions did not have to be based on a most-likely interpretation of what would happen in the real world; instead, they had to be both plausible and in service to the game’s pedagogical goals. The game lasted a total of eight hours conducted over two days, including an introductory session to brief players on the game scenario and rules, a post-game plenary session, and the actual turns.

Because the possibility of misperception was a key element of the game’s learning objectives, we used the remote nature of our wargame as an asset instead of a hindrance. The online nature of the wargame was an opportunity for the White Cell to control the communications environment more carefully than is possible during in-person games. The literature on pedagogy recommends clear and open communication among all parties for the best learning outcomes, especially during pandemic-related virtual learning (Daigle and Stuvland 2021; Glazier 2021; Loepp 2021; Ray 2021). However, wargame design poses a different set of problems. In many wargames, whether lines of communication between certain parties are open or closed is a deliberate part of the game design that mimics real-life misunderstandings and incomplete information. For example, in the commercial wargame Memoir ’44: Operation Overlord, theater commanders and field commanders have limited communication avenues, which mimics the challenges of pursuing a cohesive strategy in a massive engagement.

Because the possibility of misperception was a key element of the game’s learning objectives, we used the remote nature of our wargame as an asset instead of a hindrance.

In our game, national teams had separate Zoom rooms and could communicate with other teams only through five-minute summits monitored by representatives of the White Cell. Teams were limited to three summits per turn. Because they were corralled carefully within the digital framework, game participants were unable to communicate outside of game channels.

We wanted to limit the number of software platforms used and to avoid platforms with which participants were unlikely to be familiar. Therefore, we used Zoom for all audiovisual communications and Gmail for all text-based communications, such as turn updates from the White Cell and move orders by the various player teams.

To fulfill our second learning objective—to bridge the divide between civilian and military players—we made two key design decisions. First, we divided the PRC, the US, and Japanese national teams into civilian and military subteams to model civil–military dynamics. Given its more limited military capabilities vis-à-vis the other teams, we divided Taiwan into two teams based on political orientations rather than military and civilian subteams.

Second, we restricted communication not only between but also within national teams. Military and civilian subteams spent the first few minutes and the last part of each turn together but were separated for most of it. This separation forced subteams to agree on national goals at the beginning of turns but to pursue these goals in isolation from one another, thereby modeling real-life divisions of responsibility.

To fulfill our third learning objective—to introduce wargaming to students as an active-learning experience—we chose a unique trigger for our game. Wargames and crisis situations centering on Taiwan abound in both military and civilian gaming centers. To engage our players, we wanted to force them to “think on their feet” rather than trod familiar ground while still examining a plausible scenario that would model a realistic regional crisis. Instead of a cross-strait incursion, the spark for the crisis was a 7.9-magnitude earthquake centered near Taiwan that devastated Taipei and led to a Fukushima-like meltdown at a nuclear plant just outside of the capital city. The specific details of the spark created an intense time pressure for immediate action for each team, and the nature of the disaster allowed the White Cell to increase its severity when needed to keep players engaged. This plausible regional crisis also enhanced the students’ knowledge about regional political dynamics.

We wanted to keep teams small enough for lively discussion and to avoid large, unwieldy groups on Zoom. Additionally, we assigned each player a specific role and limited subteams to three to six participants. The player roles were designed to ensure that certain actions had a responsible party. Although we assigned players to roles, we did not mention specific names of real-world individuals, which might have limited participants from advocating for policies they personally favored or from using their own expertise. To ensure that all players were engaged and that team decisions were collaborative, heads of state were not present in the game.

FULFILLMENT OF LEARNING OBJECTIVES

The MIT–NPS wargame was conducted over two days in November 2020. To examine whether the game accomplished its three learning objectives, we administered pre- and post-game surveys. The surveys asked a series of multiple-choice questions that allowed us to compare responses from before and after the game. In addition, the post-game survey asked several questions about the experience of the game itself, including free-response queries. We supplemented the survey results with post-game interviews with multiple participants. We fully informed students of the purpose of the surveys and interviews, made participation entirely optional and confidential, and clarified that grades and course evaluations would not be affected in any way.

All of the multiple-choice questions discussed were five-point Likert scales. We coded “strongly agree” as 5 and “strongly disagree” as 1—with the intervening values coded appropriately—for basic quantitative analysis. Although assuming that Likert-scale responses are continuous data points rather than ordinal is somewhat inaccurate, the small number of survey responses (i.e., 35 in the pre-game survey; 29 in the post-game survey) precluded a more sophisticated quantitative analysis (Harris and Freeman 2023).

To provide p-values for significance, we used simple Welch’s two-sample t-tests. We conducted non-paired tests because of the differing number of responses in each survey. To keep survey responses anonymous, military affiliations were the only demographic data that we collected. Our pre- and post-game surveys were comparable on this dimension: the pre-game survey consisted of 26 military and nine civilian responses and the post-game survey consisted of 23 military and six civilian responses.

First Learning Objective: Deepen Understanding of Major International Relations Concepts, with a Focus on East Asia

The results of pedagogical games are important only in that they serve to advance the learning objectives; they do not predict how a real-world crisis would unfold. In our game, the PRC used disaster relief as a pretext to position military units on Taiwan and to stoke division among Taiwanese political factions. The United States struggled to coordinate effectively with its allies while also communicating with the PRC to control escalation risk. Japan attempted to walk the line between aligning with its most important ally while avoiding being overtaxed. The Taiwan teams vied for future positioning while trying to deliver needed aid to their devastated population. In summary, the players’ decisions led to a chaotic, confusing outcome with no clear winner—the exact type of outcome suited to teach about the messy reality inherent to any Taiwan crisis scenario.

Our surveys and interviews found substantial albeit indirect support for our first learning objective. We did not test students’ knowledge about specific concepts because we did not want them to feel as if they were being graded or that their participation in pre- and post-game surveys was not fully voluntary. To fulfill ethical requirements of human-subject research, we needed to draw a distinction between our own research efforts on the efficacy of gaming as a pedagogical tool (which had to be fully voluntary) and required material for the NPS and MIT courses. Instead, we relied on self-reported measures to gauge the effectiveness of wargaming as a teaching tool.

Between the pre- and post-game surveys, responses to the question, “I have a good understanding of the political and military factors that would affect how a Taiwan crisis scenario would unfold,” increased by 8.6%, with a p-value of 0.017. In the post-game survey, respondents answered positively to the question, “I feel like I learned a lot from the crisis simulation.” Only one respondent was neutral whereas all others answered “agree” or “strongly agree.”

Wargames can be a great pedagogical tool because they are immersive, but this very immersion may introduce bias: even students who learned nothing but had fun might respond positively to the previous question. However, in free-response answers, participants highlighted lessons learned about alliance dynamics, perception and misperception, and the difficulties posed by Taiwan’s special status. One civilian respondent answered that their biggest takeaway from the game was “the importance of attention for alliance maintenance: while the US and Japan appear to have patched things up at the end of the scenario, tempers really were high at one point.”

…in free-response answers, participants highlighted lessons learned about alliance dynamics, perception and misperception, and the difficulties posed by Taiwan’s special status.

In an interview, a military officer remarked how the game was directly relevant to their career path. The officer, whose next position was in an embassy in East Asia, stated:

I feel like I will definitely take the information with me, in terms of understanding the nuance between how China would behave regarding a Taiwan incident, and everyone worrying about not stepping on each other’s toes or losing face. Understanding how complicated that makes planning and execution of any humanitarian or military mission. I think I would walk into any crisis [with] a lot less confidence than before this wargame.

It is important that the officer stated that the game would make them less confident about approaching a real-world crisis scenario like that of the game. This result is in accordance with a learning wargame as opposed to an analytic game, in which the focus is not on reaching a generalizable finding but rather exploring the nuances of an issue. Adding to players’ overall confidence in crises is an expected outcome of wargames, which sometimes is desired and sometimes unwarranted (Allen 1987; Fine 1983). However, our learning game challenged players’ preconceived notions and helped them to delve into the nuances of Western Pacific political dynamics.

Second Learning Objective: Bridge the Real-Life Civil–Military Divide While Exploring the Complexities of Civil–Military Relations During a Crisis

Our measures found support for both civil–military learning objectives. On the one hand, participants found the communication barriers between the military and civilian subteams to be frustrating, demonstrating that the game design worked as intended to show the difficulty of civil–military coordination in crisis decision making. Whereas the average response to the question, “I feel that communication between my team and the other subnational team in my country was good,” was 3.79, in the free-response answers and interviews and during the game itself, participants repeatedly requested more communication channels between the civilian and military sides.

On the other hand, participants indicated that they collaborated across the real-life civil–military divide and learned from one another’s experiences. One military officer told us that the game allowed for more free and open communication than traditional instruction, remarking, “I think it was a fantastic collaborative environment. No uniforms, no ranks.” Furthermore, a civilian member of the US team noted that civilian scholars and military officers provided unique insights to their counterparts. “The civilian side would spin their wheels and talk about jargon. Author’s names, escalation dynamics,” the participant stated. In contrast, “The military mostly did the US military thing: ‘tell us what you want to do and we’ll give you options.’”

The players directly stated how helpful these experiences were. Our civilian players were academics, many of whom study conflict. One noted that it was “helpful to get a sense of how US military officers think about these issues.” Military players interviewed also indicated that they had meaningful exchanges with civilian academic players, stating that the wargame was helpful “to bridge the gap between academic learning and the real world.”

Third Learning Objective: Use Wargaming to Create an Active-Learning Experience

The survey measures and interviews demonstrated that participants were extremely positive in their views about the utility of wargaming and also indicated that they were active and engaged throughout the process. Three pre- and post-game survey statements showed significant movements in attitudes toward wargames, as follows:

• Q16: Wargames are a useful tool to study important foreign-policy decisions.

• Q17: Wargames can accurately capture the information uncertainty and decision-making dynamics of real-life crisis scenarios.

• Q18: In-person wargames have an energy and dynamic that cannot be matched by a remote alternative.

The difference between pre- and post-game survey means is illustrated in figure 1, with 95% confidence intervals. The plot demonstrates that playing the wargame increased players’ belief that wargames were a useful tool and decreased their beliefs that virtual games could not capture the same dynamics as in-person games. The graph also demonstrates that controlling for military affiliation does not substantively change results.

Figure 1 Wargaming Questions, Average Treatment Effect of the Wargame

In the post-game survey, we also asked respondents a series of questions about their attitudes during the game; their responses are listed in table 1. Participants indicated that they were neither bored during the game nor able to predict its course. Moreover, they also indicated that they lacked perfect information and were not always able to communicate as they desired. These are key elements of real-life crises.

Table 1 Attitudes During the Wargame

In the free-response questions, respondents highlighted what they considered the utility of wargames. One military officer wrote, “Wargames help to understand possibilities. They allow for actors to test theories and better understand situations from different perspectives.” A civilian participant explained that wargames “help provide insight into decision making and reveal issues that may not be immediately obvious.”

One military officer wrote, “Wargames help to understand possibilities. They allow for actors to test theories and better understand situations from different perspectives.”

In interviews, participants also stressed the value of wargames as a learning tool compared to more-traditional static methods. One civilian participant stated:

I am not an expert on education, but there is research that when people engage emotionally with your subject and their teachers, they learn more. This is true for children, true for university students. Seminars can get at this, but wargames are uniquely helpful tools in this regard because they are so immersive. Virtual learning is poorly situated to get this emotional reaction because of the distance, but the wargame structure really helped with that.

This logic was echoed by a military officer, who stated: “For myself as a kinetic learner, it is very helpful to use a wargaming exercise. Especially when the objectives are clear, it helps to learn.” The same officer continued, “Wargame simulations are a great way to put the pieces together. In an academic environment, you are just learning the pieces, you don’t see what happens when you put the pieces together.”

CONCLUSION: TWO ROLES FOR WARGAMING IN THE CLASSROOM

A wargame was particularly well suited to accomplish the three learning objectives of the MIT–NPS teachers. The strengths of wargaming make it a useful tool for instructors in two large arenas: fostering active learning and bridging gaps between disciplines.

First, wargaming excels as an active-learning method. Participants were engaged with the subject material and motivated to participate for long intervals despite the challenges of virtual learning. The future of higher education in the face of continuing COVID-19 challenges is unclear, but virtual learning undoubtedly will persist in some form.

The players in our virtual wargame had positive learning outcomes and an experience that was stimulating in a different way than a traditional lecture or discussion-based course. As a further test of whether wargaming succeeds as an active-learning method in the context of an in-person classroom, we administered the same pre- and post-game surveys to an in-person class of MIT undergraduates before and after they played a different (but also Taiwan-focused) crisis simulation as part of an international relations course. We found similar results in our surveys: respondents in the post-game survey statistically were significantly more likely to agree that they have a good understanding of potential Taiwan crises and that wargames are a useful tool. There was one notable exception: students in the in-person wargame did not change their views on virtual wargaming, which makes sense because they did not have a virtual wargame experience.

Second, wargaming provides an effective means of bridging gaps between different fields. In our game, we brought together military practitioners and civilian international relations experts. However, wargaming has the potential to be applied outside of the martial realm. Wargames could unite disparate audiences such as city hall disaster planners, emergency workers, federal coordinators, and academic experts by playing either analytical or learning games. This is not only a theoretical possibility: in 2020, academics at the Transition Integrity Project used an analytic wargame to bring together “some of the most accomplished Republicans, Democrats, civil servants, media experts, pollsters, and strategists around” to investigate possible crises that might arise during the American presidential election (Brooks 2020). Learning games also could be designed for similar audiences.

Remote wargaming holds special promise because it can bring together disparate audiences for minimal cost. Prohibitive travel costs are a core impediment to the power of experimental wargames. In a teaching game, “bridging the gap” means exposing different audiences to ideas from another professional space to create the opportunity for each group to learn from one another. As one of our players noted, there was “no need to fly someone out, so you can bring in more viewpoints.” In 2021, despite MIT and NPS returning to in-person instruction, we again ran our Taiwan-focused wargame remotely because the online format was resource feasible.

Future research on wargaming as a method of instruction has several promising directions. First, educational games should examine different contexts in which wargames can be played—whether online, in-person, or hybrid. Second, games should tackle different fields that can be brought together—other than security studies—such as education, election security, and trade politics. Third, educational games should be integrated more directly with classroom instruction to provide more direct tests of knowledge assessment. For example, a wargame exercise paired with an actual exam and compared with traditional instruction would provide direct evidence of its benefits (or drawbacks) as a method of active learning.

As in all new methods, wargaming must be further studied. More testing and experimentation across iterations are necessary before scholars can draw definitive conclusions. However, our results show that wargaming holds promise as a bridge-building tool of instruction that can engage students, scholars, and practitioners in achieving positive learning outcomes.