The Impact of Oral Argument Attendance

Abstract

How does oral argument attendance impact public perceptions of the judiciary? Judicial independence is partly contingent on public support, but the conditions that generate institutional good will are not well understood. We examine how judicial outreach and court exposure inform public attitudes. Leveraging a field-experiment randomizing in-person attendance at oral arguments conducted by a federal circuit court of appeals on a university campus, we find that exposure increases perceptions of institutional legitimacy and the extent to which judicial decisions are motivated by law versus politics. The results have important implications for judicial politics and policy debates concerning reform initiatives involving circuit riding, courtroom cameras, and public outreach.

Introduction

How does oral argument attendance impact public perceptions of the judiciary? Judicial independence is partly contingent on public support (Caldeira 1986; Stephenson 2004; Vanberg 2004), but the conditions that generate institutional good will are not well understood. The public can provoke political attacks on courts that threaten judicial independence (Staton 2004; Clark 2009; Helmke 2010). To forge support, courts can, for example, decide cases consistent with public opinion (McGuire and Stimson 2004; Epstein and Martin 2010; Casillas, Enns, and Wohlfarth 2011) or engage in various outreach efforts (Savchak and Edwards 2016; Curry and Fix 2019; Glennon and Strother 2019). To the extent public understanding of the judicial process is limited, outreach may be appealing to judges, though the extent to which it impacts public attitudes is unclear.

We evaluate the causal effect of oral argument attendance on public perceptions of the judiciary using a field experiment. Specifically, we randomized in-person attendance at oral argument proceedings conducted by a traveling federal circuit court of appeals on a university campus. This type of outreach initiative is common among state and lower federal courts. Consistent with positivity theory (Gibson and Caldeira 2009; Gibson, Lodge, and Woodson 2014; Gibson and Nelson 2017), we find that oral argument attendance increases perceptions of institutional legitimacy. We also find that exposure increases the extent to which people think judicial decisions are motivated by law as opposed to politics. The results are consistent across two oral argument sessions, with nonoverlapping control and treatment groups in each session.

This project has important implications for our understanding of judicial politics and the effect of exposure to government proceedings. With respect to the former, the results enhance our understanding of why oral argument matters (Johnson 2004; Black, Johnson, and Wedeking 2012; Ringsmuth and Johnson 2013) and the impact of court exposure on public attitudes (Black et al. n.d.; Benesh 2006; Krewson 2019). The results also contribute to policy debates concerning reform initiatives involving circuit riding, courtroom cameras, and outreach initiatives. With respect to the latter, the results contribute to our broader understanding of how exposure to government proceedings (Blair, Karim, and Morse 2019; Malesky and Taussig 2019; Peyton, Sierra-Arevalo, and Rand 2019) as well as deliberation and transparency initiatives (Simon and Sulkin 2002; Gottlieb 2016; Hollyer, Rosendorff, and Vreeland 2019) impact the public.

Oral argument exposure

Oral argument can serve a variety of purposes. Internally, judges may use oral argument to acquire information for purposes of informing decisions on the merits, coalition formation, opinion writing, and separation of powers considerations (Johnson 2004; Black, Johnson, and Wedeking 2012; Ringsmuth and Johnson 2013). Externally, oral argument can enhance institutional legitimacy, increase public awareness, guard against noncompliance, and provide judges with a platform for political position taking (Black et al. n.d.; Krehbiel 2016; Jacobi and Sag 2019).

The external impact of oral argument remains relatively understudied notwithstanding its potential importance. As discussions of judicial legitimacy proliferate, oral argument exposure has been at the forefront. With respect to the Supreme Court, for example, it has been said that “the public spectacle of oral argument assures the parties in the case at hand that their arguments have been heard and considered,” while also “allow[ing] the public to see the Court as an impartial tribunal exploring issues of national importance through a balanced adjudicative process” (Jacobi and Sag 2019, 1168).

Institutional maintenance is complicated at the appellate level by a public information deficit. It may be that “to know courts is to love them, because to know them is to be exposed to a series of legitimizing messages focused on the symbols of justice, judicial objectivity, and impartiality” (Gibson, Caldeira, and Baird 1998, 345). As one federal circuit court judge put it, however, “People simply do not understand what appellate courts are supposed to do” (Medina 1961, 155). As the only part of the appellate decision-making process open to the public, oral argument may inform the public’s perception of how these courts operate.

Judges engage in a variety of outreach activities for institutional maintenance purposes (Savchak and Edwards 2016; Curry and Fix 2019; Glennon and Strother 2019), but traveling to conduct oral arguments is a particularly intriguing example because it exposes people to official court proceedings. State and lower federal courts regularly conduct oral argument at law schools.¹ Moreover, several state courts have expanded their oral argument outreach initiatives to the public more broadly. Example institutionalized programs include Appeals on Wheels by the Indiana Court of Appeals, Courts in the Community by the Hawaii Supreme Court, and Justice on Wheels by the Wisconsin Supreme Court. Officials hope these initiatives enhance judicial legitimacy and improve public perception (Deits and Keenan 2004, 243; Nelson 2005, 170), but the extent to which they do so is unclear.

Bringing courts to the people is an old idea. Supreme Court circuit riding may have primarily been a cost-saving measure, but it was also thought to be of “great value in keeping the federal judiciary in touch with the local communities” (Warren 1924, 58). When justices lobbied to abolish the practice, legislators resisted in part because of the public-interfacing component. In 1848, for example, one senator argued that abolishing circuit riding would result in the Court “losing… that responsive confidence of the people, which adds so essentially to the sanction of all the acts of the officers of government.”² Although circuit riding has long been abolished, calls for renewal regularly emphasize the value of public exposure (Calabresi and Presser 2006; Stras 2006).

Positivity theory provides a theoretical lens through which one can assess the potential impact of oral argument exposure on public attitudes. The core idea is simple: “[I]ncreased exposure to the judicial process, whatever the circumstances and even when citizens are displeased, results in collateral exposure to the symbols of judicial legitimacy, thereby tending to reinforce rather than undermine institutional support” (Gibson and Caldeira 2009, 4). Symbol exposure “triggers learned associated thoughts, which for most people in the United States have become connected with these symbols largely through socialization processes and experience, and which are typically ones of legitimacy and positivity” (Gibson, Lodge, and Woodson 2014, 842). As a result, comprehending court proceedings is not a necessary condition for attitude change.

Legitimizing symbols can be specific or abstract. Examples of specific legitimizing symbols that tend to be present at oral argument include “judges’ black robes” (Gibson and Nelson 2017, 593), judges occupying an elevated position in the courtroom (Gibson, Caldeira, and Spence 2003b, 553), and litigants using “honorific forms of address” (Gibson, Lodge, and Woodson 2014, 838). Experimental evidence indicates that exposure to such symbols can heighten the positive relationship between support and decision acquiescence for low-frequency consumers (Gibson, Lodge, and Woodson 2014), sever the link between decision disagreeableness and diffuse support for those expressing relatively low levels of disappointment with a court’s judgment (Gibson and Nelson 2016), and enhance perceptions of institutional legitimacy (Armaly 2018).

Court proceedings also expose consumers to more abstract legitimizing symbols such as “decorum” (Nelson and Gibson 2017, 134); “judicial objectivity” (Gibson, Caldeira, and Baird 1998, 345); “emphasizing reliance on the Constitution, precedent, and legal norms” (Nicholson and Hansford 2014, 621); and “impartiality and insulation from ordinary political pressures” (Gibson and Caldeira 2009, 9). Consistent with courtroom settings, studies in other institutional contexts have found legitimizing effects associated with exposures emphasizing transparency (Gottlieb 2016; Hawkins et al. 2019; Hollyer, Rosendorff, and Vreeland 2019), deliberation (Tyler, Rasinski, and Spodick 1985; Simon and Sulkin 2002; Persson, Esaiasson, and Gilljam 2013), legal frames (Baird and Gangl 2006; Zink, Spriggs, Scott 2009; Farganis 2012), and procedural justice (Gibson 1989; Tyler and Rasinski 1991; Baird 2001).

There is little existing empirical evidence on the impact of court exposure on public attitudes. Most relevant for our purpose, Black et al. (n.d.) conducted survey experiments exposing observers to short video clips of oral argument exchanges in two state supreme courts. Leveraging variation in audio versus video presentations, contentious versus neutral interpersonal exchanges, and dynamic versus static camera angles, the authors find mixed evidence concerning the impact of exposure on perceptions of legitimacy. With respect to court exposure more broadly, Benesh (2006) finds that respondents who report having been a juror express more confidence in local courts, while those who report having been a direct party express less confidence. And Grimmelikhuijsen and Klijn (2015) find that a randomized instruction to watch a television show featuring real trial court proceedings, commentary, and interviews with participating judges yielded increased trust in judges as the self-reported number of shows watched increased.

Building on this literature, we hypothesize that oral argument attendance will increase perceptions of institutional legitimacy and the extent to which judges are thought to base decisions on law versus politics. While there have been relatively few examinations of how court exposure impacts perceptions of decision-making compared to legitimacy, the underlying theory is similar. Legitimizing symbols presumably activate the “myth of legality,” which “holds that cases are decided by the application of legal rules formulated and applied through a politically and philosophically neutral process of legal reasoning” (Scheb and Lyons 2000, 929). While so-called “legalist” and “realist” perspectives on judging are theoretically distinct from the concept of legitimacy (Gibson and Caldeira 2011; Cann and Yates 2016), the same psychological processes that lead symbols to enhance legitimacy can reasonably be construed to reinforce perceptions that judges make decisions based on law rather than politics. Evidence indicates, for example, that exposure to a lunch or speech with a Supreme Court justice increases the extent to which decisions are thought to be driven by law relative to ideology (Krewson 2019).

The field experiment

To examine the causal effect of oral argument exposure on perceptions of the judiciary, we conducted a field experiment randomizing attendance at proceedings conducted by the U.S. Court of Appeals for the Tenth Circuit at Utah State University. The Tenth Circuit is one of thirteen federal circuit courts of appeals. It is an intermediate appellate court, sitting between state-based district courts and the Supreme Court in the federal judicial hierarchy. The Tenth Circuit hears appeals arising from district courts in six states.³ Oral arguments are heard at the court’s discretion, generally in three-judge panels. According to Tenth Circuit officials attending this event, the court travels for oral argument as an outreach initiative for students and the public.

At this event, a three-judge panel held two oral argument sessions, with two cases in each session and a short break between sessions. Case details are provided in the appendix. As is common with circuit court proceedings, the cases were not particularly salient. Advance session-specific tickets were required for admission to the event, which filled to capacity. Student access was primarily allocated through this study.⁴ Prior to the visit, we recruited undergraduate and graduate students to participate in a study concerning courts on event day in exchange for a $10.00 Amazon gift card.⁵ In the recruitment email, students were told they would be randomly assigned to attend an oral argument session or participate in a different court-related activity. Participating students selected one of two time slots corresponding to the oral argument sessions. Once students were assigned to a session slot, they were were randomized within sessions at a 50–50 split between treatment and control groups.⁶

The treatment and control groups were instructed to appear on event day at separate but nearby campus locations. On event day, control group participants completed a survey immediately upon arrival at their scheduled time and were subsequently dismissed. Treatment group participants were admitted to their respective oral argument sessions, after which they were provided the same survey with a day’s end completion deadline. All surveys were completed electronically on event day.⁷ Overall, 248 students participated in the experiment. The first session respectively included 48 and 58 students in the control and treatment groups; the second session respectively included 70 and 72 students in the control and treatment groups.⁸ As described in more detail in the appendix, control and treatment groups were balanced within and across sessions by gender, race, ideology, and prior court exposure. Although student samples present obvious generalizability issues, ours is “relevant for theory” (Coppock and McClellan 2019, 3) because students are a target audience for traveling courts.

We followed standard practice in measuring our outcomes of interest. To capture perceptions of legitimacy, we used an established battery of statements (see, for example, Gibson, Caldeira, and Spence 2003a; Gibson and Caldeira 2009; Gibson and Nelson 2015) that capture whether institutions enjoy “a widely accepted mandate to render judgments for a political community” (Gibson, Caldeira, and Spence 2003a). The items include:

• • Judges on the federal courts who consistently make decisions at odds with what a majority of the people want should be removed from their position as a judge.

• • If the federal courts started making a lot of decisions that most people disagree with, it might be better to do away with the federal courts altogether.

• • The right of the federal courts to decide certain types of controversial issues should be reduced.

• • The federal courts get too mixed up in politics.

Given the institutional context of our experiment, we modified these questions as developed for a study concerning the Supreme Court by replacing “Supreme Court” with “federal courts” (cf. Scherer, Benesh, and Steigerwalt 2010).⁹ Response choices were five-point Likert scales ranging from strongly disagree to strongly agree. A principal components analysis on the scores shows that items load on a single dimension.¹⁰ We used factor scores from the principal components analysis to create a legitimacy index and normalized scores to the [0,1] interval.¹¹

To operationalize our decision-making hypothesis, we used a battery of questions developed by Gibson and Caldeira (2011) to measure what they call “legal realism.” In essence, this battery captures the extent to which people think judicial decision-making is driven by law versus politics. The items include:

• • Judges always say that their decisions are based on the law and the Constitution, but in many cases, judges are really basing their decisions on their own personal beliefs.

• • Judges’ values and political views have little to do with how they decide cases before the federal courts.

• • Judges’ party affiliations have little to do with how they decide cases before the federal courts.

Again, given the institutional context of our experiment, we modified the questions as designed by replacing “Supreme Court” with “federal courts” (cf. Scherer, Benesh, and Steigerwalt 2010). Response choices were five-point Likert scales ranging from strongly disagree to strongly agree. We used principal components analysis to create an index concerning decision-making attitudes and normalized scores to the [0,1] interval.¹² As detailed in the appendix, the result is similar using an alternative measure of beliefs about judicial decision-making developed by Cann and Yates (2016).

Analysis and results

Legitimacy

Beginning with the legitimacy hypothesis, Figure 1 plots means, 95% confidence intervals, and underlying data distributions (jittered for clarity) for the control and treatment groups. Between control and treatment, the perceived legitimacy mean increases from 0.48 to 0.60 ( $ p $ $ < $ .001). Substantively, this change indicates that oral argument attendance increased perceived legitimacy by 57% of a standard deviation. As detailed in the appendix, these results hold across sessions. There is evidence of heterogeneous session effects, with perceived legitimacy higher in the first session, but exposure to the second session nonetheless increased perceived legitimacy by 39% of a standard deviation.

Figure 1. The Effect of Oral Argument Attendance on Legitimacy.

Judicial decision-making

Turning to the decision-making hypothesis, Figure 2 plots means, 95% confidence intervals, and underlying data distributions for the control and treatment groups. Between control and treatment, the mean index score increases from 0.34 to 0.46 ( $ p $ $ < $ .001). Substantively, this indicates that oral argument attendance increased the extent to which individuals perceive judicial decision-making to be driven by law versus politics by 61% of a standard deviation. As detailed in the appendix, the result holds across sessions. Here, however, there is no evidence of heterogeneous session effects.

Figure 2. The Effect of Oral Argument Attendance on Beliefs About Judicial Decision-Making.

Conclusion

Judicial independence is widely thought to be contingent on public support. Given increasing political attacks on the judiciary, courts are engaging in outreach to enhance and maintain institutional legitimacy. What remains unclear, however, is the extent to which courts can unilaterally manage public relations – particularly given that people are often unfamiliar with how they operate. We consider whether court exposure can change public attitudes with a field experiment randomizing in-person attendance at oral argument proceedings conducted by a traveling federal circuit court of appeals. We find that oral argument attendance increases perceived legitimacy and the extent to which judges are thought to make decisions based on law versus politics.

The normative implications are mixed. From a public policy perspective, commentators have long debated reform to increase the public’s exposure to court proceedings. The results presented here suggest that proposals to institute circuit riding and install cameras in courtrooms may increase perceived legitimacy and change the way people think about how judges make decisions, which may be arguments in their favor. But these policy questions are complex, and a comprehensive assessment will require more empirical evidence concerning a variety of potential costs and benefits. With respect to cameras in courtrooms, for example, impact may depend on a variety of circumstances (Black et al. n.d.), which cautions against interpreting the evidence presented here as indicating that oral argument exposure is necessarily positive.

A potentially concerning normative implication is that oral argument exposure may generate misleading public perceptions of the judiciary. Although we find that exposure increases the extent to which judges are thought to base decisions on law versus politics, for example, it is not clear that such updating is warranted based on what transpires at oral argument. Interestingly, some Supreme Court justices have argued against courtroom cameras out of concern that excerpts may mislead viewers (Segall 2016, 793–794), but any misdirection may work the other way as well. To the extent oral arguments disproportionately emphasize neutral principles relative to our broader empirical understanding of the determinants of judicial decision-making, exposure may distort public perception.

The results have important implications for judicial politics. While much of the empirical research on oral arguments emphasizes internal dynamics, we show that these proceedings can impact the public as well. We also contribute to the literature on positivity theory with application to a new setting using a real-world exposure, both of which enhance generalizability. Moreover, notwithstanding the literature’s emphasis on legitimacy, it is important to understand how exposure impacts other outcomes of interest (cf. Black et al. n.d., 17). We advance this goal by examining how oral argument attendance impacts public perceptions of judicial decision-making. Last, our research design is transportable to other areas where exposure to court business may impact public attitudes, such as briefs, conference deliberations, and internal correspondence bargaining over opinion language.

Notwithstanding this project’s contributions, there are several important limitations. Generalizability is limited by reliance on college students who self-selected into exposure. Although students are “relevant for theory” (Coppock and McClellan 2019, 3) here insofar as they are a target demographic for judicial outreach efforts, it may be, for example, that older participants would be less impressionable. Theoretically, there is reason to expect heterogeneous treatment effects based on factors such as case salience and topic familiarity, but the proceedings analyzed here did not involve constitutional or otherwise politically contested issues. Moreover, we do not explore treatment effect persistance or isolate the mechanisms underlying observed attitude changes. Future research will help develop a more comprehensive evaluation of how exposure to court proceedings impacts the public.