The hidden impacts of the ICC: An innovative assessment using Google data

Abstract

Commentators now regularly declare that the International Criminal court (ICC) – and international criminal law as a whole – is in crisis. It is certainly the case that the ICC faces a number of operational challenges, and that these challenges worry its defenders. However, one unexamined rationalist assumption is that the Court’s inability to deliver consistent outputs will mean waning influence. This article explores an alternative constructivist theory that the ICC produces diffuse social impacts that are not necessarily tied to its operating effectively. This theory is tested statistically using Google Trends data. Specifically, the article examines whether ICC intervention in a country is associated with more internet search for ‘human rights’. Taking this to be a measure of changing discourse in countries, the article finds that some types of ICC involvement are associated with a far higher interest in human rights, and that this interest only increases as ICC involvement extends in time. In short, despite its disappointments, evidence suggests that the ICC still serves a socio-pedagogical function. Though it does not fit well within a rational evaluation framework, this kind of information should be considered in ICC performance reviews.

1. Introduction ¹

It is now ordinary to read that the ICC is ‘failing’ ² or ‘in crisis’. ³ Common critiques focus on the judiciary’s lengthy and expensive trial proceedings; the Office of the Prosecutor’s (OTP) inability to conduct quality investigations or consistently win convictions; or the lack of collegiality and climate of distrust among Court personnel. ⁴ On 30 September 2020, an Independent Expert Review (IER) panel released a much-anticipated report on ICC operations. ⁵ Pursuant to an Assembly of State Parties (ASP) establishing resolution, ⁶ the IER performed its work with the aim of ‘enhancing the performance, efficiency and effectiveness of the Court …’ ⁷ Its report is one manifestation of the ASP’s long-standing emphasis on the ‘performance question’: what are the best ways to measure and improve the Court’s progress towards its goals? ⁸

This is not an easy question to answer. In evaluation science, assessing the performance of an organization requires a systematic consideration of how efficiently or effectively it operates to produce outputs that translate into intended outcomes. ⁹ Conducting such an assessment is challenging enough when the organization is a small business or an academic department, ¹⁰ but it is far more onerous when examining an institution as complex as an international court. Some have wondered whether it is even possible to reliably assess the ICC’s accomplishments: the Rome Statute does not clearly outline the Court’s aims or objectives; ¹¹ and its extrapolated goals are multi-faceted and possibly contradictory. ¹² A ‘frank and forthright’ 348-page document, ¹³ the IER Final Report is a thorough evaluation of the ICC’s operational performance, but it largely punts on the question of the ICC’s real-world impacts. ¹⁴ The report states that civil society organizations and academics are best situated to perform impact assessments. ¹⁵

With this in mind, this article argues that, despite its many problems, the ICC has diffuse yet measurable impacts on social discourse, and these impacts are overlooked in most assessments. ¹⁶ The data presented demonstrate that ICC interventions – including preliminary examinations and investigations – are associated with a significant and sustained increase in Google searches for ‘human rights’ in situation countries. This discovery derives from a larger comparative political science project that uses internet searches to gauge latent interests within countries’ populations. The statistical relationship uncovered in this article, between ICC involvement and measurable jumps in a population’s human rights interest, is quite powerful. In fact, there is a no more significant predictor of sudden spikes in a country’s Google searches for human rights than the onset of an OTP investigation. This mostly likely indicates that ICC involvement leads to information-seeking among the public, hinting that the Court may indeed have ‘socio-pedagogical’ effects that often remain hidden. ¹⁷

Before presenting the design and results of the analysis, the article first considers different perspectives on the ICC’s performance. As explained in the next section, studies centred on the Court’s diffuse social effects do not fit comfortably into a ‘rational-system approach’, which assesses organizations based simply on whether they accomplish their goals. ¹⁸ In practice, this means analysing indicators of progress following a ‘logic model’ that connects inputs to outputs to outcomes in a linear chain. However, the bedrock assumptions of this rationalist approach may result in incomplete assessments of the ICC, for two main reasons. First, it is unclear whether the Court’s observed impacts emanate directly from specific outputs like fair and expeditious trial proceedings, an adequate level of victim participation, or a high conviction rate. In other words, the ICC’s broader effects on domestic societies are not necessarily attributable to Court conduct in relation to key benchmarks. Second, it is entirely plausible that the ICC’s effects follow a constructivist rather than a rationalist logic. It might be that the Court’s contribution comes not from accomplishing clearly defined legal goals with intentional actions, but by engaging in social relations that create stigma and embolden local rights advocates. ¹⁹ None of these mechanisms depend on the ICC making efficient or effective use of its resources. Instead, they trace back to its role as a focal and symbolically important institution of accountability.

2. Two perspectives on ICC performance

2.1 The rationalist approach

The most common approach to evaluating the ICC’s performance is rationalism. Rationalism is a philosophical position which holds that individual human actions should be, and often are, based on reasoned intent. This ontology thus assumes social, legal, and political behaviour can be perfected through good insight and design. ²⁰ Journalists informally adopt rationalism when they point to the OTP’s low conviction rate, or when they call into question the ICC’s sluggish process, great expense, or lack of ‘bang for the buck’ as evidence of inefficiency. ²¹ Some ask outright whether the ICC has ‘failed’, citing the fact that atrocities still take place in states like Syria, Yemen, or Myanmar. ²² The implication is that, if only it followed better procedures or its personnel were more competent, the Court would achieve greater success in deterring violence in the world.

Rationalism also undergirds more formal ‘effectiveness criticism’ directed at the ICC. ²³ This involves identifying the Court’s mandated aims – like providing victim satisfaction or deterring future atrocities – and positing that certain operational shortcomings prevent those aims from being reached. One common underlying assumption is that the ICC has carefully calculating audiences that are keeping a close watch. When those audiences observe missteps, they receive signals to update their attitudes and behaviour in relation to the Court. For instance, consider the often disparaged slow pace of investigations and proceedings, ²⁴ which results not only from the institutional structure of the Court but also prosecutorial strategy. ²⁵ Concern over the ICC’s expeditiousness, carried over from criticism of the ad hoc tribunals of the 1990s, ²⁶ is rooted in the presumption that slow justice leads to waning hopes among victims. ²⁷ If victims are not satisfied, the thinking goes, the Court will lose legitimacy. Similarly, drawing on criminology, scholars argue that selective prosecutions and infrequent convictions will do little to change the cost–benefit analysis of would-be atrocity criminals. ²⁸ In other words, insufficient trial outputs will fail to achieve deterrence. Why? Because atrocity criminals are plodding, careful actors who follow the Court’s record, and update their prior beliefs with new information. ²⁹ But each of these effectiveness critiques is theoretical: we possess very little hard evidence that victims are disappointed with the ICC, or that violent state leaders discount the Court for being toothless.

Rationalist performance evaluations are not just employed for criticism. The ICC itself has adopted a highly rationalist, some say technocratic, approach to monitoring its own operations. ³⁰ The Second Court’s Report on Development of Performance Indicators outlines a set of four goals – fair and expeditious trials, effective leadership and management, adequate security, and victim access – and presents a bevy of indicators for auditing the Court’s pursuit of those goals. ³¹ Yuval Shany’s Assessing the Effectiveness of International Courts is similar, but far more comprehensive, presenting the most meticulous ‘rational-systems’ assessment of the ICC to date. ³² Shany and his co-authors break the Court’s work into four overarching goals, and dissect the logic linking its inputs to its outputs and outcomes. They conclude that the ICC has achieved some of its aims, though they write that the ‘literature about the ICC includes very few empirical studies about the Court’s actual outcomes’. ³³ They follow: ‘mixed indicators regarding prospective goal attainment are not resolved by analysis of the Court’s performance’. ³⁴ This means that the causal link between ICC outputs, like conviction rates or victim participation rates, and the achievement of clear-cut goals is not as well-established as one might suppose. In short, while the ‘rational’ approach to performance evaluation is logical and theoretically helpful, it has yet to prove its empirical value. We still do not know whether the Court’s impacts are in fact tied to its day-to-day operations.

2.2 The constructivist approach

An alternative approach to assessing ICC performance is available. ³⁵ Drawing on international relations theory, one may call this alternative the constructivist approach. ³⁶ Constructivism is concerned with the ways that ideas and norms permeate interactions, thus conditioning ‘the nature of agents and subjects’. ³⁷ Philosophically, this diverges from rationalism in many ways, most notably by assuming that humans are social beings whose preferences are shaped by a dense network of rules that constitute society. ³⁸ In short, people are not individual cost-benefit calculators; they are complex rule navigators.

Constructivist performance evaluation has been proposed by a handful of management scholars, but not widely adopted in the study of law and society. ³⁹ This alternative essentially involves considering how an organization may itself alter social relations. There are two main reasons that reviewers might consider this departure from a rational systems approach. First, the ICC is not easily reduced to a corporate forum for creating predictable legal ‘outputs’ that produce desired ‘outcomes’. The trajectory of Court practice does not follow a clearly designed logic model because some of the Court’s outputs and outcomes are unpredictable. For example, the purpose of the OTP’s preliminary examinations was given relatively scant attention in the Rome Statute, but over time these examinations have evolved, playing an increasingly important role in monitoring states’ compliance with international criminal law. ⁴⁰ Furthermore, research demonstrates that advancing to the investigation stage has unintended positive impacts on situation countries, encouraging legal mobilization and increasing low-level prosecutions of state agents. ⁴¹ Sarah Nouwen refers to the Court’s jumpstarting domestic judicial processes as its ‘catalytic’ effects. ⁴² However, while Court actors emphasized positive complementarity early on, it is harder to say that preliminary examinations or investigations were originally designed with these catalytic effects in mind, or that they resulted from the OTP achieving clearly articulated benchmarks.

A second reason to consider a constructivist approach is that an overly instrumentalized view of the Court’s work ignores that international legal institutions and global society are co-constitutive. The society of states created the Rome Statute in a contingent historical moment. ⁴³ In so doing, it instantiated and codified anti-atrocity norms now seen as foundational to an evolving ‘moral community’. ⁴⁴ The need to merely belong, or be recognized as belonging, to that community, might alter behaviour. Understanding this means moving beyond a narrow, managerial understanding of organizational outputs and outcomes. In Carsten Stahn’s words, assessment ‘requires a holistic account which views institutional performance in the context of systemic considerations and perceptions by a variety of stakeholders’. ⁴⁵ For example, empirical research demonstrates that Rome Statute ratifications alone are associated with declines in indiscriminate violence committed by government and rebel forces, even while civil war is ongoing. ⁴⁶ States party to the Rome Statute are home to almost no massacres, and very few new civil wars. ⁴⁷ That political violence in increasingly concentrated in non-party states suggests the Rome regime is influential, though it is hard to attribute this pattern to individual choices made by rational leaders in direct response to Court actions. It could be that the role of the ICC regime is not just causal, but constitutive. ⁴⁸ It not only produces independent outputs that translate into outcomes; it embodies and defines rules of global society. ⁴⁹ This is why Jo and Simmons call the relationship between ICC jurisdiction and declines in violence ‘social deterrence’. ⁵⁰

Contrary to rationalist accounts, evidence-based impact studies conceive of the Court’s catalysis or social deterrence functions as reliant on the ICC merely establishing jurisdiction or becoming enmeshed in a situation, rather than operating effectively or efficiently. ⁵¹ However, one major deficiency of these studies – and constructivist accounts overall – it that they lack ‘linking evidence’. In other words, the existence of the ICC, and its intervention into certain situations, may be associated with fewer overt acts of violence and more domestic steps toward accountability over time. But how? The answer, theorized in the next section, is that the Court inspires subtle and indirect changes in social discourse.

3. The ICC and interest in human rights

Early supporters hoped that the ICC would lead to a worldwide cultural shift. In 1999, William Pace and Mark Thieroff predicted that ‘the significance of the adoption of the [Rome] Statute may well lie, not in the actual institution itself … but in the revolution in legal and moral attitudes towards the worst crimes in the world’. ⁵² For the ICC to perform such a role, it would have to transform discourse, or alter the way that people communicate and act. A necessary condition for such a transformation is the conveyance of new information and ideas, which might lead to learning. This is what some have labeled the Court’s potential ‘socio-pedagogical’ function. In 2005, Payam Akhavan wrote:

The interests of global justice include deterrence or general prevention not only through pragmatic considerations such as shifting the boundaries of legitimacy and thereby changing the cost-benefit calculus of using atrocities as an instrument of power, but also through the more subtle, but far-reaching, socio-pedagogical influence of judicial stigmatization to induce subliminal inhibitions against criminal conduct. ⁵³

Akhavan asks us to look beyond a narrow economic account of how the Court might change atrocity criminals’ calculated preferences – and to widen our gaze toward its didactic social effects. Some of these may be ‘subtle’ or ‘subliminal’. How then, could one observe whether the ICC has subtle effects on the way that people think, talk, and act?

One suggestion is to study how local populations respond to ICC interventions. Mirjan Damaška, who defines the socio-pedagogical function of the court as ‘one of strengthening the public sense of accountability for human rights violations’, writes that the ‘the importance of considering local responses to the decisions of international criminal courts can hardly be overemphasized’. ⁵⁴ Survey researchers regularly study the attitudes toward the Court in countries that have no direct experience with the ICC. ⁵⁵ Other surveys do focus on affected populations, but their respondents’ attitudes are frozen in specific moments in time, meaning that we cannot observe whether those attitudes change. ⁵⁶ This is no fault of research design, but results from the limitations of survey research itself: it is simply not feasible to draw samples from all countries continuously over time, so we cannot know how or why certain populations’ attitudes are changing, or why.

One innovative new strategy for observing the ICC’s hidden socio-pedagogical impact on local populations is to examine variations in internet search patterns. ⁵⁷ To see the discursive splash of ICC preliminary examinations, investigations, and proceedings, one might study how these processes are associated with reverberations in aggregate Google searches for ‘human rights’ over time. Doing so can help us evaluate the merit of a constructivist understanding of ICC influence. Before presenting an analysis along these lines, two issues must first be addressed in greater detail: why searches for ‘human rights?’ and second, why Google searches?

3.1 Why ‘human rights’?

A central claim of this study is that one can, in part, observe the ICC’s social effects by analysing the relationship between Court involvement in a country and changing interest in ‘human rights’. Why then would the ICC, an international criminal tribunal, have any effect on thoughts or talk about ‘human rights’? The first reason is that despite being distinct regimes, international criminal law (ICL) and human rights law are intertwined. Article 21(3) of the Rome Statute states that the ‘application and interpretation of law pursuant to this article must be consistent with internationally recognized human rights’. This article is arguably ‘quasi-constitutional’ because all other applicable law must be interpreted in light of the Court’s human rights obligations. ⁵⁸ The scope of these obligations remains ill-defined, but at the very least this provision represents the ‘importation’ of assumptions and goals from human rights law to ICL. ⁵⁹ Furthermore, human rights advocates like those at Amnesty International regularly align themselves with the project of international justice, lending public, vehement support to the ICC. ⁶⁰ Well before the ICC even existed, activists envisioned a criminal court that would ‘enforce human rights’, ⁶¹ and for better or worse, many continue to think of ICL as serving human rights aims. ⁶²

Second, empirical research shows it is not just advocates and practitioners who draw connections between ICL and human rights, but also laypersons. The American Bar Association’s International Criminal Court Project finds that US citizens are 10–15 per cent more supportive of international human rights organizations if they are aware of the International Criminal Court. ⁶³ Another experimental study demonstrates that random Americans respond more favourably to the ICC if its work is framed in terms of pursuing human rights goals. ⁶⁴ These kinds of findings are not limited to the US. Unpublished research by Stephen Chaudoin shows that, beginning with the onset of the OTP’s preliminary examination into the Philippines in February 2018, the proportion of local media coverage devoted to the human rights ramifications of the country’s violent drug war increased. ⁶⁵ For Chaudoin, this ‘script’ change is evidence of the ‘subtle and indirect effects’ of the ICC. ⁶⁶

Third, human rights ‘talk’ has been globally ascendant since the mid-1970s. ⁶⁷ This provides methodological advantages when studying the effect of ICC intervention on internet searches. Because the phrase ‘human rights’ is almost ever-present – used in written language as much or more frequently than the phrase ‘national security’ ⁶⁸ – then it is less likely that local changes in the use of this language represent a passing fad, or curiosity about a popular new term like ‘Covid-19’. Furthermore, because the language of human rights is so widespread, it makes it easier to study regular fluctuations in its use across countries.

3.2 Why Google searches?

The second question is, why Google searches? Google Trends data have been used in research on topics like the focality of the World Trade Organization trade because they arguably do a better job than survey data at providing a measure of shifting public interests. ⁶⁹ People Google search for information that they are not willing to talk about in public, or admit to a survey researcher. ⁷⁰ Extrapolating, we make the following assumption: populations with more aggregate searches for human rights evince greater overall interest in human rights. This could indicate general curiosity about the subject, or it could reflect greater demand for information on an available claim-making technology.

Google data have two major advantages as an indicator of the changing salience of human rights discourse. First, ‘human rights’ is searched on Google in high volume across countries, more so than in other web browsers like Bing. As of the time of this writing, estimates place Google around 85–90 per cent of search engine market share. ⁷¹ The high volume of average global Google searches for ‘human rights’ every year makes for richer and more meaningful variations across cases in comparison to very specific (thus low-volume) search phrases like ‘International Criminal Court’. The diffusion of the Google browser and the human rights search term is beneficial. Through data processing and management, one can examine aggregate searches for ‘human rights’ in the Google search engine made by nationals of 170 countries in five different language groups: English, Spanish, Portuguese, French, and Arabic. These language groups are chosen because they provide for cross-national comparison (i.e., they are spoken in many countries). They are also spoken in every country subject to an ICC intervention, save for Myanmar and Georgia.

Second, the Google Trends tool, used to download search data, not only allows for wide geographical analysis, but also extensive coverage in time. One can observe aggregate search totals at the weekly level over a series of years, yielding very detailed temporal trends. For any given country, one can record hundreds of observations measuring the frequency with which the population uses Google to access certain pieces of information, the city-wide and regional breakdown of aggregate searches, and what additional search terms are most commonly associated with information-seeking on the phrase ‘human rights’. This data give provide a remarkable amount of leverage over questions about interest in rights across space and time.

4. Hypotheses

How would the ICC’s involvement in a country be related to aggregate searches for human rights? According to a constructivist model, if Google activity is an indicator of latent social interest in a topic, then searches for ‘human rights’ in a country’s population should increase when it is subject to ICC involvement. The reason is that ICC legal interventions serve as an external shock. The Court’s promise is to marshal outside resources to fight impunity and provide justice. Much like transnational advocacy campaigns, domestic groups seek opportunities for reform that comes with this foreign attention. ICC interventions are constituency-building. ⁷² They provide a rallying point for local groups promoting human rights and rule of law.

Some ICC activities will attract greater attention than others. Drawing on previous research, there is reason to think that the ICC’s focality in a country situation is not uniform; it varies by level of involvement. ⁷³ While a state’s ratification of the Rome Statute may establish a basis for promoting new domestic legal standards, ⁷⁴ and could even dampen leaders’ willingness to engage in mass violence, ⁷⁵ treaty ratification is probably not enough to catalyse a discursive shift in a country’s population. However, when a preliminary examination is announced, one might expect this to be accompanied by changing expectations, and an attendant information-seeking and talk about human rights, as in the Philippines. ⁷⁶ This was certainly the expectation for early practitioners holding out hope for positive complementarity. ⁷⁷

ICC involvement deepens with the approval of a full investigation. At this stage the OTP gains more legal powers. ⁷⁸ For instance, it can request from a Pre-Trial Chamber the issuance of arrest warrants. This has a couple of implications:

First, specific individuals are singled out as potential perpetrators of international crimes, increasing the likelihood of prosecution and punishment. Second, the naming of individuals serves as a symbolic marker for investors, diplomats, and other international audiences. The opening of an investigation introduces into a country’s political discourse the notion that atrocity crimes have been committed. ⁷⁹

The stigma that comes with this recognition creates ‘social costs’ for leaders, but also may attract more citizens to the cause of accountability. ⁸⁰

Finally, once trial proceedings begin, one might expect the level of attention to reach a critical mass. At this point, the ICC’s engagement with potential witnesses, outreach, and capacity-building efforts are in implementation phase. ⁸¹ Survey research suggests that registered victims are on average pleased with their direct interactions with the ICC, but they often hold ‘insufficient knowledge to make informed decisions about their participation in ICC cases’ because Court actors have not been able to maintain long-lasting relationships or regularly update locals, especially in rural areas. ⁸² This combination of focal intervention, combined with outreach that is insufficient to reach the entire population, is likely to result in greater individual efforts to seek and discover more information about the Court and its goals. These efforts could, in many cases, lead to more internet searching.

Taken together, the expectations that derive from the constructivist approach would suggest that levels of interest graduate with stages of ICC involvement. If this is accurate, we should expect the following:

Hypothesis 1: National interest in human rights will increase with each level of ICC Involvement.

Sceptical rationalists may find reason to doubt the discourse-shaping potential of ICC interventions. This, again, is due to the assumption that outcomes are tied to specific outputs, which are often seen as lacking. First off, capacity and outreach are nearly always insufficient, meaning that criminal prosecutions usually do not attract the kind of attention advocates hope for. ⁸³ Secondly, even when prosecutions do become part of the local narrative – receiving a good deal of media coverage – the effect is just to raise expectations unreasonably. ⁸⁴ The ICC routinely promises swift justice to victims, only to drag out cases for years without producing convictions. In such cases, rationalists expect the Court’s interventions to be counterproductive. As populations grow impatient with the ICC’s slow pace and lack of results, they will simply lose interest. As atrocity criminals observe the Court’s regular failures, they will commence organizing more violence. In short, for rationalists, the gap between the Courts goals and its operational outcomes will inevitably widen.

This expectation is really about the durability of interest in the Court’s project following an intervention. Rationalists might concede to constructivist theorists that an external shock will gin up curiosity in the ICC in the short term. It may even inspire human rights politics. However, rationalists would likely question whether any externally driven change will last. ICC activity is a blip, and whatever discursive practices it may or may not provoke will certainly fade with time – especially after people in target states witness the tribunal’s steady state of crisis, and realize it cannot deliver. If the constructivist approach is a better representation of the ICC’s impact, however, then the data would show not only a short-term uptick in human rights interest at each stage of ICC involvement, but also higher levels of interest that sustain into the long term. The reason is that, if the ICC is serving a socio-pedagogical function, it will generate a discursive shift toward human rights in the country that will be hard to roll back, or undo.

Hypothesis 2: higher levels of National interest in human rights will sustain even as ICC involvement extends in time.

5. Research design

To test these hypotheses, I construct two sets of statistical models. The first analyses the relationship between the different stages of ICC involvement – preliminary examination, investigation, and trial proceedings – on national Google searches for human rights. The second set analyses the duration of each stage of ICC involvement and human rights searches.

5.1 Google hit rates

This analysis utilizes Google’s Trends data on the phrase ‘human rights’ in the period 1 October 2013–1 October 2018 across 170 countries. These countries represent five different language groups: English, Spanish, Portuguese, French, and Arabic. ⁸⁵ For each language group, the most commonly used translation of the phrase ‘human rights’ was employed. For example, in Spanish, the Google query data is for ‘derechos humanos’. Analysing temporal trends in aggregate searches in each major language allows for rates to be compared across countries over time.

Google Trends does not provide the analyst with raw search totals. Instead, it provides ‘search ratios’, which are the total number of human rights searches divided by the total number of all browser searches in a country. Though it would be very useful to have raw search totals, examining search ratios is arguably better because it controls for population size and internet penetration. The denominator in the ratio includes total searches, which is in part a function of how many searchers are in the country. Search ratios, then, reflect not how wired a country’s population is, but how much that population demonstrates interest in a particular topic as a proportion of all topics. It captures the relative popularity of the concept in a population.

The dependent variable for this analysis is not search ratio but Google ‘hit rates’. The weekly hit rate is a sample-adjusted transformation of all countries’ search ratios. More specifically, the hit rate represents the proportion of total searches in a state relative to the weekly search maximum in the global sample of all states. It is a proportion of a proportion. The country with the highest search ratio in a single week over the 2013–2018 period is given a score of 100. Every other country-search-week is assigned a score equalling its proportion to that global maximum human rights search ratio. ⁸⁶ The global maximum occurred in Uganda in 2015. Based on hit rate calculations, one can take this information and determine, for example, that the average Ugandan searches for human rights approximately 13.2 times for every one time an average American does. Uganda is the top human rights-searching country in the world.

The dataset is not at the weekly unit of analysis because other control variables – like GDP Growth – are only available at the country-year level. Therefore, weekly hit rates were converted into a single country-year observation. The dependent variable, Hits Mean, represents the average weekly hit rate for a country in an entire year (it is a yearly average of 260 weekly hit rates values). Higher scores on Hits Mean indicate that nationals of a particular state more regularly search for ‘human rights’ over an entire year compared to populations in other states. The range of values for Hits Mean is 0 to 58.1, with a standard deviation of 7.1. This measurement is meant to capture the latent interest in human rights in different countries.

Yearly averages have the benefit of placing greater emphasis on sustained attention, rather than short-term weekly jumps in interest. This is useful because weekly data on human rights searches is spikey. Figure 1 shows the globally aggregated weekly data on English searches over the five-year period included in the dataset. On can note that the data are seasonal, decreasing in June and December months. Using yearly Hits Mean accounts for seasonal patterns that repeat across all countries, like school or holiday breaks from ‘human rights’ searching. In the end, the dependent variable captures continuous interest. The higher the score, the more a country’s nationals showed curiosity about human rights over an entire year’s time.

Figure 1. Global weekly trends in human rights Google searches.

5.2 Independent variables

To assess the hypotheses presented above, four pooled regression models are estimated. These correlate independent variables with Google Hits Mean. Each model includes a fixed effects parameter to account for searches that occur in different language groups. The indicators used to capture ICC involvement in a country are straightforward. In Models 1 and 1b, four binary variables are included to measure level of involvement: (1) ICC State Party, recorded ‘1’ if the state has ratified the Rome Statute; (2) ICC Preliminary Examination, recorded ‘1’ in years during which a country’s situation is subject to an examination; (3) ICC investigation, recorded ‘1’ in years during which at country’s nationals are under investigation by OTP; and (4) ICC Trial, recorded ‘1’ in years during which at least one of a country’s nationals is on trial.

The second set of models – 2a and 2b – follows the same idea, but each of the four variables is summed by year to account for the duration of each level of ICC involvement. For example, if in 2015, a country’s situation had been under investigation for 11 years – as was the case with Democratic Republic of Congo – then the country would receive a score of ‘11’ on Years of Investigation. This rule is also followed for Years of Preliminary examination and Years of Trial Proceedings. These duration variables allow us to test whether interest in human rights increases or decreases as a country carries on in the ICC’s crosshairs.

Of course, the ICC is not the only factor that would potentially affect a nation’s latent interest in human rights. The models must also include a series of important confounding variables. These are based on previous research theorizing why human rights discourse takes hold in certain contexts. One theory is that human rights ‘talk’ is imposed by forces external to a state. To control for this, variables for how many international human rights non-governmental organizations (HRNGOs) operate in the country ⁸⁷ and FDI inflows as a percentage of total GDP are incorporated, ⁸⁸ each of which is thought be a vector of rights-based influence from the outside.

A second theory is that human rights discourse emerges endogenously within countries. This can come from the economic growth, which produces a desire for self-expression and political representation. ⁸⁹ Or it can emerge in direct response to state violence. Seeking empowerment against government coercion, citizens choose human rights claims out of a menu of repertories for resistance. In this sense, rights are self-liquidating: the more rights are protected, the fewer claims are made. ⁹⁰ To account for the economic explanation, a measure of GDP Growth is included. ⁹¹ And to account for the resistance explanation, the most recent update of Chris Fariss’s Human Rights Violations data is added to the model; the higher the score, the more regular is government violence directed at citizens. ⁹²

The models also include a final control for internet tampering. A valid concern with this research design is that Google searching for human rights will be less likely in countries like China, where the central government makes a concerted effort to limit internet access, or to block certain information from users. While the dependent variable, Hits Mean, includes information on search volumes within national contexts, which would account, in part, for the impact of censorship, it could still be that government censors specifically inhibit searching for human rights content. A variable called Internet Censorship collected by the Varieties of Democracy Project captures how much, in practice, states governments filter the internet. ⁹³

6. Results

Table 1 presents the results from the four statistical models. Models 1 and 2 are fully specified using all 170 countries over the five years (850 country-years). Many of these countries are home to human rights searching in multiple languages at once; so, they account for 623 additional observations. Models 1b and 2b are different specifications that exclude OECD countries from the analyses. This addresses concern that including developed countries would bias the results in one direction or the other. Countries in the Global North are also commonly credited (or accused) of being major exporters of human rights ideals, ⁹⁴ so it is possible that including them in a study of discursive diffusion would ignore their unique influence on the process being modeled. When examining the results, four findings come to the foreground.

• Finding one: ratification of the Rome Statute alone is not associated with a noticeable change in human rights interest, as measured by Google searches.

Table 1. Regression Models of Google Searches for ‘Human Rights’

^p. 10; * p. 05; ** p. 01; *** p. 001.

The variable ICC State party is not statistically significant, ⁹⁵ save for when OECD countries are excluded. In Model 1b, the coefficient of 1.19 is significant at the .05 level. What this suggests is that state party status is associated with more human rights searching in middle- and low-income states. This lends only weak support to theories that treaty ratifications alone encourage information-seeking, or help build constituencies, in certain contexts. ⁹⁶

• Finding two: all preliminary examinations are not associated with more human rights searching, but long preliminary examinations are.

In Models 1 and 1b, the coefficients for ICC Preliminary Exam are positive but not statistically significant. However, Years of Preliminary exam is positive and statistically significant at the .01 level in Models 2 and 2b. For each year that preliminary examinations extend in time, people in targeted countries increases the average hit rate by around 0.4. This amounts to a little over a 1 per cent increase in aggregate searches every two years. Undoubtedly the results here are driven primarily by Colombia. In 2018, its preliminary examination hit the 14-year mark. Over the sampled period, the overall salience of human rights in the country, as measured by search volume, also increased. One could reasonably dispute whether this relationship is causal. However, it should be noted that this confirms theories that the ICC preliminary examination produced pressure on Colombian authorities to resolve civil war violence, and to address local demands for accountability. ⁹⁷ In these Google data, we may have clear evidence of the relationship between long-run ICC involvement and increased demand for human rights in the national public.

• Finding three: ICC investigations have a very large effect on human rights interest.

The statistical relationship between ICC investigation and Hits mean is, across the board, hugely robust. The coefficients in Models 1 and 1b hover around 5, and each is statistically significant at the .001 level. In terms of magnitude: in any country-year in which an ICC investigation is ongoing, average human rights hits rates are around 8 per cent higher than in contexts without an ICC investigation. This is a massive effect. In fact, unexpectedly, it is the most powerful predictor of human rights searching in all of the models. If you want to find people who show a greater interest in the concept of human rights – as a proportion of other interests – it would be a safe bet to travel to a country under investigation by the ICC.

One can clearly observe the magnitude of this statistical relationship in Figure 2, a visual depiction of the models’ coefficients. In Model 1, shown in the left panel, the effect of ICC Investigation dwarfs all other determinants. This is remarkable because other research suggests that human rights ‘talk’ is mostly a function of measurable rights violations and GDP Growth – endogenous factors. However, ICC investigations seem to be one exogenous factor associated with quite drastic discursive changes.

Figure 2. Coefficient plot of Models 1 and 2.

Furthermore, the effect of investigations is not a short-term affair. It gets even stronger as the investigation wears on. For each additional year of an investigation, shown in the right panel, there is a 1.7 per cent increase in human rights browser searches. People in investigated countries do not lose interest in human rights as the investigations drags on. Surprisingly, the same cannot be said of actual trials proceedings at the ICC.

• Finding four: Interest in human rights does not increase when the ICC moves from investigation to trial, and interest appears to decrease slightly as trials extend in duration.

The variable ICC Trials is negatively associated with Google searches, though the correlations are not statistically significant in Models 1 and 1b and are significant only at the .10 level in Models 2 and 2b. Again, looking at Figure 2, one can see that the Trial and Years of Trials coefficients are on the left side of the zero line, meaning that ICC trials are associated, on average, with fewer searches for human rights. This relationship is not statistically or substantively robust, but taken together with the other findings, we can say the following with certainty: interest in human rights increases with each level of ICC involvement in a country, until an actual trial begins. At that point, people’s curiosity or information-seeking about human rights levels off or declines slightly.

6.1 Discussion

These results demonstrate relationships that were previously hidden. ICC involvement in a country is very clearly associated with an impossible-to-ignore increase in human rights interest. This might very well be taken as evidence of the Court’s socio-pedagogical function: it inspires local communities to seek information and learn about human rights norms. Insofar as contributing to a ‘public sense’ of these norms is a goal of the ICC, it appears on some level to achieve this goal. ⁹⁸ Furthermore, in support of Hypothesis 1, national interest in human rights increases in stepwise fashion as ICC involvement deepens. State ratification of the Rome Statute is not accompanied by a significant surge in Google searches for ‘human rights’, but this changes as the OTP moves forward. Preliminary examinations correlate with a gradual increase in human rights searches as the examinations extend in time. And when the OTP moves forward to the investigation stage, there is a punctuated upswing in human rights-related internet searching that maintains its higher level even as the investigation becomes lengthier. This supports Hypothesis 2, that the discursive shift associated with ICC involvement is sustained over time.

There is one very important exception to these findings. Google searches hit a ceiling at the trial proceedings stage. By the time a trial begins, there is no jump in internet activity similar to that attached to an investigation and as trial proceedings become more protracted, human rights-based internet activity tapers off.

In one way, these findings gel with the newest research that emphasizes the ICC’s diffuse social effects: because they introduce great uncertainty but also possibility, OTP investigations are associated with a flurry of activity and the mobilization of various political coalitions. ⁹⁹ Amid this mobilization, elite members of pro-justice coalitions – including sympathetic members of the judiciary and recognized advocates within civil society – engage in campaigns to promote human rights. ¹⁰⁰ As these campaigns fan out, general members of the public will catch wind and seek to educate themselves on human rights norms. By the time actual proceedings begin, though, the situation country is already saturated with information on the ICC and human rights. For instance, recent survey evidence demonstrates that, amid proceedings in 2015, almost every Kenyan respondent answered 100 per cent of questions about ICC cases correctly, meaning that the Kenyan population was by that time highly knowledgeable about the Court. ¹⁰¹ The reason is that Kenyans were for years awash in information on the ICC and its pursuit of justice against its sitting president and vice president. In such an information-rich environment, the need for internet search may decline. This interpretation aligns with a constructivist understanding of ICC impact.

However, a competing rationalist interpretation is that the plateauing of human rights searches associated with trial proceedings reflects a growing realization among the pro-accountability constituency that the ICC cannot deliver. If that same constituency draws a parallel between the work of the Court and human rights politics writ large, then it may abandon rights claim-making in favour of other options for reform that are more promising. ¹⁰² Another possibility, feared by many rationalists, is that pro-accountability mobilizing in the wake of ICC involvement will actually invite a government crackdown on human rights defenders. ¹⁰³ Should this occur, we may witness an observable decline in expressed interest in the concept of human rights. ¹⁰⁴

One valid criticism of this discussion is that both of these interpretations is speculative. Indeed, more research sophisticated and context-sensitive research is needed to assess perceptions at various levels of ICC involvement. One might also challenge the use of internet search data itself. After all, Google searches for ‘human rights’ is a crude measure. How could one possibly know that Google searches for human rights indicate some a need for information, or a change in discourse, rather than a frustration with the politics of human rights? We can in part address both of these concerns at once. Google Trends does provide tools for observing what other search terms are most related to queries about human rights, including additional phrases people tend to search for. Across all English-speaking countries, the top four searches most related to human rights are ‘human rights commission’, ‘human rights declaration’, ‘human rights law’, and ‘universal declaration’. ‘UN’ is in the top ten, and ‘court’ appears in the top 15. While we cannot present all related queries from all countries, we can say that this pattern repeats in nearly every single case, and that the valence of related queries does not become more negative – e.g., paired searches for ‘bias’, ‘Western’, or ‘neo-colonial’ as ICC involvement moves from one stage to the next. In short, this browser searches for human rights are connected mostly to fact-finding or information-collection about law and institutions. Individuals turn to Google not to dig up dirt but to learn and make connections. This kind of activity increases when the ICC becomes involved in a country.

7. Conclusion

Returning to our initial discussion, the findings in this article have three implications for ICC performance reviews. First, while some suggest that there is a deep-seated backlash to human rights in the world, and disappointment with the ICC may be part and parcel of that backlash, there is little evidence of that in the data analysed here. ¹⁰⁵ Quite the opposite. In countries where the ICC has intervened, interest in human rights spiked, and sustained. The new and higher level of Google searches for human rights is quite enduring, lasting for years. Because this is a thin measure, it might be easy to dismiss as frivolous or lacking in substance. It is accurate to say they tell us very little about specific attitudes toward, or perceptions of, the ICC. Still, Google data have time and again been shown to convey latent population traits that are hard to ignore. ¹⁰⁶ It would be a mistake to write off Google search trends as meaningless. The data suggest that the ICC drives sustained curiosity about human rights, which may signal a broader ability to contribute to long-term social and ideational change. This hope has not yet been extinguished by failures perceived or actual. ¹⁰⁷

Second, an excessively rationalist account ICC outputs and outcomes will probably miss some broader constitutive social effects attributable to the Court. Evaluating performance solely based on its use of resources, the perceived fairness and expeditiousness of trial proceedings, or the extent to which victims participate is too managerial an exercise. While valuable factors to consider, they do not exhaust the full spectrum of what makes the ICC a potentially important institution. It is also not enough simply to assume that ICC processes will follow a clear logic from inputs to outputs to outcomes, or that its effects will become more predictable if only this logic is streamlined. It is quite possible that most of the ICC’s impacts are subtle and indirect. Uncovering these impacts means being open to a holistic understanding of international criminal law and engaging in the persistent and creative search for evidence of law’s social effects among audiences in various contexts.

Third and finally, evaluators need not choose between a rational systems approach or a more constructivist approach to evaluation, or trade hard-nosed practicalities of Court operations for judicial mythmaking. Rationalism and constructivism are both mental models, and they can complement one another. As the empirical data in this article suggest, neither model offers flawless predictions. After all, investigations inspire discursive shifts toward human rights, but trial proceedings do not. Understanding this in greater depth might require considering how ICC activities may possess both social and rational-strategic dimensions. Of course, this is difficult work, but considering the multi-dimensional nature of ICC performance would at the very least push us beyond either blind apology or obsession with crisis, toward a pragmatic grounding in the real opportunities the ICC offers.