CompLaw: A Coding Protocol and Database for the Comparative Study of Judicial Review

Abstract

A growing theoretical literature identifies how the process of constitutional review shapes judicial decision-making, legislative behavior, and even the constitutionality of legislation and executive actions. However, the empirical interrogation of these theoretical arguments is limited by the absence of a common protocol for coding constitutional review decisions across courts and time. We introduce such a coding protocol and database (CompLaw) of rulings by 42 constitutional courts. To illustrate the value of CompLaw, we examine a heretofore untested empirical implication about how review timing relates to rulings of unconstitutionality (Ward and Gabel 2019). First, we conduct a nuanced analysis of rulings by the French Constitutional Council over a 13-year period. We then examine the relationship between review timing and strike rates with a set of national constitutional courts in one year. Our data analysis highlights the benefits and flexibility of the CompLaw coding protocol for scholars of judicial review.

Introduction

Constitutional review of legislation is a common institutional arrangement designed to ensure the fidelity of statutes with the Constitution. A growing theoretical literature, however, suggests that this ideal may prove elusive. For a variety of reasons, we might question both the motivations and ability of courts to strike unconstitutional laws (Helmke 2005). Furthermore, the threat of review may fail to induce legislatures and executives to act constitutionally (Vanberg 2001; Fox and Stephenson 2011). Notably, many of these theoretical arguments demonstrate how variation in the process of judicial review affects both the judicial and legislative behavior and their normative implications for constitutional review (Elkins, Ginsburg and Melton 2016; Krehbiel 2016; Vanberg 2001)

It is difficult to empirically interrogate these theoretical accounts due to data limitations. We often lack a consistent and sufficiently broad – both in temporal range and institutional features – dataset. Relevant variation in judicial processes can occur within a court, between different courts in the same country, and across courts in different countries. And, of course, these processes and their effects may vary significantly over time (Navia and Ríos-Figueroa 2005). As such, developing a common coding protocol that ensures information about judicial process and rulings is comparable within and across constitutional courts and over time is not trivial.

Bearing this challenge in mind, we offer CompLaw, a new coding protocol of high courts in the world’s national judicial systems. We designed CompLaw to capture the multi-level structure of constitutional decision-making in a global context. In the cases that constitutional bodies hear, judges may be asked to resolve potentially multiple questions about the constitutionality of potentially multiple policies, each of which can be tied to a distinct responsible state authority. In a comparative context, this implies that we will want a structure that allows us to report validly on judicial resolutions of questions related to policies that are challenged within cases before courts of particular countries across years. CompLaw is designed precisely to track this kind of information.

As proof of concept, we used the protocol to code case materials and rulings from 40 countries – including courts working in a range of languages and legal traditions – for a single year. We nevertheless stress that CompLaw is designed to be built upon by our large and growing community of constitutional scholars. To show what can be gained by building upon this structure, we expanded the current dataset’s coverage to include multiple years of decisions in France, a particularly relevant case for the theoretical argument we test. Empirically, we illustrate what can be done with the 40-country, single-year dataset, as well as the dataset limited to a single country and multiple years. The multi-level structure of CompLaw offers some opportunities that are not available to some familiar cross-national datasets in law and courts. Yet, clearly scholars will need to be both careful and transparent about the consequences of the designs they choose for the interpretation of results, from either a descriptive or causal perspective.¹

We use the data to evaluate competing theoretical arguments about how the judicial review process impacts the constitutionality of legislative enactments. Separation of powers (veto player) models provide an intellectual justification for why we might believe that the threat of constitutional review improves the constitutionality of law (Brouard and Hönnige 2017). In them, purely policy-focused legislators should strategically only pass laws that can withstand judicial review, and therefore be judged constitutional by the court. But, as Fox and Stephenson (2011) show, legislators that are not exclusively policy-focused may adopt more unconstitutional laws in anticipation of judicial review than they would otherwise. The electoral incentive to pass laws that may be reversed can exceed the policy-based incentive to pass laws that are not reversed.² Thus, the normative implications of constitutional review on the legislative process differ significantly across these two theoretical accounts.

We examine one empirical claim that discriminates between these two theoretical accounts. Specifically, we investigate the relationship between the timing of judicial review and the likelihood of a high court striking a legislative action.³ The standard purely policy-motivated veto player model predicts no relationship or reversals happening more often with post-implementation review. A theoretical generalization of Fox and Stephenson (2011) that incorporates time till review, Ward and Gabel (2019, henceforth “W-G”), predicts reversals to happen more frequently under pre-implementation review. This evidence allows us to draw inferences about how judicial review influences the legislative process, and, in particular, the likelihood of a legislature to produce unconstitutional law.

We begin by introducing the CompLaw database and the coding protocol upon which it is based. We then review the theoretical arguments presented in W-G (2019) and a standard veto player model. Finally, we present both a longitudinal within-country and a single year cross-national analysis to evaluate W-G’s (2019) empirical implications and showcase the benefits of the CompLaw coding protocol.

Comparative constitutional review data

Empirical research in the field of law and courts has centered historically on the United States, yet, its full empirical scope has always been international (Dyevre 2010; Tate and Vallinder 1997). There has been an explosion of comparative research in the 21st century, and this burgeoning literature on law and courts outside the United States has addressed a diverse set of topics, often with a focus on key issues of constitutionalism. Why do governments attempt to build independent courts endowed with constitutional jurisdiction (Finkel 2008; Hirschl 2006; Ginsburg 2003), why are individuals or groups able to translate political conflicts into constitutional questions, often based on rights claims (Epp 1998; Sieder, Schjolden and Angell 2005); once accessed, what explains the decisions courts reach and the methods of interpretation they use (Carrubba, Gabel and Hankla 2008; Helmke 2005; Lasser 2004; Kapiszewski 2012); following a resolution, what explains differences in the implementation of court orders (Vanberg 2005; Carlin et al. 2022); and, ultimately, why are some courts able to constrain governments while others are not (Alter 2009).

One of the most glaring holes in our empirical arsenal remains the absence of a broad, cross-national database of constitutional review decisions. To be sure, the field has numerous cross-national datasets summarizing the constitutional review powers of courts (e.g., Elkins, Ginsburg and Melton 2012; Brinks and Blass 2018; Ginsburg and Versteeg 2014). There are many single-country datasets of constitutional decision-making of various types, many of which are described in Garoupa, Gill and Tiede’s (2021) excellent review of the field. What we lack are data that allow for the comparison of constitutional review across varying institutional contexts. The National High Courts Database (Haynie et al. 2007) project is a notable exception, yet it summarizes decisions from only 11 English-speaking common law countries. While extremely helpful, this sample captures limited variation in constitutional review around the world.

There are two consequences of this lacuna. First, our most basic descriptive information about constitutional conflicts depends almost entirely on stitching together studies of single courts. The worldwide coverage of such studies is incomplete and uneven, and for this reason, we lack an unbiased picture of the participants, questions, sources of law, methods of interpretation and outcomes of constitutional review around the world. Compare this to the Supreme Court Database in the United States, which provides precisely this type of information for one court with constitutional jurisdiction. It has fueled hundreds of studies over the last 30 years. Second, insofar as many of our theoretical claims involve causal factors that often do not vary much within a country (e.g., public legitimacy of the court) but do vary substantially cross-nationally, the lack of cross-national data means that we are simply unable to adequately test our models.

The CompLaw approach

Creating a large, representative sample of cross-nationally comparable data requires addressing a number of practical challenges. Constitutional review is carried out around the world in many different ways, by a variety of different types of courts and in multiple languages (e.g., Navia and Ríos-Figueroa 2005). The massive number of constitutional resolutions produced by the world’s legal systems, the lack of international standards for data storage and the fact that in many parts of the world there are simply no electronic records at all for constitutional decisions, make it practically impossible to capture anything close to the universe of resolutions. Since the quality of data storage is likely correlated with economic development, any sample of constitutional decisions, however and wherever obtained, will likely overrepresent wealthier states. As we will show, this is true of our own data, and it likely represents a practically impossible problem to solve without sufficient resources to conduct detailed historical research in the archives of the judiciaries of all states.

With these challenges in mind, our team set out to establish a protocol for collecting, recording, and sharing information, one that could be a model for future projects and explicitly expanded upon by other scholars. This work produces a dataset, which we use to evaluate empirical claims about constitutional review; however, it is important to stress that the coding protocol and database management system we developed is intended as the foundational contribution. The data created through this system are constructed as a sample designed to evaluate the utility of our proposed solution. We invite the scholarly community to adopt and expand upon what we have created. We thus organize this section into two parts. We first describe what the CompLaw database measures. We then describe how we collected the information to ensure validity, reliability, and reproducibility.

What we measure

We organized our process around what we take to be the universal structure of constitutional review. Although constitutional procedures vary tremendously across constitutional instruments, legal traditions, and over time within the same tradition, the essential context in which constitutional review takes place is remarkably constant. This structure takes the following form:

A court is asked by a complainant to declare whether a policy of the state violates one or more provisions of a constitution.

This structure clarifies a few elements of our task. First, we are concerned with state “policies,” which we will conceptualize broadly as including statutes, executive orders, enforcement actions, administrative acts, or decrees. Second, we are concerned with the allegations of some party that the behavior of the state is inconsistent with constitutional limits on its authority. Third, a court must find or declare whether these policies are consistent with potentially multiple elements of a constitution. For this reason, each case may have multiple declarations. This also sharpens our focus on declarations of constitutionality, rather than whether a particular party to a litigation “won” a case, though we have compiled sufficient information to allow for this kind of measurement should it be of interest.

It is important to recognize that measuring constitutional review in this context will result in hierarchically oriented data, information that varies at the level of the state in which the court operates; the court itself; the case before the court; the policy or policies challenged in that case; and, since the court may be called upon to declare the constitutionality of a policy with respect to multiple elements of a constitution, the precise constitutional question that the court answers within the case.

The scope of the CompLaw dataset is limited by several selection rules, which are largely focused on the court carrying out the review. First, we focus on courts of last resort that exercise constitutional review in the highest level of a state’s legal system. This can be a regular “Supreme Court/Tribunal.” In countries in which there is a separate constitutional court, we selected that entity. In federal states, we concentrate on the federal legal system, so that we are measuring the decisions of the court of last resort exercising constitutional review in the federal system. Second, we code only cases involving constitutional review. Third, we included courts that published their full text decisions online. We are not limited to English language translations of decisions. Fourth, due to resource constraints, the cross-national sample consists of decisions issued in (or near) the year 2003.⁴ We also coded a longitudinal sample for France to allow for over-time analysis. Fifth, and again for tractability, for courts with more than 200 rulings that fit our criteria, we coded a random sample of at least 200 rulings. In such instances, we still uploaded all of the cases to our database management system. Our final case selection rule focuses on parties. We required either that (1) the state (national government or the national legislature) was an active participant in the case or (2) that a “policy” of the state was challenged as unconstitutional.

The CompLaw courts and their states

In total, the CompLaw database currently includes information on 10,540 cases, of which 2,968 were considered germane under our case selection rules and thus were coded. The highest level of aggregation in the database concerns information on the state within which each court of last resort operates. We have assembled a database of over 100 extant measures of judicial independence, legal traditions, regime characteristics, and public trust in courts and the legal system. All scores are compiled from well-known sources in the fields of law, comparative politics and political economy (e.g., Elkins and Ginsburg 2012; Goemans, Gleditsch and Chiozza 2009; Marshall, Gurr and Jaggers 2010; Theodore et al. 2001). These measures are then linked to courts and their case files.

Table 1 lists the courts included in the database. To date, the database consists of 44 courts, though some of the cases, we selected resulted in no germane cases for our analysis.

Table 1. States whose courts with constitutional jurisdiction are included in CompLaw

Note: “CC”: constitutional court. “CE”: high administrative court. * no cases were germane under CompLaw rules. We also have uploaded the documents necessary to code cases for an additional 28 countries: Algeria (CE), Bahamas, Barbados, Belgium (CE), Brazil, Bulgaria (CE), Cambodia, Czech Republic, Denmark, Estonia, Finland, Georgia, Guam, Iceland, Jamaica, Japan, Latvia, Lebanon, Mexico, Netherlands, Norway, Portugal, Romania, Slovenia, Taiwan, Trinidad & Tobago, Uganda, and United Kingdom. They are available for scholars wishing to analyze those countries.

Figure 1 allows us to consider how well the current CompLaw database reflects the distribution of all states along theoretically important features. The figure shows kernel density estimates of the distributions of four key concepts with respect to the full CompLaw sample of states and the universe of states during the year 2003. The upper-left panel shows the distribution of economic development as measured by the natural log of gross domestic product per capita, expressed in current U.S. dollars (Inklaar et al. 2018). Although the full range of development is captured by the CompLaw sample of states, our database covers far fewer of the poorest states in the world. The sample average for CompLaw is approximately 9.33, whereas it is only 8.84 in the full sample of states. The upper-right panel, which shows distributions of the level of democracy as measured by the Polity IV project’s 21 point scale of regimes (Marshall, Gurr and Jaggers 2013), helps explain the oversample of wealthier countries. The CompLaw sample consists almost entirely of states that were highly democratic in 2003. Only Niger, Burkina Faso, and Algeria have Polity IV scores of 6 or less.

Figure 1. Kernel density estimates of economic development, level of democracy, de facto and de jure judicial independence for the year 2003.

The bottom-left panel shows Linzer and Staton’s (2015) estimates of de facto latent judicial independence (LJI). The bottom-right panel shows Voigt, Gutmann and Feld’s (2015) estimates of de jure independence. Both of these indices lie on the unit interval, with higher scores reflecting a higher degree of independence. Although the CompLaw sample average for LJI (0.64) is higher than the average for all states in 2003 (0.48), it captures the bimodal shape of the LJI distribution – i.e., the CompLaw sample includes courts with low levels of independence. Further, the bottom-right panel suggests that, with respect to de jure independence, the CompLaw sample is highly representative. Although the CompLaw sample has a higher average (0.67 vs. 0.63) and a lower standard deviation (0.12 vs. 0.14) than the full sample of states in 2003, the differences are both very small and not substantively meaningful. Our selection of a sample based on whether courts publish their 2003 rulings online does imply that the CompLaw sample overrepresents economically developed states. That said, with respect to democracies, the courts in the CompLaw sample generally represent the variation in judicial independence and other salient characteristics globally.

Information below the level of the state or court

Table 2 summarizes the information that we collect at the three levels of the data hierarchy below the level of the state or court. In the supplemental materials, we provide the full codebook for this information as well as training materials. As we hope is clear, the information we collect is relatively sparse. Although a more fine-grained approach would be appropriate for a single-country source, we found that this sparse approach allowed us to successfully code all of the courts selected for the study. Another feature of our approach is that measurement involves recording directly observable information. Relevant dates, names of parties and judges, titles of policies, elements of constitutions invoked, the type of constitutional review, among other features are explicitly named in the record. The declaration of whether a policy is unconstitutional is also directly observable. Notice here that we are not attempting to measure the meaning of constitutional rules developed in many of these cases as they might bear on future behavior. That exercise would have surely required inference to deal with considerable uncertainty about what a case might stand for. Although we have collected sufficient data to make these types of judgments, the current dataset does not attempt to measure this kind of information. For these reasons, the issue of measurement validity is eased in terms of identifying what is meant by the concepts we measure. In many cases (e.g., names, dates, what is a plenary session, etc.), this involved very little trouble at all. In limited cases, training was required to ensure that all members of the team were on the same page. But even in these cases, the task was not overly complicated. For example, once you know that “concrete review” requires the resolution of legal conflict caused by the application of a state’s policy, it is not particularly difficult to read the case facts and infer whether the review was carried out concretely or in the abstract. This is not to say that there were no sources of uncertainty in the process, but relative to measuring a concept like “judicial independence,” “judicial activism,” or “clarity of language,” these items were fairly straightforward to measure. Critically, our approach described in the next session ensures that all choices can be inspected and questioned and re-coded if a scholar would like.

Table 2. The CompLaw variables and their descriptions

Note: For further details about the collection and description of the French Constitutional Court cases that we collected, see Gabel and Ziegler (2021).

Case-level data Case level variables in CompLaw include a variety of identifiers, as well as information on the dates of admission and decision. CompLaw also contains information on the type of review (e.g., concrete or abstract), as well as the precise name of the constitutional institution (e.g., amparo) being used. The case’s complainant, that is, the party alleging a constitutional violation, is recorded at the level of the case; however, we also allow the complainant to vary by policy challenged. This is theoretically possible, though it did not occur in the study. Figure 2 summarizes that information for our current sample of coded cases. By far the most common complainant is an individual (e.g., writs of amparo from courts in Latin America). We record the party responsible for the constitutional violation at a lower level of analysis.

Figure 2. Distribution of the CompLaw sample by complainant type and policy type.

Although CompLaw does not code individual judge decisions, we do include indicators for whether there was any disagreement among the judges assigned to the case, as well as a count of the number of dissenting positions. This allows researchers to measure the strength of the majority position, as well as identify the cases with judge-level variation for future coding.

Policy-level data Three key pieces of information are recorded at the policy level – the precise name of the policy, the type of policy being challenged, and the year in which the policy was enacted or otherwise promulgated. Notably, some cases involve the review of multiple parts of law that may have been written at different times, with the constitutionality of each part coded separately. We consider each of these parts of the law as a “policy.” Among the set of cases that are currently coded, there are 4,192 distinct policies. As we can also see in Figure 2, although national statutes are the most common policies attacked in our sample, a variety of other policy types are challenged, including agency actions, executive orders, and international treaties.

Question-level data Policies are attacked on numerous constitutional grounds. For each policy, we code the constitutional provision on which the complainant’s argument is based. There are 10,476 distinct constitutional questions (or bases for a constitutional challenge) in CompLaw. The average number per case is 4.1. For each question, we code whether the court found that the policy was unconstitutional with respect to the piece of the constitution motivating the argument. Preliminary analysis suggested that there were cases in which the court’s decision was somewhat ambiguous as to the constitutionality with respect to a specific provision. To address this possibility, coders indicated their level of certainty of the policy’s constitutional status in light of the ruling. For the most part, our coders were fairly sure of the policy’s constitutional status. Only 81 questions were coded to indicate significant uncertainty.

How we collected information

Given the scale of the measurement task and the relative uniqueness of the endeavor, our first goal was to ensure that our work could be easily shared, replicated, and built upon. Thus, the first step in our process involved the construction of a user-friendly database management system, which could be accessed by any team member with the appropriate credentials, from anywhere in the world with access to the internet. The system continues to function and may be used by scholars who wish to contribute to the project.

Figure 3 displays a screenshot of the main page from the database management system. The system provides access to all training materials, instructions for data collection, as well as the full-text final sentences of the cases we coded. All information entered into the system is updated in real-time. All records are kept so that errors can be tracked and fixed if necessary. Thus, every decision that we made can be questioned and evaluated. All of the work we summarize can be reproduced exactly. The system also provides a way of tracking work in real-time, and most importantly, for allowing communication between individuals recording information about a case and project leaders (accessible via the “Ask Your PI” link in the upper-right). This feature allowed trained coders to check in with PI’s on questions about how to code particular issues when uncertain. The system permits amendment as well: Additions can be made to the coding protocol, and conceptual definitions can be changed. To carry out the work, we built an interdisciplinary and multilingual team of research assistants drawn from social science programs and schools of law at Washington University, Emory University and University of Rochester. The assistants were tasked with a four-step process, divided into an “uploading” and a “coding” stage. For each country in the world in 2003, uploaders were asked to identify whether a court in that country exercised constitutional review. If so, they were asked to identify the court of last resort in this system. Once identified, we asked uploaders to consider whether the full-text, final sentences of this court from 2003 were available on the internet. If so, assistants uploaded these final sentences to our server, subject to the 200 case limit. With the cases uploaded for each court, coders took over and applied the coding protocol to each case.

Figure 3. Screenshot of CompLaw’s database management system landing page for a team member with administrative access. Burkina Faso was counted twice in the system and results in 45 countries.

Project staff extensively trained assistants assigned the task of uploading and coding. In the supplemental materials, we include the training manual as well as a representative example of uploading instructions. Uploading involved close contact with project leaders (the authors) to ensure that we had identified the correct courts. The scale of the coding task was substantial, even with the information we collected being directly observable. We developed a training manual for all coding activities, which we applied to a set of training final sentences (excluded from the final analysis), all translated into English. All coders trained until there was complete consensus about how to evaluate the training set. Once coders reached agreement, they were permitted to code cases for the database.

Having a common protocol and data at the case-, policy-, and question-level are valuable for examining numerous empirical implications, especially in the growing theoretical literature on constitutional review. To illustrate the value of CompLaw, we use the data generated by our coding protocol to investigate an empirical implication of one such theoretical argument by Ward and Gabel Ward and Gabel (2019).

The timing of judicial review and rulings of unconstitutionality

W-G (2019) present a game theoretic model that examines how the timing of judicial review affects 1) the behavior of legislators in adopting statutes and 2) the behavior of the constitutional court in reviewing those laws.⁵

Specifically, they compare a setting in which legislation is reviewed ex ante – before the law is promulgated – with ex post review, which takes place after the law has been implemented and has had an opportunity to have concrete policy effects. These are common forms of judicial review, with some courts exercising both forms (Navia and Ríos-Figueroa 2005; Ginsburg 2008; Corkin 2015). The key distinction is that they differ in the level of concrete policy effects realized from the legislation before review.

The model involves three actors: a legislator, a constitutional court, and an electorate. The legislator cares about achieving their preferred policy and about other concerns unrelated to policy. These could be electoral concerns, which is the substantive example used in W-G (2019), but they could also be career or personal goals (i.e., leisure time) that are affected by the legislative process and its outcome. For example, legislators might use the legislation to posture for important electoral constituents, which could require passing legislation that is dubious constitutionally. It could also involve constitutionally questionable legislative procedural moves (e.g., introducing non-germane amendments or restricting parliamentary debate), or careless lawmaking due to inattention to detail that constitutional review would discover and reject. For its part, the court seeks to issue correct constitutional rulings on legislation and does so as a function of 1) its own assessment of constitutionality and 2) the strategic incentives of the legislature to adopt unconstitutional laws.

Finally, the electorate wants to retain competent legislators and remove incompetent ones. The electorate infers the competence of the legislator from the observed legislation and court rulings on constitutionality. Voters use that inference to inform their choice to retain or replace the legislators. Based on these strategic considerations, W-G (2019) solve the model and compare the behavior observed in equilibrium under the two forms of judicial review.

The formal analysis shows that lawmakers generally legislate constitutionally more under ex post than ex ante review. The reason is that the review timing affects the severity of the “moral hazard” problem caused by judicial review. Because legislators will be “bailed out” by the court for irresponsible lawmaking, legislators are induced to behave unconstitutionally to achieve their non-policy goals. But that inducement decreases with the expected policy costs that will be realized prior to constitutional review. That is, the shadow of future policy costs from unconstitutional lawmaking induces more responsible legislative behavior.

Moreover, like Fox and Stephenson (2011), the model shows that moral hazard from judicial review reduces voter welfare. When legislators are induced to posture, they reduce voters’ ability to discriminate among types of legislators and to use elections to remove incompetent representatives. Critically, though, this pernicious effect varies in W-G (2019) with 1) the timing of review and 2) the relative importance of legislators’ policy and non-policy goals. If legislators are primarily motivated by policy, ex post review is better for voters because representatives posture less. However, once legislators’ relative value of policy sufficiently declines, voters fare better under ex ante review because it prevents bad policies from being implemented.

The key empirical implication of W-G’s (2019) model for our purposes is that, as long as legislators are motivated by some non-policy goals, the court’s propensity to rule laws unconstitutional (i.e., the strike rate) should be lower when review follows some policy consequences (ex post) than when the law is reviewed before policy consequences are realized (ex ante) (Ward and Gabel 2019, Appendix). This relationship is a result of 1) the decreasing incentive for legislators to posture as the shadow of future policy costs from unconstitutional laws grows and 2) the incentives for the court to rule laws unconstitutional given the observed legislative incentives and behavior. Intuitively, the court’s willingness to defer to the judgment of legislators depends on whether legislators have incentives to posture. Stated more generally, strike rates should decline with the time elapsed between the enactment of a law and the court’s review of that law.

This empirical implication is inconsistent with expectations from the class of separation of powers, or veto-player, models (e.g., Brouard and Hönnige (2017)). In that model, legislators are motivated exclusively by policy.⁶ Under ex ante review, the legislators would never pass a law that they anticipate would be rejected by the court upon immediate constitutional review. The legislators would have instead adjusted the law to accommodate the court’s preferences or, if that were not possible, simply refrained from legislating. This self-disciplining is referred to as “auto-limitation” (Vanberg 1998; Brouard 2009). Note that if legislators practice auto-limitation, judicial review has benign properties with respect to the constitutional quality of lawmaking, a sharp contrast with the consequences of moral hazard described above.⁷

What does the veto player model imply about the association between review timing and strike rates? For laws reviewed ex post, the legislators can achieve their policy goals from unconstitutional laws for at least a limited period of time. For laws reviewed ex ante, they cannot realize any policy goals before review. Consequently, legislators should pass weakly more unconstitutional laws under ex post review, and we should see these laws struck. More generally, we should observe weakly increasing strike rates as the expected time before review grows (and the opportunity for policy effects increases), which is inconsistent with the implication from W-G (2019).⁸ Thus, bringing evidence to bear on how review timing is related to strike rates allows for a discriminating test of these two theoretical models.

Note, while the W-G (2019) model considers ex ante or ex post review separately, the implications of the model apply in a context where both ex ante and ex post review of legislation are available. The main insights from the model apply so long as legislators, when legislating, can anticipate if and when their legislation might be reviewed for constitutionality. The moral hazard problem should be strongest when legislators expect ex ante or very early review, and it should dissipate with the expected duration before review. Similarly, the implications of the veto player model for review timing should also apply to contexts where legislation could face review soon after or much later than the law’s enactment. Again, this should hold, so long as legislators can reasonably anticipate the timing of review when writing laws. We address this assumption below, where we describe our empirical strategy for estimating and interpreting the association between review timing and strike rates.

The empirical association between review timing and strike rates

Our empirical strategy for estimating the relationship between review timing and strike rates has three general elements. First, we use the CompLaw coding protocol to code rulings and relevant information about the timing of review. Specifically, our unit of analysis is the “policy,” which is the law or part of a law reviewed by the court, and we use the “strike” variable, which is coded (1) if the court struck that policy as unconstitutional and (0) otherwise to indicate whether the policy was found unconstitutional. We focus on reviews of statutes (ordinary legislation).

Second, while we provide estimates based on cross-court comparisons, we focus primarily on comparing strike rates across review procedures for laws generated by the same legislature (and reviewed by the same constitutional court). This allows us to control for the myriad national factors that would confound a cross-national analysis. For example, a variety of court-level factors (e.g., the constitutional order available for review or attributes of court membership). Similarly, differences in the legislative context – e.g., the distance to elections, domestic political concerns, or the substantive agenda of the government – could affect the incentives for legislators to posture and pursue unconstitutional legislation. Thus, we constrain our analyses to courts with the ability to review laws at different durations since the laws’ adoption. As noted above, the theoretical expectations should apply to such settings.

Third, we use the actual timing of review (the time since the law was enacted) and the form of review (abstract or concrete) to distinguish laws that legislators would have anticipated as targets of early or ex ante review. As we describe in the two empirical applications, the prospect of abstract review – the review of constitutionality without a concrete claim of harm – is often easy to anticipate because the limited set of actors who can initiate such review (e.g., the legislative opposition or a sub-national government) typically register their constitutional concerns publicly and during the legislative session. This is particularly true of ex ante review, which is always abstract and which can generally only be initiated by a small set of political actors that commonly voice concerns and forecast their appeals during the legislative process. Thus, we expect that the legislators who write laws that face ex ante review or abstract review immediately after the law’s enactment were generally aware of the ensuing review. And, when those same legislators write a law and ex ante or very early abstract review is not forecast during the legislative session, we would expect them to assume their law will realize some policy consequences before any review.

Based on this empirical strategy, we conduct two separate analyses. We first estimate the relationship between timing and strike rates with rulings over multiple years by one constitutional court – the French Constitutional Council (FCC). The FCC has had the authority to rule on the constitutionality of laws challenged prior to their promulgation (ex ante) since its creation under the Fifth Republic. And, due to a procedural change in 2010, the FCC gained the authority to hear ex post constitutional challenges after the law is promulgated and has had time to have policy impact. Second, we extend this analysis of strike rates and the timing of review to the courts in the CompLaw dataset described in the previous section. First, we examine the same relationship as in France, comparing the strike rates under ex ante and ex post review. In addition, we expand the analysis to include courts that lack the ex ante form of review but that can review laws immediately, or very soon after their adoption, and thus before the law has had much, if any, policy effect. We then compare how strike rates compare between laws reviewed very soon after adoption and laws reviewed later. While neither analysis is causal, each does provide evidence consistent with the W-G theoretical models while accounting for an extensive set of important potential confounding factors. Both analyses highlight the benefit of the CompLaw coding protocol to study important questions of judicial review.

Judicial review by the French Constitutional Council

In our first application, we use the CompLaw protocol to code rulings by the FCC on the constitutionality of legislation from 2002–2015. The FCC has several attractive features for testing the effect of review timing on strike rates. First, the FCC currently decides constitutional appeals of legislation brought through ex ante and ex post procedures. All ordinary legislation can be challenged on constitutional grounds after it is approved by the legislature but before its promulgation. The bulk of these appeals are submitted by a minority of legislators (e.g., opposition parties) (Brouard 2009).⁹ A small fraction of laws are challenged ex ante. ¹⁰

In March 2010, the FCC began to exercise ex post review of constitutional challenges to laws. These rulings are on appeals from litigants in active cases in French courts through the la question prioritaire de constitutionalité (QPC) procedure. Any litigant engaged in an active case in a French court can request that the FCC review the constitutionality of a statute relevant in the instant proceedings. The appeal is indirect, as it must survive screening for admissibility by both the instant court and the top appellate court.¹¹ This review is designed to filter out cases that are not serious or valid constitutional appeals.

Second, the French legislative process and its anticipation of constitutional review fit nicely with the theoretical argument from W-G (2019). French legislators generally learn during the legislative process if a law under consideration raises constitutional issues and whether the law is likely to face ex ante review or not. The government’s legal adviser, the Council of State, routinely reports on the constitutionality of proposed statutes. Furthermore, legislators (including the opposition) have specific procedural rights to raise constitutional questions during debate (Stone Sweet 2000, 104). This provides a clear signal about the intent of the opposition to bring an ex ante constitutional challenge (Stone Sweet 2000, 104). Finally, French legislators have electoral considerations that could motivate them to posture to the electorate and exploit the “bail out” afforded by ex ante review, as in the W-G model (2019) (e.g., Brouard 2009). For an excellent example, consider Brouard’s (Brouard 2009, 395) account of the 2007 bill introduced by the legislative majority of the newly elected President Sarkozy. During the campaign, President Sarkozy had promised to adopt legislation that introduced a mortgage deduction for homeowners. Soon after this election, his party put forward legislation with that purpose. But, as the finance minister admitted publicly just before the legislative vote, a mortgage deduction was clearly unconstitutional. Nevertheless, the legislative majority made good on President Sarkozy’s campaign promise and passed the law. The FCC then struck the law as unconstitutional.

Third, the detailed contextual information about these review procedures allows us to evaluate whether the selection process of constitutional appeals biases our analysis. In particular, we are concerned that, for reasons unrelated to the argument in W-G (2019), ex post appeals may be systematically weaker than ex ante appeals.¹² As a result, the laws challenged ex post would be less likely to be struck. For example, one might suspect that ex post appeals are of lower quality because the QPC procedure was relatively new and litigants and lawyers would have had little experience with such constitutional challenges.

This selection bias concern is unfounded. As mentioned above, all QPC appeals must typically survive two screenings for admissibility. The vetting ensures that constitutional questions clearly pertain to a valid piece of legislation, that the question has not been decided previously, and that the constitutional issue is serious (De Visser 2014, 137). The screening process is, thus, designed to identify questions that are “worth sending to the Constitutional Council" (Dyevre 2013, 743). In practice, this process has eliminated the vast majority of requests for review, with the appellate courts alone having rejected 76 percent of requests before they reached the FCC.¹³ Indeed, French legislators have considered the level of scrutiny excessive and have called for reforms to allow more cases to be reviewed by the FCC under the QPC procedure (Dyevre 2013, 750). And, as a former FCC judge concluded, one could easily argue that ex post challenges are, in fact, of higher quality than ex ante challenges (de Lamothe 2012).¹⁴ Consequently, we would not expect ex post appeals to be systematically weaker in quality than ex ante appeals.

In sum, the legislative majority, when drafting, debating, and ultimately passing a bill, is typically aware if the bill raises constitutional issues and if the law will be subject to ex ante review or not. Consequently, legislators can anticipate whether a law is likely to have concrete policy effects or not before review. If the argument in W-G (2019) holds, we should expect to see higher strike rates on laws reviewed ex ante than on laws reviewed ex post. ¹⁵

To examine this claim, we used the CompLaw protocol to code all FCC rulings on the constitutionality of ordinary legislation adopted from 2002–2015.¹⁶ This period covers three legislative sessions – 2002–2007, 2007–2012, 20012–2017 (partial). It includes 1,259 ex ante rulings and 434 QPC (ex post) rulings on articles in ordinary laws. Many appeals, particularly for ex ante review, involve rulings on the constitutionality of multiple articles in the same law. Thus, the number of rulings (1,693) in the analysis is higher than the number of cases (464).¹⁷ We fit a logistic regression model to estimate the relationship between the timing of review and likelihood of the FCC striking legislation. The key independent variable is Ex Ante Review, which is coded (1) for instances of ex ante review and 0 for review under the QPC procedure.¹⁸ We estimate standard errors clustered by case. Since the same case can produce multiple rulings, we should not assume these observations are independent. The estimated coefficient from the logistic regression is presented in Model 1 of Table 3. Legislation reviewed ex ante was about 3 times more likely (odds ratio = 2.97, 95% confidence interval: 1.91–4.62), to be found unconstitutional than legislation reviewed ex post. Model 2 in Table 3 adds controls for potential confounding factors that vary across legislative sessions. For one, the level of ideological congruence between the FCC and the National Assembly could affect strike rates. During the 2002–2015 period, the ideological balance of the FCC was consistently center-right. At least 2/3 of the judges on the FCC had been appointed by an institution controlled by that center-right Gaullist party (the Union for a Popular Movement or its predecessor, the Rally for the Republic).¹⁹ From 2002–2012, that same party also controlled the presidency and the legislature. This changed in 2012 with the election of a Socialist president and, shortly thereafter, a Socialist led National Assembly with the Gaullists in opposition.²⁰ As a result, for most of this period, the legislative opposition (the Socialists) faced an ideologically opposed Constitutional Council. This is important for what it implies about the expected strike rate. According to Hönnige (2009) and (Brouard, 2009, 394), the divergence in preferences between the FCC and the Socialists should cause Socialist-authored laws to face higher strike rates by the FCC than when the Gaullists held legislative power. In particular, this effect under the Socialists could be higher for laws that are reviewed ex ante, since the Gaullist opposition would be the main source of those appeals. Thus, we would want to distinguish the unified ideological period from the period of ideological divergence.

Table 3. Association between review timing and the likelihood of striking legislation at French Constitutional Council

Note: The estimated logistic regression coefficients (or log odds ratios) are shown with clustered standard errors by case in parentheses.

We might also be concerned about other legislative contextual factors. For example, although the Gaullists maintained control of the presidency and the National Assembly from 2002–2012, the leadership changed. President Chirac was replaced by President Sarkozy, who appointed a new prime minister, Francois Fillon. As a result, the 2007–2012 legislature was distinct from the 2002–2007 legislature in a number of ways (e.g., legislative agenda). It also faced a different political and economic climate (e.g., the financial crisis and a dismal showing for the Gaullists in regional elections). These differences could influence the incentives for legislators to posture for electoral advantage, which is a factor in the W-G (2019) model.

To control for these contextual factors, Model 2 includes controls for the legislative session of the law under review. By including these controls, our estimate for review timing is based on comparisons of ex ante and ex post review of laws from the same legislative period. The results of Model 2 are extremely similar to those in Model 1, with laws reviewed ex ante 2.97 times (95% CI: 1.91–4.62) more likely to be struck than laws reviewed ex post.

The final potential confound we consider involves the electoral cycle. Brouard (2008, 399) argues that legislators have a stronger incentive to posture and take potentially unconstitutional policy positions in the run-up to and the period immediately following an election than in the middle of the legislative term. This argument is generally consistent with the theoretical motivation of W-G (2019), which assumes such non-policy goals affect legislative behavior. However, this argument is not about review timing, and it could confound our estimation of the relationship between strike rates and the timing of review.²¹ Our concern is that, to the extent the election cycle motivates legislators to generally pass more constitutionally suspect laws (regardless of the anticipated timing of review), then we would expect higher strike rates in general in the period before and after the elections. And, if those laws are reviewed ex ante, then an observed higher strike rate on ex ante appeals than ex post appeals during a legislative session may simply be a reflection of the differential rate of ex ante appeals over the election cycle. It would not reflect legislators’ anticipated timing of review. We address this concern by including a control variable for the six-month pre-election and post-election periods. Model 3 of Table 3 shows that the inclusion of these two controls does not affect the inferences about review timing: laws reviewed ex ante were 3.18 (95% CI: 2.06–4.91) times more likely to be struck than laws reviewed ex post. As a last robustness check, in Model 4 of Table 3, we estimated the same model, but only for legislation passed after the introduction of the QPC procedure in March 2010. Thus, we consider only legislatures that faced the potential for both forms of review. The results again show that laws reviewed ex ante are more than 3 times (3.67, 95% CI: 1/90–7.08) more likely to be struck as unconstitutional. This shows that the general results were not driven by laws passed by legislatures pre-reform, when laws that escaped ex ante review were not expected to ever face constitutional review.

These results are consistent with W-G (2019): laws that are reviewed ex ante are more likely to be deemed unconstitutional than laws reviewed ex post. These results are also inconsistent with the veto player model. It is important to highlight the magnitude of the strike rate for ex ante review: 48% overall. That is consistent with a similar analysis (e.g., Brouard (2009)) and is extremely difficult to reconcile with the veto player model. The legislative majority, writing laws in anticipation of ex ante constitutional review, fails almost half the time in passing constitutional review. And this cannot be explained by legislators’ unfamiliarity or uncertainty about the views of the court (Brouard, 2009, 393). For most of the period under study, the legislative majority was writing laws reviewed by a majority on the court that they or their co-partisans had appointed.

We now examine whether this relationship between review timing and strike rates in France occurs more generally among constitutional courts.

Judicial review by constitutional courts in the CompLaw dataset

Our second analysis focuses on constitutional courts in the CompLaw dataset that feature forms of review that allow distinct differences in the timing of review within the same court. We first select courts that allow a straightforward extension of the analysis from the French Constitutional Council: courts that review laws ex ante and ex post. Second, we expand that set of courts to also include ones that do not have ex ante review powers but that conduct abstract review of laws immediately after laws are adopted and also review laws at later points in time.

Strike rates under ex ante and ex post review

We begin by comparing the strike rates for rulings from ex ante review with rulings from ex post review. As in the FCC setting, the unit of analysis is the “policy” and we use the “strike” variable to indicate whether the policy was found unconstitutional. Ex ante rulings could only be delivered by 10 courts in our database.²² For rulings from those courts, we used the CompLaw variable for form of review (abstract or concrete) and the variable that identifies the year the statute was adopted to identify the relevant rulings. All ex ante rulings are abstract in form. Specifically, we let $ ExAnteReview=1 $ denote when a ruling was from abstract review of a 2002 or 2003 statute, and $ ExAnteReview=0 $ for ex post review rulings.²³ We fit a logistic regression model to estimate the effect of review timing on the likelihood of the court striking legislation. These data have an obvious multi-level character. The outcome and key explanatory variables are measured at the level of the individual ruling, but rulings are associated with specific courts embedded in their own national political and institutional setting. We therefore estimate a multi-level model with random effects for each court and robust standard errors.²⁴ The results, presented in Model 1 of Table 4, provide preliminary evidence consistent with our expectations. Based on ruling by 39 courts, ex ante rulings were more likely to be struck than ex post rulings.²⁵

Table 4. Association between review timing and striking legislation by courts in the CompLaw database.

Note: The estimated logistic regression coefficients (or log odds ratios) are shown with robust standard errors (Models 1 and 4) or clustered standard errors by case (Models 2, 3, 5, and 6) in parentheses. We include random effects for each court in Models 1 and 4 and fixed effects for each court in all other models. Models 1 and 4 report results for all courts. Models 2 and 3 report results for courts that exercise both ex ante and ex post judicial review. Models 5 and 6 report results for courts that exercise abstract review and that rule ex post. Models 3 and 6 report results only for courts where missing data about the age of the legislation was small or nonexistent. Full results for these analyses are presented in the Appendix.

While these results are suggestive, the analysis reflects comparisons across courts, some of which only rule in ex ante or ex post cases. Ideally, we would like to compare only strike rates within the same constitutional, judicial, and legislative system. Accordingly, we include court fixed-effects in the model and focus the analysis only on courts that ruled both ex ante and ex post. This limits our analysis to nine courts: the 10 courts with ex ante review identified above but without the French Constitutional Council, as it only issued ex ante rulings in 2003. This empirical strategy ensures that the estimated effect of the review timing on rulings is due to intra-court differences in strike rates. We estimate robust standard errors clustered by case. Since the same case can produce multiple rulings (multiple articles in a law could be contested), we do not assume these observations are independent.

The results of the logit model for the nine courts are presented in Model 2 of Table 4 and match our expectations based on W-G (2019) and are consistent with the FCC findings. A law reviewed under ex ante review had 2.06 (95% CI: 1.26–3.37) greater odds of being struck as unconstitutional compared to a law reviewed ex post.

We are concerned with data limitations for the Polish and El Salvadoran courts, which had some missing data for the year of the law under review. Consequently, we estimate the effect of review timing for the subset of seven courts (without Poland and El Salvador) as a robustness test. Model 3 in Table 4 reports the results, which are very similar to those for the set of nine courts. The strike rate is 2.31 (95% CI: 1.33–3.99) times higher for rulings under ex ante review. These results comport nicely with those from the French Constitutional Council, suggesting that the W-G (2019) model applies more broadly.

Strike rates under early abstract and later review

This pattern should apply in a broader set of courts, provided we can identify conditions that would distinguish rulings according to the likelihood legislators anticipated concrete policy effects before review. To this end, we use the form of review – abstract or concrete – and the year the legislation was adopted to make the distinction between rulings on laws that were written by legislators that expected early judicial review and rulings that involve laws for which the legislators likely anticipated review to occur well after the law had policy effects, if at all.

As discussed in the French context earlier, legislators can receive signals about the prospect of judicial review, and its timing, during the legislative process. The nature of that signal depends on the forms of judicial review available in that country. Abstract review does not require a concrete claim of harm from the law, and thus it could occur before any policy consequences from the law are realized. Further, standing to bring abstract review is often limited to salient political actors (e.g., a minority in the legislature or a sub-national government) who can credibly threaten to challenge a law and who can articulate those concerns during the legislative process. As Bricker (2016: 7) notes, “in many ways, this system of standing provides the best opportunity for interested and knowledgeable parties to quickly utilize the judicial process and resolve constitution issues.” Where such parties intend to pursue immediate abstract review, legislators typically learn of these intentions during the legislative process and anticipate the law will likely have little, if any, concrete policy effects before review (e.g., Landfried 1994; Brouard 2009; Vanberg 1998).

While this is most apparent with ex ante review, legislators can also reach similar conclusions in settings where abstract review is conducted ex post but soon after the law is passed. Consider, for example, Germany, which only allows ex post abstract review of statutes. Parties with standing to initiate abstract review (e.g., a minority of legislators) can and do reveal their intentions to seek immediate ex post abstract review during the legislative process and legislative deliberations involve the prospect of such review (Kommers 1997; Landfried 1994). Consequently, we expect instances of ex post abstract review that are concluded soon after the law was passed to involve statutes for which the legislature anticipated little or no concrete policy effects to occur before review.²⁶

In contrast, legislators should generally anticipate review to follow some – and perhaps a great deal of – policy effects from laws that do not face immediate judicial review. The form of delayed review could be concrete or abstract. Concrete review explicitly requires the law to have been in effect long enough to have produced at least alleged concrete policy effects. And, while abstract review does not require a concrete claim, legislators should expect their laws to have concrete policy effects if the law is ultimately reviewed by abstract review initiated years after the law was passed.

Note, we are not arguing that legislators completely discount the prospect of future judicial review of laws that they consider unlikely to undergo imminent abstract review. Rather, the legislature should generally have expected a higher likelihood of meaningful concrete policy effects before review for legislation considered under concrete review or delayed abstract review than legislation challenged immediately under abstract review. Thus, if the theoretical argument is correct, we should expect to find a court’s propensity to strike a law to be higher under early abstract review – concluded ex ante or ex post but soon after adoption – than under concrete review or delayed abstract review.²⁷

To test this claim, we create a dummy variable, Early Abstract Review, coded as (1) for rulings under abstract review that involve laws passed in 2001, 2002, or 2003. This abstract review could be conducted ex ante or ex post. The variable is coded (0) for all other rulings. The set of courts with early abstract review rulings on statutes in 2003 includes the 10 courts with ex ante rulings defined above and an additional 15 courts.²⁸ As in the previous analysis, we estimate logistic regression models of the effect of review timing on the likelihood of the court striking legislation. We begin by analyzing all relevant rulings in the CompLaw database, which is the same set of rulings from the same 39 courts analyzed for ex ante review in Model 1 in Table 4. And, as in Model 1, we include random effects for each court and estimate robust standard errors. Consistent with expectations, the estimated coefficients in Model 4 in Table 4 indicate a recent law that is reviewed under abstract review is more likely to be ruled unconstitutional than a law reviewed longer after its adoption.

As was this case with Model 1 of Table 3, this preliminary evidence, may suffer from omitted variable bias, as the estimates reflect comparisons of rulings across courts. We therefore report estimates from models that only involve intra-court comparisons. In Model 5 of Table 3, we report results for rulings made by courts that issued rulings based on early abstract review and later abstract or concrete review. In addition to the nine courts from Model 2 of Table 3, we add 15 courts with ex post abstract review and that vary in the timing of review since the law was passed.²⁹ As in Model 2, we include fixed effects to ensure the estimates reflect intra-court comparisons, and we report robust standard errors clustered by case. The results indicate that laws reviewed under early abstract review had 1.67 (95% CI: 1.78–2.37) times greater odds of being struck than laws reviewed longer since their adoptions.

Some courts had missing data about the date of passage of the law under review. As a robustness check, we re-estimate the model excluding the courts of El Salvador, Poland, and Venezuela. This left only a very small set of rulings with missing data (approximately 1%). For that small set, we replaced the missing values with the mean year for rulings on statutes from the relevant court and coded Recent Abstract Review accordingly.³⁰ The estimated coefficients based on this more conservative estimation strategy are presented in Model 6 in Table 4. The results for the smaller set of rulings are very similar to those in Model 5. A recent law reviewed under abstract review is 1.78 times (95% CI: 1.22–2.59) more likely to be ruled unconstitutional than if it were reviewed later. Thus, the results for early abstract review match the empirical expectations of W-G’s (2019) theory of review timing.

Conclusion

The CompLaw coding protocol provides a common template for collecting information about rulings by a broad range of constitutional courts. Based on this protocol, we assembled a dataset of rulings by courts from around the world, in a variety of languages, and including a diverse set of constitutional and institutional features relevant to theoretical and empirical concerns regarding the process of constitutional review and judicial decision-making. As such, we expect it to be valuable to the growing literature on how these features shape judicial behavior (Krehbiel 2016; Staton 2010).

We use CompLaw to estimate the relationship between review timing and the likelihood of a ruling of unconstitutionality. This relationship is relevant to prominent theoretical arguments about how constitutional review influences legislative behavior and the constitutional quality of legislation (Brouard and Hönnige 2017; Vanberg 2001; Ward and Gabel 2019). Given the nature of the observational data, we cannot engage in causal empirical estimation. Rather, we can conduct inference by observing empirical trends consistent with the competing theoretical predictions while ruling out plausible alternative explanations as best as possible.

Toward this end, we find that legislation reviewed ex ante, or soon after implementation, is much more likely to be struck than were laws reviewed later. We analyze strike rates under ex ante and ex post review in a multi-year sample of the French Constitutional Council. We conduct the same analysis in a multi-country, single-year analysis of within-court variation. And, we expand the single-year analysis to include a larger set of courts where we can compare early review (both ex post and ex ante) with later ex post review. The result is a robust finding that is stable across all three estimation strategies. Thus, the finding is not dependent upon the country sample, or how we treat early ex post review. To our knowledge, this systematic empirical relationship between review timing and strike rates has not been identified in previous research.

These descriptive results are potentially valuable for important normative concerns about constitutional review and its influence on legislative behavior. A standard benign account of constitutional review argues that the threat of review causes “auto-limitation” by legislators hoping to avoid an adverse court ruling (Vanberg 1998; Brouard 2009). In such a veto-player model (Brouard and Hönnige 2017), laws would only be struck because of errors by legislators in anticipating the preferences of the court. We would expect rulings by ex ante review, where legislators who wrote the law would know the composition of the court and the likelihood of review, to result in fewer such errors – and thus fewer strikes – than rulings made via ex post review. That we find exactly the opposite casts doubt on this common account and its benign consequences for the quality of lawmaking.

Our data analysis highlights two valuable features of the CompLaw coding protocol. Many laws have multiple articles reviewed in one case by the court, and these articles sometimes were authored at different times and were challenged on different constitutional grounds. In particular, the detailed information about the objects of review (articles in laws) and timing of their adoption allows for a nuanced estimation of the relationship between review timing and the propensity for laws to be struck as unconstitutional. The CompLaw protocol captures these distinctions, which would be difficult to describe or aggregate if one only analyzed decisions at the case level.

Further, the CompLaw coding protocol can be used to expand the time period of rulings in the CompLaw dataset and to include new courts. Any court with physical or electronic filings can be organized according to the CompLaw coding scheme. Therefore, any researcher or team can extend our current dataset, which is primarily limited to one year, and still make structurally uniform comparisons across courts and time. Our study is only one example of how the CompLaw coding protocol, and the data it produces, can be used to answer broad, important, cross-national questions of how judicial systems operate.

Finally, the single year of data assembled here can provide novel evidence about several important questions in comparative judicial politics. For example, one could use our year of data to describe which constitutional provisions (e.g., civil and political rights) are most commonly used in constitutional challenges. The CompLaw dataset includes information about the constitutional provisions associated with each challenge, and these provisions can be categorized according to the common constitutional topics provided in the Comparative Constitutions Project (Elkins and Ginsburg 2022). In addition, these data allow broader tests of Benjamin Bricker (2016) hypotheses about constitution court behavior when reviewing legislation that is passed by the current versus past (and ideologically different) governments. Because the dataset has information about the year the law in question was adopted, one can supplement the CompLaw database with information about the ideological and partisan composition of the government responsible for the law.