A Common-Space Scaling of the American Judiciary and Legal Profession

Abstract

We extend the scaling methodology previously used in Bonica (2014) to jointly scale the American federal judiciary and legal profession in a common space with other political actors. The end result is the first dataset of consistently measured ideological scores across all tiers of the federal judiciary and the legal profession, including 840 federal judges and 380,307 attorneys. To illustrate these measures, we present two examples involving the U.S. Supreme Court. These data open up significant areas of scholarly inquiry.

1 Introduction

This paper extends donor-based scaling methods to jointly scale the legal profession and federal judiciary in a common space with other political actors. We do so by linking together two sources of data: (1) a newly collected dataset that includes nearly all of the nation’s attorneys, gathered from online legal directory Martindale-Hubbell; and (2) the Database on Ideology, Money in Politics, and Elections (DIME) (Bonica 2013). Combining these data sources allows us to identify the campaign contributions—and corresponding ideological common-space scores—for thousands of U.S. lawyers and judges.

These data are appealing for two reasons. First, they represent the first consistently measured ideology estimates for judges across the federal judiciary that do not rely on the identities of appointing actors. Indeed, while the U.S. Supreme Court has seen substantial innovation in scaling methods (e.g., Martin and Quinn 2002; Lauderdale and Clark 2014; Bailey 2013), measuring ideology has proven more difficult at the lower levels of the federal judiciary. This owes to the fact that district and appeals court judges seldom vote on cases together, and, when they do, it is often in three-judge panels too small to be scaled. Estimates of ideology of federal judges have therefore relied on the identity of the relevant nominating political actors (e.g., Boyd 2011; Epstein et al. 2007; Giles, Hettinger, and Peppers 2001). ¹ Our measures, however, do not rely on the identities of the appointing actors; neither do they rely on additional bridging assumptions beyond those used in the calculation of CFscores (Bonica 2014). Second, our measurement strategy scales lawyers alongside federal judges, which opens possibilities for future research regarding the legal profession’s role in gatekeeping and advocacy.

We provide two illustrations of these data. First, we show that the ideologies of lawyers arguing cases before the Supreme Court closely track the directionality of case outcomes. Second, and relatedly, we show that lawyers’ ideologies map onto the ideologies of justices who vote in their favor, thus recovering the rank ordering of Martin and Quinn (2002). This application further suggests that the ideology of prevailing attorneys could be used a proxy for judicial ideology at lower-court levels, where using votes-based scaling is more difficult. We conclude by noting that these data represent a useful tool both for American and judicial politics, thus providing a rich complement to existing measures such as Martin and Quinn (2002), Bailey (2013), Boyd (2011), and Epstein et al. (2007).

2 Data

We construct our measures of attorney ideology by linking data from three sources: (1) DIME, (2) the Federal Judicial Center (FJC) biographical directory, and (3) the Martindale-Hubbell legal directory. ² A detailed discussion of the DIME is provided in Bonica (2014). The database reports DIME scores (also known as “common-space CFscores”) for all individuals and organizations making campaign contributions to state and federal candidates from 1979 to 2014. The scores place donors in a common space with other candidates and organizations spanning local, state, and federal politics. This allows for direct comparisons between attorneys, candidates, and judges. Here, we rely strictly on scores derived from personal contributions to measure the ideology of federal judges.

Our first task is to link individual lawyers and judges to their contribution records in DIME. We utilized a probabilistic record-linkage algorithm that conditions on name, occupation and employer, address, geography, and other features to automate the process of linking records in Martindale-Hubbell to DIME. (See Bonica and Sen (2015) for details.)

3 Measure Validation

The DIME scores are extensively validated in Bonica (2014) for both donors and candidates. We note some of the more important validation results. First, the scores for individual donors and recipients are robust to controlling for candidate characteristics related to theories of strategic giving, such as incumbency status. Second, DIME scores for political actors strongly correlate with vote-based measures of ideology such as DW-NOMINATE scores (Poole and Rosenthal 2007). Lastly, estimated scores for candidates who have campaigned for judicial and nonjudicial seats are robust to changes in office type. In what follows, we extend the validation results for lawyers and judges.

3.1 Comparison with candidate-based measures

We identified 2,771 individuals in our data who had run for elected office and raised funds from enough donors to be assigned an independent DIME score as a candidate. Of this group, 159 also have DW-NOMINATE scores. Figure 1 plots the relationship between contributor and candidate DIME scores. ³ The overall correlation is $\unicode[STIX]{x1D70C}=0.95$ . The within-party correlations are $\unicode[STIX]{x1D70C}=0.86$ for Democrats and $\unicode[STIX]{x1D70C}=0.87$ for Republicans. The corresponding correlations with DW-NOMINATE scores are $\unicode[STIX]{x1D70C}=0.90$ overall, $\unicode[STIX]{x1D70C}=0.62$ for Democrats, and $\unicode[STIX]{x1D70C}=0.56$ for Republicans.

Figure 1. Recipient and contributor ideal points for lawyers who ran for elected office.

3.2 Comparison with existing measures

To compare the DIME scores with existing measures judicial preferences, we calculated scores for judges appointed to federal courts between 1980 and 2014 using the methodology described in Giles, Hettinger, and Peppers (2001)—the same methodology that underlies the widely used Judicial Common-Space (JCS) Scores (Epstein et al. 2007). The scores are calculated as a function of the common-space DW-NOMINATE scores of those involved in the nomination process. If one (or both) home-state senators are of the president’s party, the nominee is assigned the NOMINATE score of the home-state senator (or the average). If neither Senator is from the President’s party, the nominee is assigned the president’s NOMINATE score. ⁴

The correlation between the DIME scores and JCS scores is $\unicode[STIX]{x1D70C}=0.70$ for federal judges. The relationship is stronger when JCS scores are constructed from the NOMINATE scores of senators ( $\unicode[STIX]{x1D70C}=0.77$ ) as opposed to the appointing president ( $\unicode[STIX]{x1D70C}=0.63$ ). The association between the DIME and JCS scores significantly weaker than the association seen in figure 1; but this is to be expected. The JCS scores are indirect measures based on those involved in the appointment process (presidents and senators). The cases where the DIME scores and JCS scores disagree help to illustrate how the measures differ. Consider the three judges with the largest residuals between measures: the Sixth Circuit’s Helene White ( $\text{DIME}=-0.92$ ; $\text{JCS}=0.72$ ), the Second Circuit’s Barrington Parker ( $\text{DIME}=-0.60$ ; $\text{JCS}=0.72$ ), and the Fourth Circuit’s William Traxler ( $\text{DIME}=1.17$ ; $\text{JCS}=-0.28$ ). In each case, the nominee had first been appointed to the district court by a president of one party before being elevated to an appeals court by a president of the other party. Moreover, unlike appointee-based measures, our measures are not prone to errors resulting from bipartisan negotiations, including packaged deals. A recent example was struck between the Obama Administration and Saxby Chambliss and Johnny Isakson, Republican senators from Georgia, to move forward with packaged group of seven nominees. Ultimately, one of the Republican picks, Michael Boggs, was rejected by Senate Democrats. Our measures correctly identify Boggs as conservative.

3.3 Sensitivity to giving to judicial candidates

Another potential concern is that career incentives might lead lawyers and judges to behave differently than other donors. For example, lawyers might face pressure to contribute to the campaigns of sitting judges. While we cannot speak directly to how professional concerns influence donation behavior of lawyers, we find no evidence that giving to judicial candidates biases the measures. When we re-estimate the DIME scores for lawyers with contributions to judicial candidates excluded, the resulting scores correlate with the original scores at $\unicode[STIX]{x1D70C}=0.99$ . Moreover, re-estimating the scores with all contributions to state elections excluded (i.e., federal contributions only) produces scores for lawyers that correlate with the original scores at $\unicode[STIX]{x1D70C}=0.97$ .

3.4 Strategy for dealing with missingness

A limitation of the measuring judicial ideology from campaign contributions is that not all judges have made donations and thus are missing scores. While only about 33% of judges appointed during 1980s have contributor DIME scores, the coverage rate rises to 71% of judges appointed since 2001. For many applications, missingness can prove problematic. We use the Amelia II package (Honaker, King, and Blackwell 2011) to impute missing values. We include in the multiple imputation model variables capturing the (1) observed DIME and JCS scores, (2) court type, (3) law school, (4) birth year, (5) gender, (6) race/ethnicity, (7) employment history, (8) American Bar Association ratings, and (9) clerkships. We also include variables reflecting the political environment at time of nomination. Rather than pool all judges into a single imputation model, we group judges by the party of appointing president and fit the model separately for each party. (See the supplemental appendix for a more detailed discussion.)

Figure 2. Pairwise comparisons of observed and imputed DIME scores and JCS scores for federal judges (1980–2014). Note: Upper panels report overall and within-party correlation coefficients.

To evaluate the accuracy of the multiple imputation, we overimpute the DIME scores, which gives us predicted values from the multiple imputation model for both the missing and nonmissing data. Figure 2 displays pairwise comparisons of the (1) contributor DIME scores, (2) JCS scores, and (3) the imputed values. The points for judges are labeled according to the partisanship of their appointing president. The upper-right panels report the Pearson correlation coefficients between measures overall and within party. A direct comparison between the observed DIME scores and the imputed DIME scores can be seen in the bottom-left panel. The correlation with the observed DIME scores is $\unicode[STIX]{x1D70C}=0.85$ for the imputed scores compared with $\unicode[STIX]{x1D70C}=0.70$ for the JCS scores. The JCS scores do a poor job of capturing within-party variation in the DIME scores. The imputed scores perform significantly better in this respect.

4 Illustrations of the Data

We provide two illustrations of these data by examining (1) how Supreme Court lawyers ideologically align with case directionality and (2) how lawyers’ ideologies map onto the ideology of the justices who vote in their favor.

4.1 Do Lawyers’ Ideologies Align with Case Directionality?

Compelling arguments have been made that lawyers are primarily “guns for hire” whose personal ideological leanings are orthogonal to those of their clients or cases; an equally strong argument is that lawyers and law firms have strong ideological leanings (Bonica, Chilton, and Sen 2016). We assess these claims using our measures. We first obtain data on the directionality of Supreme Court decisions from the U.S. Supreme Court Database for the 846 cases decided by the Roberts Court between 2005 and 2013 (Spaeth et al. 2015). Cases are assigned a binary coding depending on whether a ruling for the petitioner is in a liberal or conservative direction (1 if conservative, 0 if otherwise). This serves as a proxy (albeit an imperfect one) for the true directionality of the case. For each case, we identified the attorneys who argued on behalf of the petitioner and respondent from the Supreme Court docket. We then regress case outcome on attorney ideology using a simple logit specification.

Table 1. Predicting liberal–conservative case codings from attorney ideal points: Logit.

Note: Outcome Variable: Directionality of case outcome associated with petitioner is conservative.

The results are presented in Table 1. Model 1 includes the ideal point of the attorney arguing for the petitioner party. It reveals a robust relationship between attorney ideology and case outcomes: the more conservative the petitioner attorney, the more likely a decision for the petitioner will be in a conservative direction. Model 2 adds the ideal point of the lawyer representing the respondent party. The coefficient for the respondent attorney is of similar magnitude but, as expected, negatively signed. In Models 3 and 4, the ideological variable is calculated as the distance between the petitioner and respondent attorneys. Positive values indicate that the petitioner attorney is to the right of the respondent attorney. Again, we find a robust relationship between attorney ideology and case directionality. In the supplemental appendix, we show that (1) the results hold for cases that were decided unanimously and hence would be uninformative in the context of MCMC-IRT estimation and (2) how the patterns amplify across certain issue areas (for example, First Amendment).

4.2 Inferring Justice Ideology from Attorney Ideal Points

Second, we explore the relationship between the attorney ideology and judicial voting patterns. As shown in Table 1, attorney ideology, as revealed by donation patterns, provides an informative signal about the directionality of case outcomes. Finding a similar relationship between attorney ideology and judicial voting patterns would provide evidence of a broader congruence between attorney ideology, case disposition, and judge ideology. That is, such findings would suggest that attorney ideology could be useful for estimating the ideology of judges—including lower-court judges for whom votes-based measures are less widespread.

We begin by constructing scores for Supreme Court justices as a simple average of the ideal points of petitioner attorneys with whom they sided. The decision to focus more narrowly on petitioner attorneys—rather than both petitioner and respondent attorneys—reflects that respondent attorneys are disproportionately drawn from a relatively small set of governmental actors (e.g., the U.S. Solicitor General) that are assigned to cases by default. On the other hand, petitioner attorneys have greater discretion in bringing cases.

For our comparison set, we recover vote-based ideal points for Supreme Court justices with a one-dimensional MCMC-IRT model using the MCMCpack R package (Martin, Quinn, and Park 2011). We acquired vote data for Supreme Court decisions from the U.S. Supreme Court Database (Spaeth et al. 2015). We restrict the sample of cases to those decided by the Roberts Court between 2005 and 2013. The estimates reported below are based on a 100 000 iteration sample, with a discarded 20 000 iteration burn-in period.

Figure 3. Comparison of MCMC-IRT estimates and ideal points inferred from attorney ideology.

Attorney ideology as revealed by contribution records provides a highly informative signal about the ideological content of case outcomes and, in turn, the ideology of justices. Figure 3 plots attorney-based estimates for justices against the corresponding ideal points recovered from the IRT model. The attorney-based estimates successfully reproduce both the rank ordering and relative placement of justices recovered from IRT model. The two measures are almost perfectly correlated ( $\unicode[STIX]{x1D70C}=.99$ ).

While our approach succeeds in recovering the relative positions of justices, we caution that it does not place the justices on the same scale as the common-space DIME scores and thus cannot be directly compared without some adjustment. Attorney ideal points are a noisy signal of the location of the reverse and not reverse outcomes for individual cases. By averaging judicial voting patterns over a sufficiently large number of cases, it is possible to recover reliable estimates of where justices locate relative to each other. But measurement error introduces attenuation bias. Note also that Justice Thomas sides with attorneys with an average ideal point that is slightly left of center. This is due to the overall left-skew in the Supreme Court Bar. Indeed, it is quite common for both the petitioner and respondent attorneys on a case to be left of center.

5 Conclusions and Future Research

Scaling lower-court ideology from case decisions has proven challenging, owing to the infrequency with which lower-court judges sit together. In addition, approaches that rely on the ideology of nominating actors are prone to mismeasurement, leaving room for improvement. We take a different approach in this paper. The result is the largest dataset to date of consistently measured ideal points for judges and other kinds of legal actors. The estimation strategy here relies directly on revealed preferences, avoiding the problems associated with inferring ideology from nominating actors. In addition, the data include ideal point estimates for attorneys, which broaden the range of possible research inquiries. Taken together, these measures enable many inquiries into the political influence of the bar and of the integration of lawyers and judges in the broader fabric of American politics.

Supplementary material

For supplementary material accompanying this paper, please visit https://doi.org/10.1017/pan.2016.10.