6 - Legal Tech and the Litigation Playing Field

Summary

What effect will potent new legal tech tools have on the civil litigation landscape, and what can or should we do about it? Recent trends in plaintiff win rates and damages awards suggest the American civil justice system is growing more slanted toward the “haves” at the expense of the “have-nots.” Some say that AI-fired legal tech tools will reverse this trend and democratize the system. We disagree. Potent new legal tech tools are surely coming. Many are already here. But these tools are, and will likely continue to be, unevenly distributed because of the privileged access to data and technical know-how of emerging consortia of corporations, law firms, and tech companies. As a result, legal tech will, at least over the near- to medium-term, further skew the litigation playing field, shaping not just the resolution of claims but also the evolution of substantive law. As the American civil justice system enters the digital age, the haves will be propelled yet further ahead.

It’s well known that, in US civil litigation, the haves come out ahead.¹ For a slew of reasons – including their ready access to specialists, low start-up costs, and ability to play for rules (not just immediate wins) – well-heeled, repeat-play litigants tend to fare better than their one-shot opponents.

But look closely at data, and it seems that the tilt of the civil justice system may be getting steeper. In 1985, the plaintiff win rate in civil cases litigated to judgment in federal court was a more-than-respectable 70 percent. In recent decades, that figure has hovered at or below 40 percent.² Meanwhile, there’s state-level evidence that when plaintiffs win, they recover less. According to the Bureau of Justice Statistics, the median jury award in state court civil cases was $72,000 in 1992 but only $43,000 in 2005 – a drop (in inflation-adjusted dollars) of 40.3 percent.³

The composition of the country’s civil dockets is also telling – and increasingly skewed. Among civil cases, debt collection claims, which typically feature a repeat-play debt collector against a one-shot debtor, are on the rise. According to Pew Charitable Trusts: “From 1993 to 2013, the number of debt collection suits more than doubled nationwide, from less than 1.7 million to about 4 million, and consumed a growing share of civil dockets, rising from an estimated 1 in 9 civil cases to 1 in 4.”⁴ By contrast, tort cases – the prototypical claim that pits a one-shot individual plaintiff against a repeat-play (corporate or governmental) defendant – are falling fast. Personal injury actions accounted for roughly 20 percent of state civil caseloads in the mid-1980s.⁵ Now they make up a measly 4 percent.⁶

What might explain these trends? Possible culprits are many. Some of the tilt might be explained by shifts in the composition of case flows, toward cases where plaintiffs tend to fare poorly (prisoner rights litigation, for example).⁷ Changes in state and federal judiciaries – perhaps part and parcel of increasingly politicized state and federal judicial selection processes – might also matter. Souring in juror sentiment – traceable to the public’s relentless exposure to tales of “jackpot justice” and frivolous claiming – has played a role.⁸ And judges’ day-to-day conduct has changed. Embracing “managerial judging,” judges oversee trials differently than they did in days of yore, and there are hints that certain types of hands-on intervention – time limits, bifurcation, and restrictions on voir dire – might have a pro-defendant cast.⁹

Beyond this menu of possibilities, more cases than ever are now being formally resolved, not through trial, but through pre-trial adjudications – and this tends to benefit defendants. Following the Supreme Court’s creation of a plausibility standard in Bell Atlantic Corp. v. Twombly and Ashcroft v. Iqbal, motions to dismiss are on the rise.¹⁰ Adjudication via Rule 56 has also trended upward. In 1975, more than twice as many cases were resolved by trial as were resolved by summary judgment.¹¹ Now the ratio of cases resolved in federal courts by summary judgment versus trial is heavily skewed toward the former, perhaps on the order of six-to-one.¹²

Finally, substantive law has become less congenial to plaintiffs. At the federal level, the Private Securities Litigation Reform Act and the Prison Litigation Reform Act, among others, make life harder for plaintiffs.¹³ Alongside Congress, the Supreme Court has issued a raft of defendant-friendly decisions – tightening standing, restricting expert testimony, eliminating aider and abettor liability, expanding the preemptive effect of regulatory activity, curbing punitive damages, shunting claims to arbitration, and limiting class certification.¹⁴ State legislatures, too, have enacted significant tort reform measures, including damage caps, restrictions on contingency fees, alterations to the collateral source rule and joint and several liability, medical malpractice screening panels, and extensions of statutes of repose.¹⁵

Enter legal tech. Surveying this altered civil justice ecosystem, some suggest that legal tech can be a savior and great leveler, with the capacity to “democratize” litigation and put litigation’s haves and have-nots on a more equal footing.¹⁶ It can do this, say its champions, by empowering smaller firms and solo practitioners to do battle with their better-financed foes.¹⁷ Additionally, legal tech might cut the cost of legal services, putting lawyers within reach of a wider swath of people, including those currently priced out of the legal services marketplace.¹⁸ Meanwhile, even when Americans do go it alone, other legal tech advances – including tools that help write or interpret contracts or resolve low-level consumer disputes – might help them to enter the litigation arena with more information, and possibly more leverage, than before.¹⁹

We see things differently. We agree that tech tools are coming. We also agree that some of these tools may pay dividends on both sides of the “v.,” promoting transparency, efficiency, access, and equity. But other, arguably more powerful, tools are also here. And many of the most potent are, and are apt to remain, unevenly distributed. Far from democratizing access to civil justice and leveling the playing field, the innovation ecosystem will, at least over the near- to medium-term, confer yet another powerful advantage on the haves. Powerful repeat players, leveraging their privileged access to data (especially confidential claim-settlement data) and their ability to build the technical know-how necessary to mine and deploy that data, will propel themselves yet further ahead.

The remainder of this chapter unfolds as follows. To ground our analysis, Section 6.1 canvasses legal tech, not in a hazy distant future, but in the here and now. In particular, Section 6.1 details three legal tech innovations: (1) the algorithmic e-discovery tools that fall under the umbrella of technology-assisted review, or TAR; (2) Colossus, a claim assessment program that, for two decades, has helped the nation’s largest auto insurers to expeditiously (though controversially) resolve bodily injury claims; and (3) what we call, for lack of a better term, the Walmart Suite, a collection of increasingly sophisticated tools developed by tech companies and BigLaw firms, working in tandem, to rationalize the liability of large corporations in recurring areas of litigation such as slip-and-falls and employment disputes. All three AI-powered tools are already in use. And all three hold the potential to affect the civil justice system in significant (though often invisible) ways.

Section 6.2 steps back to evaluate these innovations’ broader impact. Here, our assessment of TAR is mixed – and contingent. Fueled by TAR, litigation discovery may, over time, emerge more transparent, more efficient, and more equitable than before. This improved equilibrium is by no means assured, and, as we explain below, bleaker outcomes are also possible. But one can at least glimpse, and argue about, a range of first- and second-best outcomes, where more relevant documents are produced, at lower cost, at faster speed, and with less gamesmanship.

Our assessment of Colossus and the Walmart Suite is more dour. Colossus shows that, using increasingly sophisticated data science tools, repeat players are already using their tech savvy and their stranglehold on confidential claims data to drive case settlements downward. With Colossus, insurers are reportedly able to settle auto accident injury cases for roughly 20 percent less than they did before adopting the software. Meanwhile, the Walmart Suite shows that well-heeled repeat players are not just dipping their toes into the litigation waters; they are already in deep – and, in fact, are already able to settle out unfavorable cases and litigate winners, fueling a dynamic we call the “litigation of losers.” As strong cases are culled from the system via early resolution and only the weak proceed to visible, public adjudication, the litigation of losers threatens to further skew the evolution of damage determinations and substantive law.

A final Section 6.3 asks how judges, scholars, and policy makers ought to respond. We consider, and mostly reject, three possible paths forward: reforms to substantive or procedural law, a broad democratization of data, and “public option” legal tech. These fixes, we suggest, are facially attractive but ultimately infeasible or unachievable. Instead, absent a softening of partisan gridlock or renewed public appetite for reform, it is judges, applying ordinary procedural law, who will be the frontline regulators of a newly digitized litigation ecosystem. And, in classic common law fashion, they’ll need to make it up as they go, with only a few ill-fitting tools available to blunt legal tech’s distributive effects.

6.1 Three Examples: TAR, Colossus, and the Walmart Suite

Despite futurist talk of robo-judges and robo-lawyers, litigation systems have always been, in an abstract sense, just machines for the production of dispute resolution. There are inputs (case facts, law) and outputs (judgments, or settlements forged in their shadow). To that extent, the myriad complex procedures that govern civil litigation – that sprawling menu of commands, practices, and norms – are, at their core, just rules that shape the acquisition, exchange, and cost of information as litigants jockey for advantage.

With this framing in mind, few could deny that legal tech tools will have a significant effect on the civil justice system. But how, exactly, will the civil justice system change, in response to the tools’ adoption?

To gain leverage on that question, we offer three real-world examples of a growing array of legal tech tools that supplement and supplant lawyers’ work: (1) new algorithmic e-discovery tools that, as already noted, pass under the label of technology-assisted review, or TAR; (2) Colossus, the go-to claim-pricing tool used by the nation’s casualty and property insurers; and (3) a cutting-edge set of tools we dub the Walmart Suite that both generates pleadings and papers and predicts case outcomes in certain recurring areas of litigation.

6.1.1 Technology-Assisted Review (TAR)

Used by lawyers on both sides of the “v.,” TAR refers to software designed to streamline and simplify the classification and review of documents, primarily through the use of machine-learning techniques.

Though TAR tools vary in their construction and algorithmic particulars, most operate with some human supervision. Virtually all require lawyers to hand-code, or “label,” a subset of a corpus of documents for relevance or privilege (the “seed set”). Then, those documents are used to train a machine-learning system to categorize additional documents. This process is iterative and may repeat over multiple rounds of labeling and training, until lawyers are satisfied that all documents have been correctly categorized.²⁰

Even the most basic forms of TAR represent a big leap from its predecessors. Prior to TAR’s advent, document discovery required lawyers and their non-lawyer staffs to hunch over bankers’ boxes or filing cabinets, and then, in time, to manually flip through scanned documents on computer screens, reviewing thousands or even millions of documents one-by-one.²¹ Not surprisingly, the cost of this hands-on review was exorbitant; in 2000, it was estimated that discovery accounted for as much as one-third to one-half of total costs where discovery was actively conducted, and perhaps significantly more in large-scale litigations.²²

In the early aughts, both keyword searches and outsourcing came to the fore to address some of the above. But neither proved wholly satisfactory. Keyword searching enabled parties to cut costs by restricting manual review to only those documents containing specific keywords, but search yields were worryingly incomplete.²³ Outsourcing – the move to send discovery to less-expensive contract lawyers in out-of-the-way US cities or abroad – was similarly fraught. Supervision was difficult; parties fretted about conflicts, confidentiality, and rules of multijurisdictional practice; and quality was wanting.²⁴

As against those halfway innovations, TAR’s advantages are profound. Estimates of TAR’s efficacy vary and are hotly contested, but the general view is that implemented well – and this is a key qualifier – TAR systems are as good as manual, eyes-on review in terms of recall (i.e., the proportion of relevant documents in the total pool of documents that are accurately identified as relevant) but systematically better in precision (i.e., the proportion of documents flagged that are in fact relevant). The far bigger difference is efficiency: Compared to its conventional counterpart, TAR achieves all of this at a fraction of the cost.²⁵

Yet, TAR is not without controversy. Much of it stems from the fact that TAR, like any machine learning system, is a socio-technical “assemblage,” not a turnkey engine.²⁶ Attorneys must label and re-label documents as the system works its way toward a reliable model. An important implication is that, much like Colossus (described below), TAR systems are manipulable by humans in their construction and tuning.²⁷ As Diego Zambrano and co-authors detail elsewhere in this volume, this manipulation can run the gamut from outright abuse (e.g., fudging the labels lawyers apply to document labels²⁸ or rigging the selection, adjustment, or validation of models²⁹) to a more benign but still respondent-friendly calibration of the system to favor precision (the proportion of responsive documents among those in a production) over recall (the proportion of responsive documents identified).³⁰ As a result, and as discussed in more detail below, if litigation’s “haves” need not show their work to the other side, they can shade discovery to their advantage and use their better technology and technologists (if the other side can afford them at all) to make sure it sticks.³¹

6.1.2 Colossus

For the nation’s casualty and property insurers, AI has not so much spawned new litigation tools as supercharged those already in use. The best example is Colossus, a proprietary computer software program marketed by Computer Science Corporation (CSC) that “relies on 10,000 integrated rules” to assist insurance companies – the ultimate repeat players – in the evaluation and resolution of bodily injury claims.³² Initially developed in Australia and first used by Allstate in the 1990s, Colossus has grown in popularity, such that it has been utilized by the majority of large property and casualty insurers in the United States, including behemoths Aetna, Allstate, Travelers, Farmers, and USAA.³³

Colossus has radically changed the process of auto accident claims adjustment. By extension, it has profoundly altered how the tens of thousands of third-party bodily injury claims generated annually by American drivers, passengers, and pedestrians are processed and paid by US insurers.

Before Colossus, an experienced auto accident adjuster employed by Allstate or USAA would have assessed a personal injury claim using rough benchmarks, in a process that was more art than science. Namely, the adjuster would add up a victim’s “special damages” (chiefly, the victim’s medical bills) and multiply those by a fixed sum – often, two or three – to generate a default figure, called a “going rate” or “rule of thumb.”³⁴ Then, the adjuster would leaven that default figure with the adjuster’s knowledge and past practice, perhaps informed by a review of recent trial verdict reports, and possibly aided by “roundtabling” among the insurer’s veteran casualty claims professionals.³⁵

With Colossus, however, the same adjuster can now calculate a claim’s worth at a keystroke, after plugging in answers to a series of fill-in-the-blank-style questions. Or, as Colossus itself explains: “Through a series of interactive questions, Colossus guides your adjusters through an objective evaluation of medical treatment options, degree of pain and suffering, and the impact of the injury on the claimant’s lifestyle.”³⁶ To be sure, the data an adjuster must input in order to prime Colossus to generate a damage assessment is voluminous and varied. When inputting a claim, the adjuster accounts for obvious factors such as the date and location of the accident, alongside the claimant’s home address, gender, age, verified lost wages, documented medical expenses, nature of injury, diagnosis, and prognosis. Treatment – including MRI or X-ray images, prescriptions, injections, hospital admissions, surgeries, follow-up visits, and physical therapy – is also granularly assessed.³⁷ Then, against these loss variables, the adjuster must account for various liability metrics. Fault (in all its common law complexity) is reduced to “clear” or “unclear,” while the existence or nonexistence of “aggravating factors” (such as driver inebriation) is also considered, and, in a nod to the tort doctrine of anticipatory avoidable consequences, the adjuster must also input whether the claimant was buckled up.³⁸ Even the individual identity of the handling attorney, treating physician and/or chiropractor, and (if applicable) presiding judge is keyed in.³⁹

Once data entry is complete, Colossus assesses the claim in light of the enormous pool of data in its master database to generate a “severity point total.”⁴⁰ Then, aided by particularized, proprietary information that is specific to each insurer (based on each individual insurer’s “settlement philosophies and claims practice”⁴¹), Colossus converts the point total into a recommended settlement range.⁴² Insurance adjusters use this settlement range in their negotiations with unrepresented claimants or their counsel. Indeed, at some insurers, adjusters are not permitted to offer a sum outside the range, at least without a supervisor’s approval.⁴³ At others, adjusters are evaluated based on their ability to close files within Colossus-specified parameters.⁴⁴ In so doing, according to one insider: “Colossus takes the guess work out of an historically subjective segment of the claims process, providing adjusters with a powerful tool for improving claims valuation, consistency, increasing productivity and containing costs.”⁴⁵

Beyond these, allegations about further operational details abound. The most common is that, when customizing the software (the proprietary process that converts a “severity point total” into a settlement range), certain insurers “tune” Colossus to “consistently spit out lowball offers.”⁴⁶ Some insurers reportedly accomplish this feat by excluding from the database certain figures that, by rights, should be included (e.g., large settlements or verdicts).⁴⁷ Others get there, it is said, simply by turning dials downward to generate across-the-board haircuts of 10–20 percent.⁴⁸

As such, it appears that, in the hands of at least some insurers, Colossus has not only rationalized the resolution of personal injury claims and injected newfound objectivity, predictability, and horizontal equity into the claims resolution process. It has also systematically cut claims – to the benefit of repeat-play insurers and the detriment of their claimant-side counterparts.

6.1.3 The Walmart Suite

A third innovation combines elements of both TAR and Colossus. One exemplar under this umbrella, which we dub “the Walmart Suite,” given its development by Walmart in partnership with the law firm Ogletree Deakins and in concert with the tech company LegalMation, seeks to rationalize recurrent areas of litigation (think, employment disputes and slip-and-falls). It reportedly operates along two dimensions.⁴⁹ First, it reportedly generates pleadings and papers – including answers, discovery requests, and discovery objections – thus cutting litigation costs.⁵⁰ To that extent, the Suite might be thought akin to TAR in its ability to perform low-level legal cognitions and generate straightforward work product that previously required (human) lawyers. Second, and more provocatively, the Suite can evaluate key case characteristics, including the identity of plaintiffs’ counsel, and then offer a prediction about a case’s outcome and the likely expense Walmart will incur if the case is litigated, rather than settled.⁵¹ The Suite thus seems to be a beefed-up Colossus, with a focus on slip-and-falls and employment disputes rather than auto accidents.

The advantages of such tools are seemingly substantial. LegalMation reports that a top law firm has used its tools to handle 5,000 employment disputes – and, in so doing, the firm realized a six- to eight-fold savings in preparing pleadings and discovery requests.⁵² But these economies are only the beginning. Outcome prediction engines, commonly referred to as the “holy grail” of legal tech,⁵³ allow large entities facing recurring types of litigation to quickly capitulate (via settlement) where plaintiffs have the benefit of strong claims and talented counsel – and then battle to final judgment where plaintiffs are saddled with weak claims or less-competent counsel. In so doing, the Walmarts of the world can save today by notching litigation victories while conserving litigation resources. But they can simultaneously position themselves over the long haul, by skewing case outcomes, driving down damages, and pushing precedent at the appellate level. We return to these advantages below.

6.2 The Promise and Peril of Legal Tech

Section 6.1 introduced three types of legal tools that have already entered the civil justice system. These tools – TAR, Colossus, and the Walmart Suite – are hardly the only legal tech applications dotting the American litigation landscape. But they help to define it, and they also permit some informed predictions about legal tech’s effect on the litigation playing field over the near- to medium-term.

Assessing these effects, this Section observes that TAR may help to level the litigation playing field and could even bring greater transparency to discovery disputes – although such a rosy result is by no means assured, and darker outcomes are also possible. Meanwhile, Colossus and the Walmart Suite both seem poised to drive case settlements downward and even fuel a dynamic we call the “litigation of losers,” in part because the data stores that drive them are, at least currently, so unevenly distributed.

6.2.1 TAR Wars: Proportionality and Discovery Abuse

For TAR, our appraisal is mixed – though the dynamics at play are not simple and our predictions less than ironclad. That said, we predict that the next decade will feature increasingly heated “TAR wars” waged on two fronts: proportionality and discovery gaming and abuse. If, on each front, there is sufficient judicial oversight (an admittedly big if), TAR might usher in a new era, where discovery emerges more efficient and transparent than before. But there is also the possibility that, like Colossus and the Walmart Suite, TAR will tilt the playing field toward repeat-play litigants. Here, we address these two fronts – and also these two divergent possible outcomes.

Proportionality: Will TAR’s efficiencies justify more expansive discovery? Or will these efficiencies yield a defendant-side surplus? Discovery has long been the 800-pound gorilla in the civil justice system, accounting for as much as one-third to one-half of all litigation costs in cases where discovery is actively employed.⁵⁴ High discovery costs, and the controversy surrounding those costs, have powered the creation of numerous rules and doctrines that constrain discovery’s scope.⁵⁵ One such rule – and the one we address here – is the “proportionality” requirement, that is, a requirement that a judge greenlight a discovery request only if the request is “proportional” to a case’s particular needs.⁵⁶

Applied to TAR, proportionality is tricky because TAR can yield gains in both efficiency and accuracy. For a requesting party (typically, the plaintiff), more efficient review justifies more expansive review, including document requests that, for instance, extend to a longer time horizon or to a wider net of document custodians. For a producing party (typically the defendant), however, accuracy gains mean that the requesting party will already get more relevant documents and fewer irrelevant ones, even holding constant the number of custodians or the scope of the search.⁵⁷ In short, TAR generates a surplus in both efficiency and accuracy, and the question becomes how best to allocate that surplus.⁵⁸

Given these dynamics, judges might employ the proportionality principle in one of two ways. Judges could recognize that the unit cost of discovery – the cost of each produced document – has declined and compensate by authorizing the requesting party’s more expansive discovery plan. If so, the cost of each produced document will drop, transparency into the underlying incident will (at least arguably) improve, and the overall cost of discovery will remain (roughly) constant. Judges, however, might take a different tack. Notwithstanding TAR’s efficiency advantages, judges might deny requesting parties’ motions to permit more expansive discovery, thus holding proportionality’s benchmarks firm. If so, TAR will cough up the same documents as before, but at a discount.

If trial judges permit producing parties to capture TAR’s cost-savings without compensating by sanctioning more sweeping discovery plans, the effect on civil litigation, from the availability of counsel to settlement patterns, could be profound.⁵⁹ Lower total discovery costs, of course, might be a net social welfare gain. After all, a core premise of proportionality rules is that litigation costs, particularly discovery costs, are disproportionate to the social value of the dispute resolution achieved, and scarce social resources might be better spent on other projects. But shifts in discovery costs can also have distributive consequences. It is a core premise of litigation economics that “all things being equal, the party facing higher costs will settle on terms more favorable to the party facing lower costs.”⁶⁰ If TAR causes discovery costs to bend downward, TAR’s surplus – and, with it, any settlement surplus – will systematically flow toward the net document producers (again, typically, repeat-play defendants).⁶¹ Such an outcome would yield a tectonic shift in the settlement landscape – hard to see in any particular case, but potentially quite large in aggregate. It will be as if Colossus’ dials have been turned down.

The potential for abuse: TAR appears to be more susceptible to abuse than its analog counterpart. How will judges respond? The second TAR battleground will be discovery abuse and gaming. The fight will center on a core question: Can discovery rules generate enough trust among litigants to support TAR’s continued propagation, while, at the same time, mitigating concerns about gaming and the distributive concerns raised by such conduct?

Discovery abuse, of course, is not new. Nor is TAR uniquely vulnerable to discovery abuse.⁶² Indeed, one of the easiest ways to manipulate a TAR system – the deliberate failure to flag (or “label”) a plainly responsive document – is no different from garden-variety discovery manipulation, in which lawyers simply omit obviously responsive and damaging documents or aggressively withhold borderline documents on relevance or privilege grounds. But, as Zambrano and co-authors note in Chapter 5, there is nevertheless good reason to believe that TAR might be especially prone to abuse – and that is a very serious problem in a system already steeped in mistrust.⁶³

TAR’s particular vulnerability to abuse flows from four facts. First, TAR operates at scale. In constructing a seed set, a single labeling decision could, in theory, prevent an entire species of document from coming to light.⁶⁴

Second, and relatedly, TAR can be implemented by small teams – much different than the sprawling associate armies who previously performed eyes-on document reviews in complex cases. This means that, in a TAR world, deliberate discovery abuse requires coordination among a smaller set of actors. If discovery abusers can be likened to a cartel, keeping a small team in line is far easier than ensuring that a sprawling network of co-conspirators stays silent. Moreover, TAR leans on, not just lawyers, but technologists – and, unlike the former, the latter might be less likely to take discovery obligations seriously, as they are not regulated by rules of professional conduct, need not participate in continuing legal education, and arguably have a less socialized sense of duty to the public or the court.

Third, TAR methods may themselves be moving toward more opaque and harder-to-monitor approaches. In its original guise – TAR 1.0 – lawyers manually labeled a “seed set” to train the machine-learning model. With access to that “seed set,” a litigation adversary could, in theory, reconstruct the other side’s work, identifying calls that were borderline or seemed apt to exclude key categories of documents. TAR 2.0, by contrast, starts with a small set of documents and uses machine learning to turn up lists of other candidates, which are then labeled and fed back into the system. TAR 2.0 thus renders seed set construction highly iterative – and, in so doing, makes it harder for an adversary or adjudicator to review or reconstruct. TAR 2.0, to invoke a concept in a growing “algorithmic accountability” literature, may, as a consequence, be less contestable by an adversary who suspects abuse.⁶⁵

Fourth and finally, while TAR is theoretically available on both sides of the “v.,” technical capacity is almost certainly unevenly distributed, since defense firms tend to be larger than plaintiffs’ firms – and are more richly capitalized. With these resource advantages, if defendants are tempted to engage in tech-driven litigation abuse, they (and their stable of technologists) might be able to do so with near impunity.

The tough question becomes: How should judges react, to safeguard the integrity of discovery processes? Here, judges have no shortage of tools, but all have drawbacks. Judges can, for example, compel the disclosure of a seed set, although such disclosures are controversial, since the full seed set necessarily includes both documents that lawyers labeled as relevant as well as those irrelevant to the claim.⁶⁶ Meanwhile, disclosure of TAR inputs arguably violates the work product doctrine, established in the Supreme Court’s 1947 decision in Hickman v. Taylor and since baked into Rule 26(b)(3), which protects against disclosure of “documents and tangible things that are prepared in anticipation of litigation.”⁶⁷ And, a call for wholesale disclosure – ever more “discovery about discovery” – seems poised to erode litigant autonomy and can itself be a bare-knuckled litigation tactic, not a good-faith truth-seeking device.

Worse, if analog discovery procedures are left to party discretion absent evidence of specific deficiencies, but a party’s use of TAR automatically kicks off protracted ex ante negotiations over protocols or onerous back-end “report card” requirements based on various quality-control metrics, there is the ever-present risk that this double standard will cause parties to throw up their hands. To the extent TAR’s benefits are overshadowed by expensive process-oriented disputes, investment in TAR will eventually stall out, depriving the system of its efficiencies.⁶⁸ Yet, the opposite approach is just as, if not more, worrisome. If judges, afraid of the above, do not act to police discovery abuse – and this abuse festers – they risk eroding the integrity of civil discovery and, by extension, litigants’, lawyers’, and the public’s faith in civil litigation.

Time will tell if judges can steer between these possibilities. But if they can, then out of these two gloomy visions comes a glimmer of light. If judges can help mint and then judiciously apply evenhanded protocols in TAR cases, then perhaps the system could end up better off than the analog system that TAR will steadily eclipse. Civil discovery could be one of those areas where, despite AI’s famous “black box” opacity, digitization yields a net increase in transparency and accountability.⁶⁹

6.2.2 Colossus and the Walmart Suite: The Litigation of Losers

When it comes to the slant of the civil justice system, an assessment of the likely effect of Colossus and the Walmart Suite is more dour.

Colossus: Reduction via brute force. The impact of Colossus on the civil justice system seems fairly clear and not particularly contingent. Colossus’ advent has certain undeniable benefits, injecting newfound predictability, consistency, objectivity, and horizontal equity into the claims resolution process. It has also, probably, reduced the monies paid for fraudulent or “built” claims,⁷⁰ as well as the odds that claim values will be influenced by improper factors (racial bias, for example).⁷¹ Finally, it has, possibly, driven down the driving public’s insurance premiums – though there’s little reliable evidence on the point.

But, alongside these weighty advantages, it does seem that Colossus has also reduced claim payments quite significantly, using something like brute force. When Allstate rolled out a new Colossus-aided claims program for Allstate with the help of McKinsey & Co., the consulting firm’s stated goal was to “establish[ ] a new fair market value” for such injuries.⁷² It appears that that aim was achieved. A later McKinsey review of Allstate found: “The Colossus sites have been extremely successful in reducing severities, with reductions in the range of 20 percent for Colossus-evaluated claims.”⁷³ Nor was this dynamic confined, necessarily, to Allstate. Robert Dietz, a fifteen-year veteran of Farmer’s Insurance has explained, for example: “My vast experience in evaluating claims was replaced by values generated by a computer. More often than not, these values were not representative of what I had experienced as fair and reasonable.”⁷⁴

The result is that, aided by Colossus, insurance companies are offering less to claimants for comparable injuries, on a take-it-or-leave-it basis. And, though one-shot personal injury (PI) lawyers could call insurance companies’ bluff and band together to reject these Colossus-generated offers en masse, in the past two decades, they haven’t.

Their failure to do so should not be surprising. Given persistent collective action problems and yawning information asymmetries (described in further detail below), one would not expect disaggregated PI lawyers, practicing alone or in small firms, to mount a coordinated and muscular response, especially since doing so would mean taking a significant number of claims to trial, which poses many well-known and formidable obstacles. For instance, some portion of PI lawyers operate in law firms (called “settlement mills”) and do not, in fact, have the capacity to take claims to trial.⁷⁵ Second, many auto accident claimants need money quickly and do not have the wherewithal to wait out attendant trial delays. And third, all PI lawyers are attuned to the stubborn economics of auto accident litigation: As of 2005, the median jury trial award in an auto case was a paltry $17,000, which would yield only about $5,500 in contingency fees, a sum that is simply too meager to justify frequent trials against well-financed foes.⁷⁶ This last point was not lost on McKinsey, which, in a presentation to Allstate, encouraged: “Win by exploiting the economics of the practice of law.”⁷⁷

The Walmart Suite and the litigation of losers. The Walmart Suite illustrates another dynamic, which we dub the “litigation of losers.” In the classic article, Why the Haves Come Out Ahead, Marc Galanter presciently observed that repeat-players could settle out bad cases “where they expected unfavorable rule outcomes” and litigate only the good ones that are “most likely to produce favorable results.” Over time, he concluded, “we would expect the body of ‘precedent’ cases, that is, cases capable of influencing the outcome of future cases – to be relatively skewed toward those favorable to [repeat players].”⁷⁸

The Walmart Suite shows that Galanter’s half-century-old prediction is coming to pass, fueled by AI-based software he couldn’t have imagined.⁷⁹ And, we anticipate, this isn’t the end of it. In recurring areas of litigation, we are likely to see increasingly sophisticated outcome prediction tools that will draw ever-tighter uncertainty bands around anticipated outcomes. Like the Walmart Suite, these tools are reliant on privileged access to confidential claim settlement data, which only true repeat players will possess.

The effect of this evolution is profound, for, as outcome prediction tools percolate (at least in the hands of repeat defendants/insurers), only duds will be litigated – and this “litigation of losers” will skew – indeed, is almost certainly already skewing – the development of substantive law. The skew will happen because conventional wisdom, at least, holds that cases settle in the shadow of trial – which means that, to the extent trial outcomes tilt toward defendants, we would expect that settlements, too, will display a pro-defendant slant.⁸⁰ Damages will also be affected. To offer but one concrete example, in numerous states, a judge evaluates whether damages are “reasonable” by assessing what past courts have awarded for similar or comparable injuries.⁸¹ To the extent the repository of past damages reflects damages plaintiffs have won while litigating weak or enfeebled claims, that repository will, predictably, bend downward, creating a progressively more favorable damages environment for defendants.

To be sure, there are caveats and counter-arguments. Models of litigation bargaining suggest that a defendant with privileged information will sometimes have incentives to share that information with plaintiffs in order to avoid costly and unnecessary litigation and achieve efficient settlements.⁸² Additionally, while repeat players have better access to litigation and settlement data, even one-shotters don’t operate entirely in the dark.⁸³ But a simple fact remains: Even a slow burn of marginally better information, and marginally greater negotiation leverage, can have large aggregate effects across thousands and even millions of cases.

6.3 What to Do?

As the litigation playing field tilts under legal tech’s weight, there are some possible responses. Below, we start by briefly sketching three possible reforms that are facially plausible but, nevertheless, in our view, somewhat infeasible. Then, we offer a less attractive option – judicial discretion applied to existing procedural rules – as the most likely, though bumpy, path forward.

6.3.1 Plausible but Unlikely Reforms

Rewrite substantive or procedural law. First, we could respond to the skew that legal tech brings by recalibrating substantive law. Some combination of state and federal courts and legislatures could, for example, relax liability standards (for instance, making disparate impact job discrimination easier to prove), loosen restrictions on punitive damages, repeal damage caps, return to a world of joint and several liability, restore aider-and-abettor liability, abolish qualified immunity, and resurrect the collateral source rule.

Whatever the merit of a substantive law renaissance, we are, however, quite bearish on the possibility, as the obstacles blocking such an effort are formidable and, over the near- to medium-term, overwhelming. Substantive laws are sticky and salient, especially in a political system increasingly characterized by polarization and legislative gridlock.⁸⁴ Even in less-polarized subfederal jurisdictions, it would be hard to convince state legislators and (often elected) judges to enact sweeping reforms without strong support from a public that often clings to enduring but often misguided beliefs about “jackpot justice” and frivolous claiming.⁸⁵

Nor are federal courts, or a Supreme Court, newly stocked with Trump-era appointees, likely to help; to the contrary, they are likely to place barriers in front of litigation-friendly legislative efforts.⁸⁶ And procedural rules, though less politically salient, will also be hard to change, particularly at the federal level given the stranglehold of conservative judges and defense-side lawyers on the process of court-supervised rulemaking.⁸⁷

Democratize the data. Second, we could try to recalibrate the playing field by expanding litigants’ access to currently confidential data. As it stands, when it comes to data regarding the civil justice system, judges, lawyers, litigants, and academics operate almost entirely in the dark. We do not know how many civil trials are conducted each year. We don’t know how many cases go to trial in each case category. And, we don’t know – even vaguely – the outcome of the trials that do take place.⁸⁸ Furthermore, even if we could know what happens at trial (which we don’t) or what happens after trial (which we don’t), that still wouldn’t tell us much about the much larger pool of claims that never make it to trial and instead are resolved consensually, often before official filing, by civil settlements.

This is crucial, for without information about those millions of below-the-radar settlements, the ability to “price” a claim – at least to “price” a claim using publicly available data, approaches zero. As Stephen Yeazell has aptly put it:

[I]n the U.S. at the start of the twenty-first century, citizens can get reliable pricing information for almost any lawful transaction. But not for civil settlements. We can quickly find out the going price of a ten-year old car, of a two-bedroom apartment, or a souvenir of the last Superbowl, but one cannot get a current “market” quote for a broken leg, three weeks of lost work, and a lifetime of residual restricted mobility. Nor for any of the other 7 million large or the additional 10 million smaller civil claims filed annually in the United States. We simply do not know what these are worth.⁸⁹

Recognizing this gap, Computer Sciences Corp. (the maker of Colossus) and Walmart and its tech and BigLaw collaborators are working to fill it. But, they have filled it for themselves – and, in fact, they have leveraged what amounts to their near-total monopoly on settlement data to do so. Indeed, some insurers’ apparent ability to “tune” Colossus rests entirely on the fact that plaintiffs cannot reliably check insurance companies’ work – and so insurers can, at least theoretically, “massage” the data with near impunity.

Seeing the status quo in this light, of course, suggests a solution: We could try to democratize the data. Taking this tack, Yeazell has advocated for the creation of electronic databases whereby basic information about settlements – including, for instance, the amount of damages claimed, the place suit was filed, and the ultimate settlement amount – would be compiled and made accessible online.⁹⁰ In the same vein, one of us has suggested that plaintiffs’ attorneys who work on a contingency fee basis and seek damages in cases for personal injury or wrongful death should be subject to significant public disclosure requirements.⁹¹

Yet, as much as democratizing the data sounds promising, numerous impediments remain – some already introduced above. The first is that many “cases” are never actually cases at all. In the personal injury realm, for example, the majority of claims – in fact, approximately half of claims that involve represented claimants – are resolved before a lawsuit is ever filed.⁹² Getting reliable data about these settlements is exceptionally difficult. Next, even when cases are filed, some high proportion of civil cases exit dockets via an uninformative voluntary dismissal under Rule 41 or its state-level equivalents.⁹³ Those filings, of course, may be “public,” but they reveal nothing about the settlement’s monetary terms.⁹⁴ Then, even on those relatively rare occasions when a document describing the parties’ terms of settlement is filed with the court, public access remains limited. Despite a brewing “open court data” movement, court records from the federal level on down sit behind “walls of cash and kludge.”⁹⁵ Breaking through – and getting meaningful access even to what is “public” – is easier said than done.

“Public option” legal tech. A third unlikely possibility is “public option” legal tech. Perhaps, that is, the government could fund the development of legal tech tools and make them widely available.

When it comes to TAR, public option legal tech is not hard to imagine. Indeed, state and federal judiciaries already feature magistrate judges who, on a day-to-day basis, mainly referee discovery disputes. It may only be a small step to create courthouse e-discovery arms, featuring tech-forward magistrate judges who work with staff technologists to perform discovery on behalf of the parties.

Public option outcome-prediction tools that can compete with Colossus or the Walmart Suite are harder to imagine. Judges, cautious Burkeans even compared to the ranks of lawyers from which they are drawn, are unlikely to relax norms of decisional independence or risk any whiff of prejudgment anytime soon. The bigger problem, however, will be structural, not just legal-cultural. The rub is that, like one-shot litigants and academics, courts lack access to outcome data that litigation’s repeat players possess. Short of a sea change in the treatment of both pre- and post-suit secret settlements, courts, no less than litigation’s have-nots, will lack the information needed to power potent legal tech tools.⁹⁶

6.3.2 Slouching Toward Equity: Judicial Procedural Management with an Eye to Technological Realities

Given the above obstacles, the more likely (though perhaps least attractive) outcome is that judges, applying existing procedural rules, will be the ones to manage legal tech’s incorporation into the civil justice system. And, as is often the case, judges will be asked to manage this tectonic transition with few rules and limited guidance, making it up mostly as they go.

The discussion of TAR’s contingent future, set forth above in Part 6.2.2 offers a vivid depiction of how courts, as legal tech’s frontline regulators, might do so adeptly; they might consider on-the-ground realities when addressing and applying existing procedural doctrines. But the TAR example also captures a wider truth about the challenges judges will face. As already noted, legal tech tools that cut litigation costs and hone information derive their value from their exclusivity – the fact that they are possessed by only one side. It follows that the procedural means available to judges to blunt legal tech’s distributive impacts will also reduce the tools’ value and, at the same time, dull incentives for litigants to adopt them, or tech companies to develop them, in the first instance. As a result, disparate judges applying proportionality and work product rules in individual cases will, inevitably, in the aggregate, create innovation policy. And, for better or worse, they will make this policy without the synoptic view that is typically thought essential to making wise, wide-angle judgments.

Judicial management of legal tech’s incorporation into the civil justice system will require a deft hand and a thorough understanding of changing on-the-ground realities. To offer just one example: As noted above, discovery cost concerns have fueled the creation of a number of doctrines that constrict discovery and, in so doing, tend to make life harder for plaintiffs. These include not just Rule 26’s “proportionality” requirement (described above), but also a slew of other tweaks and outright inventions, noted previously, from tightened pleading standards to court-created Lone Pine orders that compel plaintiffs to offer extensive proof of their claims, sometimes soon after filing.

Undergirding all these restrictive doctrines is a bedrock belief: that discovery is burdensome and too easily abused, so much so that it ought to be rationed and rationalized. Yet, as explained above, TAR has the potential to significantly reduce the burden of discovery (particularly, as noted above, if more expansive discovery, which might offset certain efficiency gains, is not forthcoming). As such, TAR, at least arguably, will steadily erode the very foundation on which Twombly, Iqbal, and Lone Pine orders rest – and this newly unsettled foundation might, therefore, demand the reexamination of those doctrines. As judges manage the incorporation of potent new legal tech tools into the civil justice system, we can only hope that they will exhibit the wisdom to reconsider, where relevant, this wider landscape.

6.4 Conclusion

This chapter has argued that the civil justice system sits at a contingent moment, as new digital technologies are ushered into it. While legal tech will bring many benefits and may even help to level the playing field in certain respects, some of the more potent and immediately available tools will likely tilt the playing field, skewing it ever further toward powerful players. One can imagine numerous fixes, but the reality is that, in typical common law fashion, the system’s future fairness will depend heavily on the action of judges, who, using an array of procedural rules built for an analog era will, for better or worse, make it up as they go. We’re not confident about the results of that process. But the future fairness of a fast-digitizing civil justice system might just hinge on it.