16 - The Civil Justice Data Gap

Summary

How can improving the collection, sharing, and analysis of data make the civil justice system more accountable to other government institutions, participants in the justice system, and the public at large? We tackle this question from three angles. First we show how accountability can create opportunities for civil justice reform. Drawing on work in other social spheres on large datasets, we identify three lines of research that court data could inform: the extent that structural racism and other biases shape processes and outcomes; the impact of lack of representation on litigants’ experiences and outcomes; and the antecedents and consequences of court involvement for poor people. A second focus is the obstacles that prevent us from increasing our store of knowledge about civil justice problems. These obstacles include: the lack of good data, legal barriers to obtaining data, and real and perceived institutional risks to sharing data. Finally, we report on our efforts to design and build a civil justice data commons (CJDC) addressing these barriers in order to provide fast and frictionless access for policy research as well as operational insights for courts and civil justice institutions to improve equity and service delivery.

Every year, millions of people living on the margins cycle through a civil justice system that exacts an enormous toll on their housing security and financial well-being. Eviction rates reached crisis levels before the coronavirus pandemic and are expected to continue to climb as emergency moratoria have been lifted.¹ Debt-collection cases have not received the same national attention as evictions, but may represent a quarter to a third of state civil cases according to recent statistics.² The pandemic has contributed to a rise in consumer debt, creating a looming threat of increased collection cases.³ Civil courts have become eviction and collection mills, whose apparent function is issuing judgments to help landlords and debt collectors grind down tenants and creditors.⁴ According to available evidence, the civil justice system is failing in its promise to protect the rights of poor people against powerful private interests and entrenching them further in poverty.

But “available evidence” is sparse. We know little about how courts dispense justice or how court entanglements affect peoples’ lives. Our knowledge of how civil justice institutions function, despite the system’s societal centrality, is piecemeal and relies on extrapolation from site-specific or out-of-date studies. In a system where all parties are meant to be treated equally, we don’t know whether the race, ethnic origin, or gender of litigants affects case outcomes. We can’t determine whether there are biases in the system or where they are concentrated. We don’t know what events precipitate civil justice involvement, nor the consequences that flow from it. We don’t know how unrepresented individuals fare against businesses with representation. We don’t even have an exact number of how many unrepresented litigants there are or in which types of cases they are concentrated⁵ or whether the scarce resources that are available are being allocated effectively and equitably.

As a recent report by the American Academy of Arts and Sciences emphasizes, many institutions and individuals have an interest in answering questions like these about the functioning of the justice system.⁶ State, county, and municipal courts are a central feature in the lives of low-income people and, increasingly, the middle class. Courts have an obligation to the people who pass through their doors to administer justice equitably. Litigants are entitled to know whether courts are living up to this ideal. Courts and legal services providers have particular interests in gaining operational insights about how well they serve the people before them and how effectively they allocate their resources. And the public, which funds the civil justice system, has a stake in knowing how well the civil justice system functions and whether its outcomes are just.⁷ Erosion of public trust puts the legitimacy of this system at risk.⁸

In other public spheres, large datasets are producing new and actionable knowledge about the performance of public and private institutions. Health care and health insurance datasets have been linked to show that more years of childhood Medicaid eligibility correlate with fewer adult hospitalizations for Black Americans.⁹ In Cuyahoga County, Ohio, the Center on Urban Poverty and Community Development has used data from more than thirty-five different administrative programs to create two integrated data systems for research.¹⁰ Linkages in these comprehensive data systems have shown increased shelter use by evicted families and identified downstream consequences such as childhood lead poisoning.¹¹

As these projects show, big data are particularly powerful in the study of social institutions by delivering variety, volume, and velocity. Real-time data are captured for nearly all of our everyday whereabouts, purchases, searches, and interactions with private individuals and public institutions. Everyday interactions with other people and social institutions are chronicled by the terabyte through active or passive engagement with digital technologies. Data are created and available in near-real time.¹² These features produce broader knowledge than was possible with limited size and site-specific case studies, permitting both wide generalizations and highly granular views into the functioning of institutions and their effects on people’s lives.

What explains the lack of data in civil justice? For one, not every court collects all case level information in electronic form. Some simply don’t have the resources; others don’t see value in doing so. Even where data exist, rules often restrict access. Courts are unwilling to share data, or uncomfortable doing so, for fear that vulnerable litigants will be harmed or that the data will be used to cast a critical eye on courts themselves.¹³ Courts lack resources and incentives to improve their data, resulting in civil justice data that are incomplete, inaccurate, dirty, and disorganized. Despite the uniformity of needs among court administrators and a small handful of vendors providing case management systems, there are no widely used standards for data collection and harmonization. The absence of common standards makes it impossible to compare jurisdictions or even to study individual jurisdictions across different case types and subject matters.

The lack of high-quality, accessible data is a major deterrent to producing knowledge about civil justice. To obtain data, researchers must engage in a hunt-and-peck exercise that may or may not yield useful results. As a first step, they need to locate datasets that may be useful to them. Then, they determine whether there are legal limitations to obtaining the data. If the data are not publicly available but a court has discretion to share them, researchers must engage in protracted negotiations to obtain access. Once researchers have access, they must spend weeks cleaning the data. The process of finding and obtaining data and making them usable deters researchers from studying the civil justice system and impedes the development of evidence-based policies in the civil justice arena. Although data science is producing new insights about many social institutions, few data scientists are working in the civil justice field.

This chapter considers how improving data collection, sharing data, and ease of data analysis can make civil justice institutions more accountable to government institutions, their constituents, and the public at large, and create opportunities for civil justice reform.

Section 16.1 starts by identifying key questions that analysts are well positioned to address. These include problems of structural racism and bias against people of color, the advantages of being represented, and the antecedents and consequences of entanglement in the civil justice system. Section 16.2 considers the obstacles to increasing knowledge about civil justice problems. These obstacles include the lack of good (or any) data, legal barriers to obtaining data, and real and perceived institutional risks to sharing data.

Section 16.3 then describes a civil justice data commons as one promising avenue that the authors are prototyping to address current access-to-justice challenges¹⁴ and free up the empirical opportunities offered by civil justice data.¹⁵ This Civil Justice Data Commons (CJDC) functions as a trusted intermediary among data sharers and data users. The CJDC’s technical infrastructure is automated to satisfy conditions for the supply of data and the demands of users. To facilitate access for researchers – speeding up the evidence-building process – the CJDC hosts documented, harmonized data, cleans and links files as needed, and provides tools and analytic software for research. The data commons allows for faster access to data for academic and policy research, as well as operational insights for courts and civil justice institutions to improve equity and service.

Several different tech trends examined in this volume – from ODR to online courts to e-filing to the growing menu of legal tech tools – will transform civil justice in the years to come. Each will increase both the amount of and focus on data. A key question is whether the civil justice system – and the judges, lawyers, litigants, and justice partners who inhabit it – can harness data to promote the just, equitable, and efficient administration of justice. A civil justice data commons can help answer that question.

16.1 Important Questions about the Fairness of the Civil Justice System

16.1.1 Structural Racism and Bias

In the criminal justice context, abundant high-quality policing and court data have allowed researchers to document the scope and depth of structural racism and other biases. This research underscores what smaller-scale observational and quantitative studies had previously shown: Black Americans are systematically treated worse than white Americans at every juncture of the criminal justice system. Policing data have shown that Black people are more likely to be subjected to roadside stops and are more likely to be the victims of lethal force.¹⁶ Court datasets have allowed researchers to show that Black drivers are likely to pay higher traffic fines than white drivers¹⁷ and that judicial mechanics and the peremptory challenge system adversely affect Black defendants.¹⁸ If you are Black, you are more likely to receive a longer sentence than a white person in a plea deal.¹⁹ These studies confirm the systemic pervasiveness of racism in criminal justice. Data access permits studies with sufficient granularity to pinpoint specific areas where race might have the strongest magnifying effect, which may suggest priorities for formulating and testing reforms.

Large-scale studies that investigate the impact of race, ethnic origin, and gender in the civil justice system are sparse but show promise.²⁰ One major foray comes from the Princeton Eviction Lab, which published the first study documenting the demographics of evicted renters on a national scale. Using a dataset containing 4.1 million records from public, private, and commercial sources, the study showed Black renters experienced the highest rate of eviction filings and judgments, and Black and Latin female renters faced higher eviction rates than their male counterparts.²¹ Data-driven studies like those by the Eviction Lab can demonstrate the magnitude of inequity in the civil justice system. Their study is a significant step in understanding which demographic groups in the United States are suffering the brunt of the eviction crisis and raises new questions about how the civil legal system plays into this crisis. Do differing eviction rates among racial and gender groups mirror structural biases in society at large? Or does the civil justice system magnify these problems? Accessible civil justice data are needed not only to understand the system’s role in replicating or worsening structural biases in eviction cases, but also its role in consumer debt and other areas that disproportionately affect poor people, people of color, and women. Population-level datasets contain large numbers of observations and variables to help pinpoint court contacts and processes that have the greatest impact.

16.1.2 When Do Lawyers Make a Difference (and Why?)

Civil justice data can also shed light on whether lawyers make a difference in outcomes and, if so, in what types of cases and why.²² To date, studies comparing represented and non-represented parties have been site-specific and relied on observational data or randomized controlled trials. Consequently, their findings are not easily generalized to other subject matters and other settings. Large-scale court datasets will allow comparisons across courts, subject matter areas, types of representation, and jurisdictions, producing more nuanced knowledge with broader impact.

The few studies investigating whether being represented by a lawyer affects the outcome of a case tend to show that representation makes a difference, but how much difference it makes depends in large part on the type of service provided, case type, and procedural complexity of the case.²³ Some research suggests that, in select settings, being represented by a non-lawyer advocate confers similar advantages to being represented by a lawyer. These studies raise important questions about how courts should deal with cases where one or both parties are not represented by a lawyer and how legal services providers should prioritize the provision of their resources.

Data-driven research confirming and extending these findings offers insights into how to “level the playing field” between represented and unrepresented parties. Some jurisdictions have changed their rules to allow judges a more active role in explaining the law or eliciting facts to a pro se party.²⁴ Courts might also focus on simplifying procedures or prohibiting lawyers in certain matters or fora altogether.²⁵

Research showing whether and when being represented affects case outcomes can also be important for legal services providers allocating resources. In legal aid settings, eligible applicants may receive representation depending on the availability of a lawyer specializing in their type of case. Research on the differential effects of representation might persuade legal services providers, which turn away almost half of eligible applicants,²⁶ to shift their resource priorities. Some providers report that they have already begun targeting their resources to clients with income levels just below eligibility cut-offs after data-based research showed representation did not affect outcomes at lower income levels.²⁷ Large-scale data studies about the impact of lawyers would also provide evidence for or against various approaches being promoted to increase access to the civil justice system. With the eviction crisis worsening, the movement to fund lawyers to represent tenants at risk of eviction is gaining ground.²⁸

Such studies might also be helpful in answering the question of which types of services make a difference. In the last ten years, limited scope representation has been promoted as an important means of increasing access to justice by allowing fewer lawyers to represent more people, an approach that many legal services providers are pursuing on the assumption that they can help more clients.²⁹ Despite the growth of brief advice services,³⁰ there is little evidence that these services have a positive effect on outcomes. One study found that while brief advice assisted clients in surmounting procedural barriers, it did not have significant effects on substantive outcomes.³¹ However, this study, which focused on the evictions caseload of a legal aid provider in San Mateo County during the summer of 2009, may not be generalizable to other settings.

Improving short-term outcomes – helping a client win a specific case, stay in their home, or avoid wage garnishment – is one measure of effective representation. Others include helping clients understand the law, supporting them through the process, and empowering them to advocate for themselves.³² An understanding of whether and when representation makes a difference is important knowledge to have in allocating scarce resources among eligible applicants who live in poverty and seek legal assistance.

16.1.3 Civil Justice Involvement and Poverty

The questions we have identified so far focus on whether civil justice institutions treat people fairly, allocate resources equitably, and achieve just results. Other research avenues focus on the people involved, not the court encounters themselves.

Getting a Fuller Picture from Big Data. More accurate and consistent data provide opportunities for deeper understanding about individuals’ interactions with the civil justice system: who enters the system (and why), how they navigate the system, and what impacts this interaction has on them in the long term. Specifically, linked population-level datasets that consider the antecedents and consequences of court involvement can widen the lens of research, leading to inferences and interventions that can scale across sites. By linking information from before and after court involvement, it may be possible to understand mechanisms that help avoid entanglement in the justice system or to mitigate its negative effects. This research shifts the emphasis away from court processes and legal resources to understanding and addressing the conditions and events that lead up to court involvement to begin with. In doing so, it may help people avoid being evicted by landlords or sued by debt collectors altogether.³³

This type of analysis is already being done at other intersections of poverty and policy. For example, researchers in Sonoma County, California, integrated four years of county and state records from health, mental health, substance abuse, housing, criminal justice, and human services systems.³⁴ This cross-domain dataset showed that 1 percent of the population accounted for one-quarter of jail time and behavioral health services and half of overnight homeless housing.³⁵ Another study linked administrative records from the Child Care and Development Fund, a federal subsidy program, with American Community Survey data to reveal that women who worked full-time but had less social support were more likely to receive the child care subsidy.³⁶ The study considered characteristics of the mother, such as education level, attainment of labor skills, and community ties, and found that relative to other low-income women, the women who received the child care subsidy tended to have higher educational attainment (high school diploma or higher).³⁷

Understanding the long-term consequences of a government intervention has also been made possible with linked datasets. One recent study created more than 150 million parent-child links to analyze the downstream intergenerational effects of the 1970 Clean Air Act Amendment and found that regulation-induced air quality improvements during pregnancy correlated with parental investment and higher college attendance in later generations.³⁸ This analysis was achieved through linking decennial census data, administrative records, and survey data and utilizing Protected Identification Keys (PIKs) in order to anonymize personal identifiers.³⁹ In another study, researchers linked national administrative data on the labor market and earnings with records on public housing to find that children who resided in public housing that was demolished later earned more than children who remained in non-demolished public housing, suggesting that job accessibility improved for children forced to relocate.⁴⁰

The Interaction between Poverty and Entanglement with the Justice System. Just as these studies used linked data to understand the role of government policies in the lives of people living in poverty, research linking court datasets to datasets that are indicators of health, financial circumstances, and other measures of well-being can yield knowledge about the effect of court entanglement over the course of a person’s life. Research is already under way. The Institute for Research on Poverty at the University of Wisconsin–Madison found women with an incarcerated family member have a higher risk of poor health, and Black women are, in turn, disproportionately likely to have an incarcerated family member.⁴¹ Researchers looking at Connecticut evictions found persistent, largely untreated, mental health problems and continuing housing instability.⁴² Other research, using mixed methods and blending administrative data with court data, shows correlations between eviction and criminal justice involvement,⁴³ how neighborhood disadvantage can exacerbate racial health disparities,⁴⁴ and how eviction in early childhood is associated with neighborhood poverty, food insecurity, and obesity in later childhood and adolescence.⁴⁵

Studies that investigate correlations between involvement with the civil justice system and other public institutions are beginning to emerge. For example, a study linking bankruptcy filings to administrative tax and foreclosure data found that filers who received Chapter 13 bankruptcy protections experienced increased annual earnings and decreased foreclosure rates.⁴⁶ Several studies have also appeared that focus on the short- and long-term consequences of being evicted for poor families. Using New York City housing court records, researchers found that evictions correlated with an increased risk of homelessness, long-term residential instability, and emergency room use but did not significantly worsen longer-term financial circumstances.⁴⁷ A longitudinal comparison of Cook County court records to credit bureau and payday loans data illustrated how the effects of successful evictions were small relative to the financial strain experienced by both evicted and non-evicted tenants in the time leading up to eviction actions.⁴⁸ Taken together, these studies raise the possibility that the prevailing view (which assumes that preventing eviction will have long-term benefits on alleviating poverty) may be myopic and that the locus of policy should shift from court processes to poverty-reduction programs.

More and greatly accelerated research using large, linked datasets is needed to determine the consequences of court involvement for the life cycle of poverty across sites and population subgroups. Providing lawyers and legal assistance to poor litigants assumes that the best locus of intervention to alleviate the effects of poverty must be the civil justice system – and that more lawyers and more legal assistance is the best approach. More studies may tell us, however, that assistance and opportunities upstream may be the most effective way of addressing housing and financial insecurity to prevent eviction and consumer debt actions from being initiated in the first place. Studies may also show that the impact of representation varies across race, ethnic enclaves, age groups, and population density.

Easily accessible civil justice datasets are needed to undertake such studies. In Section 16.2, we describe current challenges to obtaining such data and explain how the development of a civil justice data commons may begin to overcome these barriers.

16.2 Barriers to Accessing Data

An overarching challenge with civil justice data is the disaggregation of authority over civil dockets among states, counties, and municipal jurisdictions. In some states, managerial authority over the court system, and therefore control over data collection and sharing practices, is centralized in the supreme court or chief court administrative office. This makes it possible for the court to dictate that the same data be collected under the same standards throughout the state’s civil courts. As former Chief Justice Bridget McCormack notes in her chapter in this volume, Michigan has adopted a uniform system that is spreading to all its counties. Connecticut has also implemented a similar approach.⁴⁹ Other states have decentralized systems where each court has the power to dictate what data are collected. In some states, courts in different jurisdictions throughout the state have the option of maintaining their official records in either paper or electronic form.⁵⁰ This results in data that are siloed in multiple institutions and starkly unstandardized. Accessibility is governed by a huge and bewildering hodgepodge of laws and policies, rendering their availability for research limited. These barriers must be addressed if data-driven approaches are to produce knowledge about civil justice institutions.⁵¹ This section describes five of the largest barriers to access.

16.2.1 Data Disarray

Although courts are moving to electronic case management systems, many are still managing case files the old-fashioned way: paper. Others are using case management systems, but case pleadings are handwritten hard copies scanned into the digital system and are not machine-readable.

The absence of machine-readable pleadings, which contain valuable information about the case and the litigants, is a major technical impediment to granular research on the civil justice system. To overcome this barrier, some researchers have resorted to hand-coding pleadings, an expensive and time-consuming process.⁵²

Over time, the adoption of e-filing that is integrated with case management systems may alleviate this problem (though many mandatory e-filing systems exempt unrepresented parties).⁵³ For now, it is necessary to hand-transcribe the information or develop optical character-recognition tools that can translate pleadings that are marked with stamps and often contain handwriting into machine-readable form.

Administrative case level data scraped from online court case search websites – which pull from case management system data originally entered by hand by a court clerk – can be messy and contain incorrect or inconsistent values, making them difficult to interpret. Addressing these issues often requires a manual comparison with the corresponding docket by a specialist who can align concepts properly. That is tedious when the dockets are available online and onerous when it is necessary to contact the clerk of the court to understand which dispositions numeric codes refer to. Improving data capture is critical to improving analyses and the function of courts.

16.2.2 Barriers to a Common Taxonomy

Another data challenge is the absence of a common taxonomy among courts. The lack of shared substantive and procedural classification systems results from the diverse sites of authority among state, county, and lower-level courts. Different definitions across courts, even for concepts as fundamental as indigency, hamper coding and comparisons. Court procedures with different terms for various pleadings, requirements, and judgments also complicate matters. A court may lump evictions and contract cases together under “contracts,” or it may separate evictions and contracts but not distinguish among consumer debt cases (with some number of these classified separately under “small claims”). A rare few might use more granular categories that distinguish medical debt, consumer debt, and other debts, as an example. Coding these types of categories is necessary to study correlations between court involvement and life-course events. For instance, being able to identify medical debt is necessary to understand how much healthcare debt collection has grown since the pandemic.

In an attempt to tame some of this chaos, the National Center for State Courts (NCSC) developed the National Open Data System (NODS), intended to guide data collection in courts throughout the United States.⁵⁴ NCSC’s exhaustive effort yielded standards for basic elements such as case type, representation by counsel, case filing, disposition, and case closure date. NODS is an ambitious effort to transform the collection and classification of court data. Its adoption will be dictated by the cost of implementing or reconfiguring court case management systems and the extent to which courts believe that its implementation is consistent with the court’s priorities.

While NODs is a forward-looking strategy, there may be other strategies to rationalize court records from different jurisdictions that have already been collected. One approach might be to use natural language processing to identify subcategories in court documents to generate classifications. These generated classifications could then be matched to NODS categories. For instance, an algorithm might be trained to recognize medical debt from other types of debt based on the language in pleadings. Whether such an approach could capture subcategories of case types or definitional and procedural variation – likely more challenging – remains to be seen.

16.2.3 The Special Challenge of Assigning Demographic Characteristics

A particular challenge involves obtaining demographic information for civil court participants – a prerequisite for understanding equity concerns within the civil justice system. Under NODS, civil courts are supposed to collect race, ethnicity, and other demographic data about litigants. Most courts do not currently capture these data, but even if they attempted to, they would face a fundamental difficulty: Many, if not most, defendants in certain types of civil actions do not appear in court. One recent study of debt-collection cases in state courts found that, for courts where there was available data, 70 percent of debt cases ended in default judgments.⁵⁵ With such high rates of default, it is impossible to collect information about the race, ethnic origin, or gender of the tenant or debtor using only court data.

One promising strategy to append demographic characteristics to court data is through data linkage to census or commercial data. Linking at the per-person level requires the court records to have complete, accurate name data or other unique identifiers to make a match. Previous linkages to census data have had match rates between 40 and 90 percent, depending on the completeness of the names and addresses available, with the higher match rates possible when additional fields such as age or date of birth are available. The sparse identifiers present on civil court records have led other researchers to incorporate additional data sources. For example, Case Western Reserve University’s Center on Urban Poverty and Community Development matched Cleveland Housing Court data to Ohio Public Assistance records to obtain race information.⁵⁶ While this linkage allowed researchers to add otherwise absent demographic characteristics to eviction files, they were able to do so only for defendants who applied for and obtained public assistance.

16.2.4 Legal Barriers

While individual case files are available either online or in the clerk’s office in every jurisdiction, legal rules may limit access to bulk electronic case records. Although all fifty states and the District of Columbia have enacted public records laws, more than a third exempt the judiciary from coverage.⁵⁷ Even when public record laws apply to the judiciary, it is unclear whether they entitle people to obtain electronic case files in bulk form.

Approximately thirty-two states have rules governing the availability in bulk form of electronic public case records. Some states like Arizona make case file records generally available for bulk download.⁵⁸ Even in states that do not bar access to case data, county courts may have rules that limit their accessibility.⁵⁹ Some states prohibit the dissemination of bulk court records in electronic form, except where explicitly provided by a court rule or order.⁶⁰ At least one state disseminates bulk court data only to commercial purchasers. Other states, in contrast, bar the bulk download of court files for commercial gain.⁶¹ Finally, there are states like Hawaii, which will grant some requests for bulk data as long as the entity making the request is willing to pay for fees associated with the cost of providing the data.⁶² Many states also charge a fee or have terms of service that prohibit the scraping of court websites or CAPTCHA intended to prevent it.⁶³ On top of this confusion, states may have different privacy policies for court records with varying levels of restrictions.⁶⁴

16.2.5 Institutional Barriers

Institutional barriers are arguably the most important to address to facilitate data sharing. In preliminary planning interviews for the CJDC, we found a wide spectrum of views among judges and court administrators about the value of collecting and sharing data. Some judges did not view the data as valuable for either research or operational purposes; others strongly believed that research on aspects of the civil justice system was important for determining how well the courts served their communities.

Courts also fear scope creep resulting in data misuse, where a project approved to study one topic veers to another and publishes a study that the data may not support. Safeguards and reviews of approved uses must keep pace with growing volumes of data. Courts may worry that errors in their data will come to light. The errors may result from hurried or flawed collection methods, or through poor data management, statistical analysis, logic, or communication. When errors arise, it is important to examine their origins and impacts.⁶⁵

Even when court rules provide for public access to data, court personnel face administrative burdens to sharing them. They receive numerous requests for data and are pressed to develop internal custom-made approval processes on the fly. Most courts lack a data governance committee. Providing requested data is a time-consuming and mostly manual process fraught with risks of data misuse, errors in court data, and embarrassing findings.

16.3 What a Civil Justice Data Commons Offers

A data commons provides institutions and policy makers a one-stop shop for both high-level and detailed information about civil justice problems, by giving institutions simple, secure methods to share their data with researchers, policy makers, and other civil justice stakeholders and researchers fast, frictionless, and facilitated data access. By centralizing data sharing and research functions, the commons can serve as a meeting place for analysts, and a space where participants can share subject matter expertise, technical tools, and research methods.

16.3.1 What Is the CJDC?

The civil justice data commons (CJDC) is built according to principles that underlie data commons in the health sciences, created to accelerate research on medicine and disease using large databases.⁶⁶ Like genomic and other health data commons, the CJDC is controlled by a governance regime negotiated between data sharers, data users, and the directors of the commons.⁶⁷ The terms of service reflect the data security needs and project scoping requirements of data contributors with access demands of researchers.⁶⁸ The governance model is “hardwired” into the commons’ permissions infrastructure, ensuring that approved uses and users gain access efficiently.⁶⁹ Transparency is key: The commons makes clear what is the path to access, what requests have been granted, and what has been learned. Housed in an academic research center, the commons is able to act as an independent “trusted intermediary” for all parties.

The CJDC draws on the “Five Safes” framework to provide data access, balancing the terms and conditions of data providers and the needs of researchers.⁷⁰ Under this framework, the CJDC approves only “safe projects” intended to increase knowledge of the civil justice system. Researchers cannot use data for commercial purposes or for administrative purposes that have direct implications for affected individuals in the data. CJDC users are “safe people,” who are affiliated with universities, courts, government agencies or non-profit institutions, such as policy institutes and community organizations. The CJDC is a “safe setting” for the data, which can be accessed only on the CJDC’s platform. The data are “safe data”: Data sharers certify that the data they provide are legally and ethically sourced, and records in the commons are documented so the chain of their provenance is clear. Last, the commons follows the principle of “safe outputs.” CJDC users must submit their analytic outputs to the CJDC team, who will ensure that the data were used for the study proposed and that statistical disclosure limitation methods have been applied to prevent re-identification of data subjects.

In essence, the data commons provides a regulated marketplace for data owners and data users. For data owners, the commons facilitates data provisioning and quality control and oversees researcher use of the data. For researchers, the commons provides an interface where researchers can conduct statistical analyses on requested data. The CJDC can enable data discovery across multiple sites and jurisdictions, allowing potential users to browse metadata documentation and propose projects. Project requests are reviewed and approved in accordance with terms of use dictated by the data contributors that govern such questions as who is allowed to use the data, the scope of the research, and the purpose for which the research is undertaken. When projects are approved, CJDC provisions data into a secure workspace with statistical applications that researchers need. By making this information more readily available, knowledge about the prevalence, incidence, and consequences of civil justice problems can be produced, reducing reliance on anecdotal, outdated, or qualitative data alone. CJDC provides a simple, responsive interface that permits researchers to determine what data are available and what research has already been done with various datasets. Users can receive updates about new or refreshed data sources, recommendations on new data topics, and findings from the community.

In its mature form, the CJDC will have a secure infrastructure to automate data provisioning and an interface for data contributors to facilitate data sharing. In addition, it will have separate interfaces for authorized researchers and civil justice institutions so that they have ready access to data for academic and policy research and operational insights. The CJDC will host only machine-readable and documented data, which will be encrypted and held according to industry standards for securely stored data. The CJDC will offer technical assistance or services to help with cleaning, harmonizing, data documentation, and linkages. Funding for the operations and expansion of the Commons might come from federal and state government and foundation grants. The prototype data commons integrates extracts of data from data contributors. A future version of the data commons may have a federated structure as well, enabling the controlled access of data from a data controller’s servers. This may alleviate concerns about storage and control.⁷¹

16.3.2 Addressing the Barriers

Institutional Interests and Concerns. As noted above, the greatest barriers to making civil justice data more easily accessible are institutional. Courts are hesitant to share data out of concerns about data security, misuse, and errors, and findings that are embarrassing or ignite unwanted political controversy. The CJDC addresses these concerns by intermediating data sharing and access processes. Court administrators state their terms and conditions for data use, and CJDC screens requests and users, approving only requests that meet these conditions for data access. CJDC also takes over the monitoring role for approved data uses, ensuring scope and data security compliance. By using the well-established Five Safes framework for data access control, the CJDC ensures balance between the concerns of data providers and the needs of researchers. Institutions may also be sensitive to the optics of using these data. The CJDC provides greater privacy protection for data subjects than the open data portals already in place.⁷² Finally, the CJDC helps courts avoid surprises by permitting output review. It is critical to have independent research and transparency but equally important for courts to be able to confirm that studies have interpreted local laws or policies soundly. Giving the courts this “heads up” can allow them to frame the findings as opportunities to address issues rather than being surprised by negative headlines.

The CJDC also provides valuable services for data sharers through the production of knowledge to increase their own transparency. We expect that CJDC documentation and data standardization will enable application developers to create tools that work across sites. CJDC data can inform institutions on where things are working well and where they are not, providing evidence to support resource reallocation.

Controlling the Data Disarray. Working with the commons is not an all-or-nothing proposal. Sites can contribute what they have and increase their data sharing over time. For example, a court may share their machine-readable docket information, knowing that key data elements are not yet digitized. Commons users may help the court address this gap, such as through character recognition software or clerical keying. This creates a win-win situation where both the court and researchers gain more usable data. Other courts may share data they know is discontinuous, perhaps because of a change in records management vendors, updates to laws or policies, or transitions from paper to electronic records. The CJDC staff will document the data, and CJDC users will enhance detail about the datasets and likely create code that bridges the discontinuity. By fostering a community of users, this knowledge will be shared back to the court and subsequent researchers.

A necessary step in building out the CJDC is the harmonization of court data across jurisdictions, with their disparate data collection policies and case management systems. The CJDC seeks alignment with and adoption of NODS. Important first steps are determining what court data are machine-readable across sites and documenting challenges to NODS standardization within and across datasets. CJDC also standardizes data elements across contributed files, for example, creating a common vocabulary for eviction and debt cases and geocoding location data to facilitate area-level analyses. This standardization can include both manual matching and algorithmic entity resolution, allowing an accurate picture of the actual cases despite typos or name changes in the data.

Much of this initial work will continue to be labor- and time-intensive until we understand paths to automate the process of aligning new sites and keeping information up-to-date from courts we have been working with. A key feature of the CJDC is user-friendly metadata (data about the data) noting the sources, time periods available, present or absent fields, and incompleteness rates. As this metadata conforms to NODS, both research and court users will benefit. This metadata goes hand-in-hand with tools to visualize the available data. This data cleaning and sorting is a significant benefit to data contributors, who likely do not have the resources or expertise to perform this work on their own, and provides a provides a clean foundation for research.

Appending Demographic Characteristics. Because court files cannot capture the demographic characteristics of litigants who do not appear in court, linkages to external data sets are necessary to produce this knowledge. The CJDC is pursuing linkages among civil justice, census, and administrative datasets to enable analyses of civil justice eviction and debt-collection cases to develop demographic information about court litigants. These data will be critically important for understanding systemic biases in the aftermath of the pandemic.

16.4 Conclusion

Data gaps are not filled overnight, nor by individuals or single institutions. The CJDC is as much a movement as a project. To succeed, it requires participation by courts, practitioners, researchers, policy makers, philanthropists, private industry, and the public. By making civil justice data findable, accessible, interoperable, reusable, and linked to other data, the CJDC will increase knowledge about biases that distort civil justice processes and outcomes. It can also reveal promising strategies to increase access to civil courts and deepen understanding of how civil justice involvement impacts people living in poverty. If successful, the CJDC can create the basis for evidence-based approaches to civil justice reform.