The Learning Model of Use-of-Force Reviews

Abstract

Since the 1970s, police departments have subjected the use of force by their officers to increasingly stringent oversight, but those efforts have struggled against the difficulty of codifying the complex and idiosyncratic judgments that police work requires. In response, some departments have developed new models of oversight that use routine incident reviews partly as a tool for learning in order to document the continually surprising circumstances that officers encounter in the field, to scrutinize existing responses, and to articulate alternatives. This article analyzes the logic of this emerging model through a case study of use-of-force reviews in the Portland Police Bureau. I argue that this emerging model relies on an approach to practical inquiry that has not been adequately understood in criminal justice scholarship and practice—one that uses the routine review of organizational experience to pursue normative progress as well as technical understanding and that makes it possible to adapt complex policing practices to the local environment in which they operate.

Beginning in the 1970s, critical incident reviews brought about a revolution in the use of force by American police. Until then, most agencies had done little to guide or monitor the way their officers used force (Chapman and Crockett 1963; President’s Crime Commission on Law Enforcement and the Administration of Justice 1967, 189; Walker and Archbold 2014, 70), leaving only the minimal standards established by the law to play that role (Harmon 2008; Klockars 1996). By the 1980s, however, most large agencies finally subjected the use of deadly force to more or less clear administrative restrictions and oversight, and guidelines and monitoring for nonlethal force soon followed (Fyfe 1979; Walker and Archbold 2014, 68–70). Today, nearly all police departments have written use-of-force policies, and most review use-of-force incidents to monitor whether officers comply with those policies (Bureau of Justice Statistics 2007). I will refer to this strategy as the compliance model of incident review.

The compliance model remains important; some agencies still have not yet established meaningful controls on the use of force, and even in those that have there may be further room to refine existing policies (Harmon 2008; Miller, Toliver, and Police Executive Research Forum 2014; Police Executive Research Forum 2016; Garrett and Stoughton 2017). Nevertheless, efforts to subject the use of force to stricter and more vigorously enforced rules have run into significant challenges. The situations that officers encounter are mind-bogglingly diverse and unpredictable, and simple rules stated in advance often provide incomplete or even counterproductive guidance. Beyond a few categorical prohibitions—no warning shots; do not (usually) shoot at moving vehicles; do not Tase anyone engaged only in passive resistance—most of the judgments that police officers need to make seem to resist codification (Bittner 1990, 174–75; compare Doyle 2014, 7). Policies that try to capture this complexity become unwieldy as guides to action, yet, despite their detail, they may remain obtuse in the face of the unpredictable circumstances that continue to arise. Subjected to an increasingly long list of rules about how they should handle complex situations, officers worry that unsympathetic managers (perhaps under pressure from the public) can find something wrong with almost anything they do (Moskos 2015; Levine 2019, 866–67).

As the limits of the compliance model have become apparent, alternatives have begun to take shape. The most sophisticated police agencies do more than establish rules and monitor how well officers comply with them. They use incident reviews partly as a tool for learning in order to document the complex circumstances that police encounter in the field, to scrutinize existing responses to them, and to articulate alternative approaches that might be taken, not just by the responding officers but also by others who shape the possibilities available to them. These reviews are informed by rules but not rigidly constrained by them. Their lessons take many forms: revision to the rules themselves, an expanded sense of the priorities that should guide the police response to critical incidents, a broader repertoire of options for handling those incidents, new equipment and organizational support for frontline officers, and many others. This model abandons the enterprise of prescribing how police should handle complex incidents in advance but remains committed to the idea that it is possible to learn something from experience. I will refer to this emerging approach as the learning model of use-of-force review.

Policing scholars have begun to recognize the emergence of the learning model, but its nature and logic remain poorly understood. In The New World of Police Accountability, Samuel Walker and Carol Archbold (2014, 20, 67–74; compare Walker 2010; Davis 1975) provide the best brief description of the new approaches to use-of-force review, but they do so within an “administrative rulemaking” framework that cannot fully make sense of them. The point of the learning model is precisely to transcend the limits of rule-based accountability. A newer and more apposite framework applies ideas about “organizational accidents,” “root cause analysis,” and “high-reliability organizations,” which were developed in fields like aviation and medicine, to policing, particularly to the police’s use of force (Pickering and Klinger 2016; Hollway, Lee, and Smoot 2017; Schwartz 2018; Sherman 2018) and to wrongful arrests and convictions (Doyle 2012; Shane 2013). Hospitals learned years ago that it makes no sense to blame medication errors entirely on the nurse or anesthesiologist who administered the wrong dose; they should also investigate the overly complicated systems that contributed to such mistakes and figure out how to improve them (for example, by pre-dosing medication or redesigning confusing machines) (Schwartz 2018, 544). Similarly, avoidable use of force results not just from poor decisions made by the officer who actually used force but also from the organizational systems that influenced her actions (Doyle 2012; Hollway, Lee, and Smoot 2017; Sherman 2018). To reduce the use of force and other troubling outcomes in policing, agencies must engage in relentless self-scrutiny to spot and rectify these systemic flaws, just as aviation and medicine have done for years (Doyle 2012, 2014; Hollway, Lee, and Smoot 2017).

This “systems perspective” captures important features of the emerging practices of use-of-force review, but it does not yet encompass some of their most important components—in particular, their pursuit of normative inquiry as well as technical analysis, their attention to local context as well as generalized best practices, and their commitment to routine review of everyday incidents as well as occasional scrutiny of high-profile tragedies. These components of the learning model are not entirely absent from the fields where the systems perspective originated, but the distinctive nature of American policing makes them especially salient in that context.

First, police work is more normatively complex than fields like medicine and aviation, so normative questions loom especially large when police try to learn from experience. The use of force is always regrettable, but it is not always wrongful in the same way that a plane crash or operating on the wrong patient is;¹ regardless, the system features that contribute to the use of force sometimes serve valuable purposes, so efforts to redesign them must confront difficult normative questions that are less insistent in aviation or medicine. “It is hard to argue against checklists before surgery, or against the redesign of anesthesiology machines so that the knobs of every machine turn in the same direction,” Joanna Schwartz (2018, 560) recently observed, “but it is more challenging to balance the costs and benefits associated with limiting traffic stops.” Systemic reform in policing must often engage with “fundamental questions about the scope of police authority and the relative importance of competing law enforcement values” (560; compare Sherman 2018, 431). Learning from error in policing requires normative reflection, not just technical analysis. The systems perspective on the use of force in policing has devoted too little attention to this aspect of learning.

Second, American policing remains resolutely decentralized, intertwined with idiosyncratic and constantly changing local contexts and guided by the diverse priorities of local communities. An important aspect of the learning model lies in its efforts to continually scrutinize practice in particular contexts to adapt it to the environments where it operates. This need to adapt organizational practices to unique and dynamic local environments certainly arises in other fields as well,² but it is sufficiently pronounced in policing to justify more detailed attention than the systems perspective usually gives it. The general literature on systems thinking tends to emphasize the development of universally applicable best practices, such as the development of checklists to combat pervasive cognitive biases or the redesign of commonly used equipment (for example, Reason 2008, ch. 6; Gawande 2009; Syed 2015, 19, 28–29, 54), and recent work that applies these ideas to policing has often embraced this emphasis wholeheartedly. In particular, Lawrence Sherman (2018; Schwartz 2018, 560) recently called on police researchers to try to identify best practices in organizational design that any police department can adopt to reduce use of force or, at least, that certain general types of police departments can adopt in well-defined types of environments (Sherman 2018, 445).³ When these general lessons can be found, they are obviously enormously valuable, but critics of evidence-based policing have pointed out that many problems in policing seem to lack standardized solutions (Sparrow 2011; Thacher 2019). A key aspect of organizational progress in policing involves adaptation to idiosyncratic and constantly changing local contexts. This task is distinct from the task of identifying more generalized best practices, and its logic, preconditions, and value need separate treatment.

Finally, and relatedly, many emerging practices of use-of-force review are more routinized than those associated with the systems perspective in policing. The National Institute of Justice has pioneered one important approach with its recent “sentinel events” initiative, in which each of three jurisdictions nominated a single event and convened a new team to review it (Doyle 2014, 11; Browning et al. 2015).⁴ In this form, sentinel event analyses are separate from routine incident review, conducted occasionally rather than continually and by special groups convened for this purpose rather than by existing internal units (Browning et al. 2015, 3–7, 16). The learning model as I have described it represents a different method of learning from experience. By integrating learning into existing protocols for routine incident review, it scrutinizes organizational practices continually rather than occasionally. In that respect, it may be particularly well suited to the tasks of adapting systems and practices to dynamic environments and iteratively testing and refining experimental reforms.⁵ This routinized approach to learning also differs from other approaches to organizational learning that have recently been advanced in policing, including those that call for the informal sharing of craft knowledge among officers (Alpert 2015, 3–4; President’s Task Force on 21st Century Policing 2015, 19) and those that advocate for non-incident-specific research programs on organizational factors that influence police use of force (Sherman 2018). All of these approaches have great potential to help police reduce the use of force, but the use of routine incident review as a tool for learning raises distinctive issues that require separate treatment.

These tasks of normative inquiry, local adaptation, and continual review are important components of the learning model, but the systems perspective in policing has mostly neglected them. This gap is not easy to overcome, for it reflects limitations of leading ideas about how rational inquiry contributes to progress in policing. Most policing scholarship assumes that police goals are stable and well understood, that there exists a fixed menu of options for achieving those goals, and that each option on the menu has an inherent capacity to generate outcomes regardless (for the most part) of context (Thacher 2001, 2008, 2019). From that perspective, progress comes from better assessments of the capacity of given options to accomplish known goals (along with better compliance monitoring to ensure that practitioners actually choose those options). The task of normative progress seems impossible, and the task of adapting practice more intelligently to truly novel circumstances is hard to understand. To make sense of these important aspects of the learning model, it will be necessary to rethink the forms that practical inquiry can take.

This article aims to pursue this goal by interpreting the learning model as an instance of a broader approach to governance that is sometimes labelled “democratic experimentalism” (for example, Dorf and Sabel 1998; Sabel and Simon 2011). Experimentalism draws on many of the same ideas as the systems perspective, stressing the value and logic of tools like root cause analysis and near-miss reporting to make frontline practice more effective and robust in volatile environments (for example, Sabel 2006; Simon 2012), but it supplements them by emphasizing the tasks of normative inquiry and local adaptation as well (for example, Cohen and Sabel 1997; Sabel and Simon 2012), and it places special emphasis on practices that question organizational routines continually rather than occasionally (Sabel 2006). It unites these tasks and analyzes their logic by grounding them in a distinctive understanding of practical reasoning, which clarifies how tools like continual error detection and root cause analysis can contribute to normative progress and local adaptation. In this respect, experimentalism builds on the systems perspective by extending its reach, encompassing features of the learning model that have special significance for American policing.

Experimentalism has been applied to policing before, particularly by providing an illuminating new interpretation of problem-oriented policing (for example, Dorf and Sabel 1998 328–32; Sabel and Simon 2016), but it has not yet been used to understand how police agencies regulate the use of force. In many respects, the emerging practices of use-of-force review realize experimentalist ideals more fully than problem-oriented policing does, for they institutionalize review and continual improvement of frontline practice more systematically. Problem-oriented policing often devises successful solutions to novel problems by taking a more flexible approach than police usually do, but, to date, most of these projects have remained informal, ad hoc, and overly reliant on traditional policing tactics; they have rarely left a lasting imprint on ongoing organizational practices (Braga and Weisburd 2019). In this respect, they seem to embody the “organized informality” that Charles Sabel (2006, 114–19) distinguishes from true experimentalist organization. By contrast, the emerging practices of use-of-force review illustrate how the successes and failures that police agencies identify through continual monitoring can be used to reshape existing organizational routines, improving the agency’s ability to tackle future problems rather than simply resolving an immediate problem in isolation.

The next section begins by reconstructing the forms of practical reasoning that experimentalist practices rely on and illustrating how they contribute to routine reviews of frontline work. This theoretical material provides a framework for identifying and understanding key elements of the new approaches to use-of-force review. The two sections that follow use this framework to describe and interpret the evolution of use-of-force reviews in Portland, Oregon: first, by providing a detailed case study of Portland’s intensive review of one particularly controversial incident and, second, by analyzing the core features of the learning model that this example illustrates, indicating how these features arise in more ordinary cases as well. The fourth section then relates Portland’s experience to similar efforts in other cities, and the final section concludes.

THE LOGIC OF LEARNING BY MONITORING

The limits of the compliance model lie deep in the model of practical reasoning that it relies on. Rule-based oversight assumes that we can identify the criteria we will use to evaluate an action in advance—that we can formulate our plans before we act and then use them to evaluate the action. To do that, we must rely on an ex ante understanding of our goals (including how to handle the tensions among them) and on generalized knowledge about the consequences that alternative courses of action will have in predictable circumstances. Sometimes this model works well enough, but it fails in many complex domains of human practice (for example, Thacher 2019). John Dewey’s account of why it fails is the main inspiration for the theory of democratic experimentalism. Dewey drew attention to three features of human rationality that the familiar model neglects. First, Dewey stressed the interdependence of means and ends and the provisional nature of any plan, goal, or rule. He observed that we regularly reconsider our plans and goals once we appreciate what it will take to accomplish them, and we constantly reframe our understanding of the problem we are trying to solve as we get feedback from our initial efforts (Dewey 1922, 172–277; compare Schön 1983). Second, he explored the social nature of human practices, emphasizing that most of the options available for handling current predicaments are a social inheritance—a menu of possibilities worked out by fellow travelers in the long-standing effort to solve recurring problems. While the familiar picture of practical reasoning hardly rules out this social dimension of human action, it does not draw attention to it either; it highlights the task of selecting the best option from the available alternatives while ignoring the process of generating alternatives to select from in the first place (Dewey 1922, 58–74). Third, Dewey stressed the complexity of human practices, which are not free-floating interventions with an intrinsic capacity for success or failure in any circumstances but, rather, flexible adaptations to the continually evolving and highly localized environment they aim to reshape (14–23). When we reflect on how well our current practice works and on possibilities for reform, we should not focus myopically on the practice in isolation but also consider how it interacts with the surrounding context: “We must work on the environment not merely on the hearts of men” (20). There can be no master list of disembodied practices that “work” or “don’t work”; the goal, instead, is to seek mutual adjustment between particular practices and the local context that surrounds them (compare Cartwright and Hardie 2012).

In short, our goals and plans are provisional, the available alternatives are constantly expanding, and action and context are intricately intertwined in highly particular local circumstances. These considerations led Dewey to conclude that practical reasoning does not mainly involve ex ante attempts to establish definitively which option among the known alternatives will best accomplish existing goals in predictable circumstances. The alternatives are too complex and indeterminate and the goals too uncertain for the conclusions of that kind of analysis to provide very useful guidance. Practical reasoning also encompasses efforts to identify significant problems that had not been recognized previously and to articulate new alternatives for addressing them. It constantly considers how existing tools, support structures, and evaluative standards need to be adjusted to idiosyncratic environments that do not stand still. It intervenes provisionally, monitors the results of intervention, and makes continual revisions in response to feedback from dynamic environments. These features of practical reasoning underlie the improvisation and judgment that practitioners rely on as they struggle to handle challenging problems that they have not faced before and that confound the available technical knowledge (Schön 1983).

Experimentalist Review

This abstract account of practical reasoning comes to life in the distinctive approach to oversight that experimentalist organizations have developed. By conducting continual reviews of frontline practice and requiring workers to report atypical experiences, experimentalist organizations do more than verify whether practice conforms with existing standards. They also try to identify problems that have not yet been recognized, draw attention to novel ways of handling those problems, and identify how the context surrounding frontline practitioners shapes the possibilities available to them.

To conduct these reviews, experimentalist organizations must first nominate a set of incidents to scrutinize. Some empower frontline workers to nominate problems for immediate review by managers (as when an assembly line worker pulls an “Andon cord” in Toyota’s influential lean production model); others systematically review every critical incident defined by predetermined criteria (as when aviation officials review accidents and dangerous flight and landing patterns); and still others conduct periodic reviews of a sample of all frontline practice (as in quality service reviews in some state child protection systems) (Liker 2004, 129–32; Noonan, Sabel, and Simon 2009; Syed 2015, 27–28). Organizations may scrutinize not just consummated failures but also “weak signals”—small departures from expectations that did not end in disaster in the current case but could do so in the wrong circumstances, such as near misses in air travel. Close attention to weak signals is particularly important for so-called “high reliability organizations” such as nuclear power plants, firefighting teams, and airlines, where a single mistake can be catastrophic (March 1991; Weick and Sutcliffe 2007; Syed 2015; compare Sabel and Simon 2011, 22; Pickering and Klinger 2016). Finally, organizations may broaden the base of experience they can draw from by studying vicarious experience from other organizations. Nuclear plant operators in the United States, for example, are required to report unexpected events to an industry-wide system for review, and officials may issue alerts about the events to all nuclear plants nationally; other plants are expected to analyze the local implications of problems identified elsewhere (Rees 1994, ch. 7).

Once an incident has been nominated for review, the review itself aims to evaluate how well the organization handled it and identify lessons for reform. In cases of obvious error (such as airplane crashes, medication overdoses, and wrongful convictions), reviews can move directly to technical tools like “root cause analysis” to try to diagnose and cure the sources of the error—to investigate how failures at the frontlines were influenced by decisions made elsewhere in the organization, such as faulty equipment design and maintenance, poor management decisions about staffing and role assignments, and overly rigid standard operating procedures (for example, Reason 1990, chs. 7–8). In other cases, it remains unclear whether an incident nominated for review really was an error—whether the outcome was appropriate or at least the best that could be hoped for under the circumstances. Child protection agencies, for example, often find it difficult to evaluate whether a caseworker handled a particular family appropriately and whether her organization and its community partners provided the appropriate support given the unique circumstances of the case (Noonan, Sabel, and Simon 2009, 538–48). Since experimentalist organizations always treat rules and goals as provisional, they cannot simply ask whether workers and managers followed existing rules and best practices or achieved preset performance targets. Such ex ante standards are relevant but always corrigible.

How, then, do reviewers know how well an incident was handled in these complex circumstances? How do they know whether a frontline worker and the people who set the stage for her did the right thing—whether their actions are substandard, merely acceptable, or truly innovative and worthy of dissemination? There is no simple answer to these questions, for Dewey’s alternative model of rationality is fundamentally open-ended. The central question that reviews must answer is whether a practitioner can justify her actions to her peers and other stakeholders with reference to the mission she is charged with accomplishing, the situation she faced, and the support she received, particularly in comparison with other practitioners who have faced similar situations (Sabel and Zeitlin 2012, 170). Even in medicine and aviation, it is sometimes difficult or impossible to reach a firm and uncontroversial evaluation of a particular decision (Syed 2015, 90). Despite these challenges, incident reviews can still contribute to learning in several ways.

First, reviews can call attention to discretionary judgments that would otherwise remain invisible, making practice more transparent and accountable by bringing tacit knowledge and assumptions to the surface where they can be debated (for example, Sabel and Simon 2011, 92; Sabel and Zeitlin 2012, 175–76). This dynamic plays an especially important role in the clarification and revision of norms (Sabel 2006, 132, 135). For example, quality service reviews in some child protection systems have helped to clarify how workers should interpret abstract guidelines in concrete circumstances, forcing review teams to openly confront and justify their answers to difficult questions like when (if ever) violence by one parent against another should qualify as a threat to the safety of a child (Noonan, Sabel, and Simon 2009, 536, 545). By this route, incident reviews can contribute to normative progress, not just technical knowledge.

Second, reviews can frame the scope of their conclusions modestly, in a way that does not definitively decide whether an incident was handled appropriately. The overriding goal is usually not to prescribe how similar situations must be handled in the future. (Even if reviewers could reach a definitive conclusion about how well an incident had been handled, this conclusion would have limited prescriptive value because few cases are repeated exactly.) Instead, the goal is to expand the organization’s repertoire of potential strategies for handling the situations it encounters. By systematically scrutinizing a wide range of incidents, organizations can identify and disseminate under-recognized options for handling problematic situations, and they can do so even if they cannot definitively specify when those options should be selected.⁶ When frontline practitioners encounter broadly similar cases in the future, they will still have to use judgment to improvise an appropriate solution, but, as they do so, they will have access to a richer range of deliberative resources—a better understanding of the goals they should consider, a wider range of options for achieving those goals, and a longer menu of comparison cases to use as analogies (Schön 1983, 137ff). This form of progress consists in expanding the resources that judgment relies on, not in specifying the conclusion it should reach.

Finally, experimentalist organizations can treat the lessons drawn by each review as tentative, insisting on the need to continually reevaluate the provisional lessons that previous rounds of review have identified. Reviews often do spark organizational changes: rules that proved inapt get revised, training that ignored the most difficult dilemmas gets expanded, equipment that contributed to failure gets replaced, and managerial practices that contributed to error get corrected (Reason 1990, ch. 7; Noonan, Sabel, and Simon 2009, 548; Sabel and Simon 2011, 80). There is no guarantee, however, that these changes will be successful, so experimentalist organizations continue to subject them to scrutiny. In place of the chimerical hope that they can know whether their reforms will succeed ex ante, they make a commitment to continual monitoring and revision ex post. Matthew Syed (2015, ch. 3) suggests that organizations should treat their current systems and practices as a corrigible hypothesis about the best way to accomplish their goals (and, Dewey would add, about what their goals really are), and he calls on them to search intensively for anomalous experiences that will falsify this hypothesis. When they discover anomalies, they must reformulate the hypothesis—restructure their training curricula, augment their equipment, alter their supervision practices, and modify other organizational systems.⁷ Just as falsification drives science, in Syed’s view (2015, 44ff; compare Dewey 1927, 202–3), continual error detection drives organizational progress. Child protection experts provide a similar rationale for reviews in their field when they suggest that “each case is a unique and valid test of the system” (Noonan, Sabel, and Simon 2009, 542).

EMERGING APPROACHES TO USE-OF-FORCE REVIEWS

Use-of-force reviews in progressive police agencies increasingly resemble these experimentalist forms of oversight. Walker and Archbold (2014, 20) describe this approach under the heading “post incident review,” which, on their account, “is designed to determine whether the incident in question raises questions about the department’s policies, training, or supervision that need to be revised in order to prevent serious incidents in the future.” This approach has been part of the prescriptive literature about use-of-force review since at least 2001 when a prominent Department of Justice publication called for reviews that scrutinized not only whether the officer complied with the existing policies and training but also “the advisability of revising or reformulating agency policy, strategy, tactics, or training” (US Department of Justice 2001). Today, the idea that routine incident review can serve this diagnostic function has become commonplace: federal consent decrees have regularly included a general statement to the effect that review protocols should consider “whether the incident raises any additional training, equipment, tactical, or other concerns” (US Department of Justice 2014), and the recent President’s Task Force on 21st Century Policing (2015, 22) recommended that all agencies should establish a review system to “identify any administrative, supervisory, training, tactical, or policy issues that need to be addressed” and conduct “nonpunitive peer review of critical incidents.”

To understand the logic of this emerging practice of post-incident review, it helps to examine in detail how one agency carries it out. Because it has been documented extensively, I will use the Portland Police Bureau as an example.⁸ I do not present this case as an unqualified example of what use-of-force review should look like; as I describe below, Portland (like all agencies) continues to struggle with many aspects of this complex task. Nevertheless, its efforts to refine and deploy its oversight model over the past fifteen years illustrate the key elements, virtues, and challenges associated with experimentalist monitoring. After presenting the Portland experience in detail, I will briefly summarize how other agencies have employed a similar approach.

Use-of-Force Reviews in the Portland Police Bureau

The Portland Police Bureau (PPB) has been known as a pioneer in community policing for years (Thacher 1999), but even at the peak of that reputation in the late 1990s, it lacked vigorous controls on the way officers used force (Police Assessment Resource Center [PARC] 2003, 135–37). A turning point came in 2001, when the city council established an Independent Police Review (IPR) within the city auditor’s department. Over a period of several years, the IPR pushed the PPB to transform its approach to use-of-force review, enlisting help from consultants specializing in police accountability along the way (particularly, the PARC and the OIR Group). For clarity, I focus here on the PPB’s review of officer-involved shootings and in-custody deaths, though some of the same principles govern the review of nondeadly force. In its first detailed report to the city in 2003, the PARC (2003, 134) outlined a new mandate for use-of-force review that has guided the PPB ever since:

Police agencies should review officer-involved shootings and in-custody deaths with two primary goals in mind. First, they must hold their officers accountable: after mastering all of the pertinent facts they must carefully assess whether the involved officers and/or their supervisors and commanders have violated any agency policy or procedure or have acted in a manner inconsistent with their training. Second, they must use the incident as a learning tool: those charged with reviewing the case must determine what lessons can be learned from the department’s experience with critical incidents and should use those lessons to inform and improve the department’s policies, procedures, training, and management.

The review process designed to accomplish these two goals has several components. Every officer-involved shooting and in-custody death is reviewed by the detective division, internal affairs division, the training division, and the involved officers’ supervisors and command staff.

The training division reviews have proven particularly important since they originated informally in 2003.⁹ Training investigations begin by comparing how officers handled a critical incident (as documented by the detective and internal affairs investigations) with the training they have received about how they should handle an incident of that kind. To do that, investigators break the incident down into key moments, examining the moment when force was used as well as the entire sequence of events that led up to it (a PPB directive requires that “officers should ensure that their actions do not recklessly create the need to use force”) (compare Binder and Scharf 1980). They then reconstruct the officers’ actions at each moment and compare them with relevant training doctrine. In conducting these reviews, training staff may solicit input from outside experts (such as a K-9 trainer or critical incident commander) when they are relevant (for example, Griffin-Valade 2012, 10). The analysis that results may contribute both to a compliance review and to organizational learning. When officers act in ways that are inconsistent with training, it may mean that they need discipline or retraining, that supervisors have encouraged inappropriate tactics, that the training doctrine is unsound, or, simply, that no one strictly follows training doctrine at all times. When officers follow their training, analysts still ask whether the use of force was avoidable. A close look at the incident by tactical experts may reveal weaknesses in the training doctrine or problems with the equipment, communications, supervision, or other organizational factors. As a 2009 report put it, “the training division analysis should do more than address whether an officer’s actions were consistent with training. It should identify the sum of the reasonably available tactical choices available to the officer, discuss whether or to what extent PPB trains officers in those other choices, and consider the suitability of those choices, as well as additional training in those choices, in light of the conditions and situation that the officer-involved shooting presents” (PARC 2009, 73). In practice, the training division’s recommendations have often gone beyond training itself to include broader organizational changes.

Conclusions and recommendations reached during these reviews are forwarded to the PPB’s Force Review Board (FRB), which deliberates about their merits and then makes recommendations to the chief. The chief has final authority to impose discipline or enact organizational reforms based on this advice. The IPR has the authority to scrutinize the quality of the entire review process, and it has often delegated a portion of that work to its outside consultants (PARC from 2003–9 and the OIR Group from 2010 to the present). The IPR’s charter gives it the authority to review closed cases both to assess the quality of the PPB’s own internal investigations and to make “policy-related” recommendations, including changes to training, hiring practices, equipment, and supervision (Portland City Code 2017). The first task is a form of second-order review, scrutinizing the PPB’s own first-order capacity to review and learn from critical incidents in a transparent and effective way. The second is an extension of the first-order reviews themselves, bringing additional perspective to bear on critical incidents in search of potential organizational improvements.

The Death of James Chasse

A series of investigations into James Chasse’s death in September 2006 illustrates how this system contributes to organizational learning. As I discuss below, the Chasse review was more intensive than most reviews in the PPB, but, precisely for that reason, it serves as a useful vehicle for illustrating the full range of contributions and challenges involved in Portland’s approach. I will focus especially on the OIR Group’s analysis of Chasse’s death and various internal police and auditor documents that reviewed the case (Gennaco, Miller, and Ruhlin 2010).

Chasse died in the custody of two officers from the multi-jurisdictional transit police division: PPB officer Chris Humphreys and Multnomah County Sheriff’s Deputy Bret Burton. While helping PPB Sergeant Kyle Nice with another call near a bus stop, Humphreys and Burton noticed Chasse behaving erratically and concluded that he was probably on drugs. (In fact, he was experiencing a schizophrenic episode.) When the officers made eye contact with Chasse, he quickly turned and left to avoid them. Suspicions aroused, Humphreys and Burton stayed in the area, telling Sergeant Nice, “if we get a chance, we’re going to try to stop and talk to that guy.” They soon found Chasse hunched over facing a tree, concluding that he may have been urinating against it or injecting himself with drugs. As the officers approached him, Chasse grabbed his backpack and began to walk away. When Burton yelled out “hey, you!” Chasse turned his head and, according to Humphreys, shot a look of “absolute sheer terror” before screaming and beginning to run. Humphreys gave chase and eventually knocked Chasse to the ground. According to his own account, Humphreys used the “knock-down technique” (that is, he pushed Chasse in the middle of the back to disrupt his balance) and lost his own balance in the process; the sergeant, by contrast, initially reported that Humphreys had tackled Chasse.¹⁰ In either case, Chasse fell hard on the pavement, and Humphreys—one hundred pounds heavier—fell on or near him. Humphreys, Burton, and Nice struggled violently to restrain Chasse, who kicked, screamed, and bit two of the officers. At one point, Burton tried to use his Taser in drive-stun mode (despite Sergeant Nice’s protests not to) with no effect, and all three officers punched, kicked, and pressure pointed Chasse to fend off his bites and kicks and subdue him. Eventually, two more officers arrived to help handcuff and apply leg restraints. Chasse was bleeding, and, at some point during the struggle, he briefly lost consciousness so police summoned an ambulance to check his vital signs. Finding them normal, paramedics told police that Chasse was fine and could be transported to jail, but they also offered to transport him to the hospital; in effect, they left the decision up to police. The officers then carried Chasse—in restraints but still kicking, biting, and screaming—thirty to forty feet to the patrol car and drove him to the jail. When two jail nurses observed Chasse lose consciousness briefly while Humphreys and Burton were trying to sign him in, they told police that the jail would not accept him. The officers brought Chasse back to the patrol car to drive him to the hospital, but Chasse stopped breathing en route. Humphreys attempted cardiopulmonary resuscitation and called an ambulance, but it was too late. Chasse was pronounced dead shortly afterwards at the hospital. An autopsy found that his ribs (which were unusually fragile for a forty-two-year-old man) had fractured in twenty-seven places, and some of the fragments had dislodged and punctured his lungs. When interviewed by the internal affairs division, the medical examiner speculated that some of the fractures occurred when Humphreys performed chest compressions, and the rest were the result of someone falling on Chasse as he hit the pavement; they were not the result of being hit or kicked because there were no external injuries corresponding to the broken ribs. Since Chasse’s vital signs initially appeared normal, it was probably the struggle while carrying him to the patrol car that had displaced the broken ribs and caused the injury to his lungs.

The investigations of Chasse’s death emphasized several issues. Compliance review focused mainly on whether the officers should have given chase, whether they used permissible force, and whether they properly attended to Chasse’s medical needs. Despite the widespread public outcry, the grand jury declined to press charges against the officers. The medical examiner ruled the death an accident that resulted mainly from Chasse’s fall to the pavement, not (as some had alleged) from gratuitous use of force during the ensuing struggle. In the end, the only discipline was eighty hours unpaid leave for Nice and Humphreys, who had violated the department’s post-Taser medical requirements by failing to tell paramedics that Chasse had been Tased and failing to recognize that he needed medical transport.¹¹ The training division had faulted Humphreys for giving chase since, under the PPB’s training doctrine, the severity of the suspected offense—public urination—did not warrant the risks involved in a foot chase, but the transit division commander argued that Humphreys did not chase Chasse “simply because he urinated.” Based on what he had observed, Humphreys reasonably suspected that Chasse may have had an outstanding warrant, weapons, or drugs. The decision to make contact and give chase was exactly the kind of proactive policing expected in the transit division, which constantly received complaints about drug activity and had a mandate to interdict it. The transit commander’s assessment is worth quoting at length:

The Transit Police Division receives a barrage of complaints about the drug dealing that happens around MAX. Years prior, TCRT (under cover team) was created and one of its missions was to interdict in the drug trade on the Transit system. Officer Humphreys was an active member of TCRT. Transit became very proactive in its attempts to eradicate drug dealing and using, as well as other nuisance behaviors on and around the system. Officer Humphreys believed that narcotics played a part in Mr. Chasse’s behavior, and it was not an unreasonable belief. The complaints that TRIMET receives as well as the complaints from the Downtown Corridor all hinge on drug activity, either selling or using and all of the aberrant behaviors that go with street usage.¹²

From the transit commander’s perspective, Chasse’s tragic death resulted from an unlikely chain of events that no one could have predicted, and the decision that happened to set that chain in motion should not be blamed for its unforeseeable outcome. The majority of FRB members agreed (Gennaco, Miller, and Ruhlin 2010, 37).

Chasse’s death led only to modest formal discipline via the compliance review, but it prompted far-reaching organizational changes. Multiple reviews of the incident identified several mistakes that warranted remedial action even if they did not warrant discipline. The most important focused on Humphreys’s failure to recognize—and, apparently, even to consider—that Chasse was behaving erratically and aggressively because of his mental illness rather than, as Humphreys believed, because he carried contraband or had a warrant out for his arrest. The entire justification for initiating the fatal chain of events, and perhaps for using the tactics the officers used to arrest and transport Chasse, rested on that mistake. Humphreys had actually completed a two-hour “mental health awareness” class almost two years earlier, but it became clear that this minimal training did not prepare Portland officers for the everyday demands of their work, particularly given the recent surge in the city’s population of people with severe mental illnesses. The training division analysis ultimately recommended that all uniformed officers and sergeants should receive forty hours of crisis intervention training, and, after further deliberation, the PPB adopted that recommendation.¹³ (Two years later, the quality of that training came into question, as did the bureau’s lack of a substantial around-the-clock unit that could respond to mental health crises.)

In addition to the major recommendation about crisis intervention training, the training division report recommended several other reforms. No one was certain whether Humphreys was telling the truth about how he had brought Chasse to the ground, and the PPB’s compliance review eventually concluded that the evidence was too ambiguous to warrant discipline. Still, the incident underscored how dangerous it could be to terminate a foot chase with a tackle; accidentally stumbling in the course of a knockdown move certainly could land an officer on top of the man he was chasing, but a flying tackle was guaranteed to do so. In response to this vivid reminder of how tragic the results could be, the PPB strengthened training on the knockdown technique during in-service classes and roll-call videos. The training division analysis called for similar reinforcement of training doctrine on when to initiate a foot pursuit and how and when to use a Taser properly at close quarters. Finally, the training analysis made several recommendations about emergency medical care and the transport of injured prisoners.

As the Chasse case continued to attract controversy, and community critics accused the chief of police of failing to hold officers properly accountable, the IPR commissioned an outside investigation of the case by the Los Angeles–based OIR Group, headed by Michael Gennaco, the chief attorney for the Los Angeles County Sheriff Department Office of Independent Review.¹⁴ The OIR Group found the PPB’s internal reviews of the incident to be generally solid, despite a few important missteps. The leading questions, uncollected evidence, and overlooked witnesses that had been common a decade earlier were absent. Internal affairs no longer left fact gathering entirely to the detective division, recognizing that administrative investigations required evidence different from what a grand jury would need to decide whether an officer should be prosecuted (Gennaco, Miller, and Ruhlin 2010, 25). The consultants found the training division review, in particular, to be “impressive” and “remarkable” compared with other large police departments.

Despite these achievements, the review process still overlooked several major issues that deserved further attention. The first problem was the possibly fatal decision to carry an injured and still-struggling subject in maximum restraints to the patrol car several dozen feet away. The OIR Group recommended changes in policy and training:

As evidenced by the Medical Examiner’s opinion, carrying a struggling individual by his extremities places inordinate strain on the body. It is puzzling in this case why officers decided to carry a struggling Mr. Chasse the entire distance of the pursuit, when it would have been easier to drive a patrol car closer to the location where he was brought into custody. The Medical Examiner’s opinion should have caused IAD to focus on this issue as critical…. If a person must be moved while in maximum restraints, best practice is to let medical personnel transport, so the person can be placed on a gurney to fully support his or her body weight and then monitored by medical personnel. At a minimum, best practices dictate that subjects not be transported in patrol vehicles in maximum restraints, but should have the maximum restraint removed and the hobble secured to the vehicle so that the individual’s legs are still restrained but he or she can sit upright…. Policy and training should be revised regarding the transport and carrying of subjects in maximum restraints. (Gennaco, Miller, and Ruhlin 2010, 47)

Second, the OIR Group criticized Sergeant Nice’s decision to take control of the scene after he had participated in the struggle. Since he had become part of the police response, he could no longer perform his supervisory role in a neutral and effective way. (Among other tasks, that role included speaking with concerned bystanders and interviewing witnesses to the event.) No policy required Nice to relinquish command of the scene to the other sergeant who soon arrived, but “common sense” should have made the problem apparent, and a new policy was needed (Gennaco, Miller, and Ruhlin 2010, 46).

Third, the OIR Group highlighted the disagreement between the training division and the transit division commander about whether Officer Humphreys had erred in giving chase in the first place. These divergent perspectives during the review process—particularly, the appeal to the transit division’s culture and mandate in the commander’s defense of Humphrey—had already raised alarms in the PPB’s top leadership, which soon replaced the transit division’s leadership in an effort to bring its operations more in line with the rest of the bureau. The OIR Group reinforced this concern, and it extended it by criticizing the bureau’s internal affairs review for failing to hold supervisors accountable for creating a climate of expectations at odds with bureau policy. The OIR Group observed that immediate supervisors often have more influence on an officer’s tactics than departmental training does, and it acknowledged that in some cases supervisory pressures can “excuse the officer’s performance in whole or in part,” but it insisted that in those cases supervisors themselves should be held accountable (Gennaco, Miller, and Ruhlin 2010, 37). In this case, the OIR Group concluded, the PPB’s internal review explicitly discouraged this kind of scrutiny. If the bureau intended to bring transit’s priorities back into line with the rest of the agency, it needed to establish and enforce clear standards for sergeants as well as for frontline officers (35). In this respect, the consultants’ second-order review identified weaknesses in the agency’s first-order reviews as a factor that perpetuated the division’s wayward culture.

Going forward, recent reforms to the transit division had apparently begun to change the division’s culture to some extent, but problems remained. Supervision appeared to be uncommonly decentralized, in that sergeants had considerable leeway to “develop their own priorities and ways of doing business,” and this tradition left transit commanders relatively powerless to mold patrol officers’ behavior in line with the bureau’s mission. The unique multi-jurisdictional nature of the transit division exacerbated this problem. Although all transit officers worked under the nominal authority of a PPB commander, fourteen other agencies detailed officers to the transit division, and each worked under the auspices of their own policies, training, and accountability protocols. This fragmentation made it difficult to shape transit division culture and strategy.

Fourth, the OIR Group highlighted how PPB partners made it difficult to conduct a high-quality internal investigation of the incident. The private ambulance company that employed the paramedics who responded to the scene refused to allow them to be interviewed during the internal affairs investigation, despite persistent requests by detectives, apparently because the company worried about how their testimony might be used in civil suits (Gennaco, Miller, and Ruhlin 2010, 20). Similarly, the county attorney refused to allow the sheriff’s deputy to be interviewed until he had been deposed for a civil suit. The OIR Group recommended that city officials should take steps to try to develop workable agreements with the private ambulance providers and county officials on which they often relied.

Finally, the OIR Group identified weaknesses in the credibility and transparency of the existing review process for the Portland community. Chasse’s death was probably the most controversial incident in the PPB’s recent history, partly because of early reports that officers had beaten him to death. The medical examiner concluded definitively that they had not, but, once the accusation had circulated, it was hard to rebut, as many critics dismissed what they saw as self-serving claims by the PPB. The crucial issue was the widespread lack of community confidence that the bureau’s internal investigations were really independent. The OIR Group gingerly broached the challenge and importance of trying to ensure trust in official investigations, finding that some community members “were frustrated by the slow pace of the investigation and their inability to learn from the Bureau how the investigation was proceeding” and that the information void left by city hall and the PPB was filled by sometimes inaccurate media reporting (Gennaco, Miller, and Ruhlin 2010, 52). By the time the chief issued a fact sheet in the wake of the grand jury’s decision not to indict the officers involved, many community members reacted skeptically because they had already lost trust in the integrity of the PPB’s investigation. The OIR Group concluded that the IPR itself, as the main source of independent oversight for the bureau, should take a more visible and proactive role investigating and communicating with the public about critical incidents from the outset:

To the extent possible given the limits of its statutory authority and confidentiality obligations, IPR should try to increase public trust by disseminating information regarding the status of investigations and its role in attempting to ensure that the investigation will be fair and thorough and that the Bureau will reach principled decisions. Such information from the perspective of an independent entity would go a long way toward retaining the public’s trust in the integrity of the process. In addition, IPR’s response to the scene of critical incidents will also bolster the public’s confidence in IPR’s ability to effectively oversee the Bureau’s investigations of these incidents. (Gennaco, Miller, and Ruhlin 2010, 53)

Over time, the IPR’s role in critical incident investigations grew. For example, the IPR director became a voting member of the FRB in 2010.

The degree of scrutiny devoted to Chasse’s death was uncommon even for Portland, and, in many important respects, the case is not typical. Moreover, it was hardly a “success story”; it was a story of how the PPB dealt with a tragic failure and then with further mistakes and missed opportunities in the initial investigations of why that failure occurred (indeed, those blunders partially explain why the case attracted such uncommonly intense scrutiny). These features, however, are precisely what give the case illustrative power. It is precisely in the face of mistakes and missed opportunities that the learning model makes its most distinctive contribution, providing a mechanism by which errors can be surfaced and corrected. All serious use-of-force and in-custody death cases in Portland undergo the same basic process of review as Chasse’s, including intensive scrutiny by the training division, the FRB, and (for officer-involved shootings and in-custody deaths) the IPR’s outside consultants. The particularly intense scrutiny devoted to Chasse’s death, however, makes this case a useful vehicle for conveying the full range of issues and challenges that Portland’s review process can encompass. I will illustrate how these same issues and challenges arise in more ordinary cases in the following section.

CHALLENGES AND LESSONS FROM PORTLAND’S EXPERIENCE

The Chasse case illustrates how use-of-force reviews can accomplish much more than simply ensuring that officers comply with existing policies. Myriad complex and interconnected decisions combined to contribute to Chasse’s death, and while the most appropriate response to some of them may be criminal or administrative sanctions, this is clearly not the case for many of them. For example, no policy or training required the responding officers to move their patrol car closer to the arrestee in full restraints rather than carrying him, but Chasse’s tragic death brought home the importance of this simple step. This single case uncovered several lessons of this type. In this respect, Portland’s experience echoes the main theme of recent literature about learning from error in policing, demonstrating how close scrutiny of exemplary incidents can help to identify systemic weaknesses as targets for reform (for example, Doyle 2014; Hollway, Lee, and Smoot 2017; Schwartz 2018).

Portland’s experience illustrates how this form of review can be built into routine organizational practices—fact-finding by the internal affairs division and the detective division, analysis by the training division, and deliberation by the standing FRB. It also illustrates several lessons about the logic, preconditions, and challenges of the learning model.

Learning How to Learn

First, in order to use routine incident review as a tool for learning as well as compliance, the PPB had to reshape the content and scope of the review.¹⁵ The IPR and its consultants often pressured the bureau to move this process forward, using their second-order reviews to point out where the agency’s own first-order review practices needed to be reformed. PARC urged the bureau to create new dispositions for cases where officers did not violate the law or agency policy but where better tactics were available, and it called on the PPB to routinize opportunities to draw lessons about agency policy and practices (PARC 2006, 156). This shift had as much to do with organizational culture as with process: the IPR and its consultants repeatedly challenged the prevailing taboo against “Monday-morning quarterbacking” by calling out internal reviews that remained overly deferential and failed to spot or candidly criticize mistakes that officers and supervisors had made (for example, PARC 2003, 136–39; Gennaco, Miller, and Ruhlin 2012, 75). To make the review effective, the PPB needed to create a cultural commitment to critical self-scrutiny.¹⁶ Second-order reviews also encouraged the PPB to pay attention to “near misses”—problems that did not contribute to troubling outcomes in the current case but might easily have done so in slightly different circumstances.¹⁷ For example, the county deputy did not play as large a role in Chasse’s death as the PPB officer or sergeant did, but, if he had, differing organizational expectations might have posed serious challenges for accountability. As the OIR Group opined, “it is not difficult to imagine a scenario where two officers from different agencies are equally culpable for some type of violation of policy form for tactical decision but only one is held to account” (Gennaco, Miller, and Ruhlin 2010, 51).

Most importantly, the IPR and its consultants pressed reviewers to expand their focus beyond the officer’s final decision to use force and consider prior actions that set the stage for it, including the officers’ own earlier actions as well as those of the rest of the organization (Gennaco, Miller, and Ruhlin 2012, 13–14). When a rifle team officer justified her decision to shoot a barricaded suspect partly based on the poor quality of the available cover, the investigators should have questioned whether the team that had originally marked the perimeter of the scene had done everything possible to ensure that adequate cover would be available (37). When another rifle team officer acted within policy but in a needlessly aggressive way, and yet there seemed to be no way to formulate a precise policy to cover similar cases in the future, the criteria for vetting and selecting personnel for the rifle team should have been revisited; the bureau did not need new rules so much as it needed better judgment among the officers who often found themselves in such difficult circumstances (57). When a precinct officer fired because he believed that colleagues he could not see were in harm’s way, it became clear that the special emergency response team (SERT) on the perimeter had failed to communicate adequately with the officers closer in—partly because SERT officers broadcast on a different frequency than the patrol force (26–27). In each case, the officers who pulled the trigger were legally and morally justified in doing so, given the options, information, and even the capacity for judgment available to them. Nevertheless, close scrutiny of decisions made by other PPB members made it clear that each incident might have played out very differently; new protocols for marking perimeters, new selection criteria for rifle team officers, and new radio frequencies might all help prevent similar tragedies in the future. To uncover these possibilities, reviews had to expand their field of vision beyond the officer who pulled the trigger. To do this, in turn, they had to gather information that compliance-oriented investigations had traditionally ignored (for example, Gennaco, Miller, and Ruhlin 2010, 20–21). In all of these ways, the IPR continually pressured the PPB to develop a capacity for genuinely learning-oriented reviews that the agency initially lacked.

Institutionalizing Change

Second, these routine reviews of critical incidents had to feed into routine mechanisms for organizational change, and those mechanisms also had to be constructed. A 2003 investigation found that the PPB failed to implement nearly half of the organizational change recommendations that incident reviews had made, and the IPR’s consultant pressed the bureau to develop a more systematic protocol for follow-through (PARC 2003, 159). In the system that eventually emerged, recommendations made by training and the FRB are passed on to the chief’s office, which then decides whether to accept them and, if so, which PPB division should take responsibility for implementation. The training division itself is a particularly important locus for reform: when the training review of a critical incident concludes that the PPB should drastically expand mental health training, reinforce instruction in the knockdown technique, or enhance officer training in other ways, the training division itself can immediately take responsibility for appropriate actions after the FRB and chief’s office endorse them. When the review concludes that the agency should change policies, personnel standards, communications protocols, equipment, or other organizational systems, the chief’s office has responsibility for implementation. The IPR and its consultants, in turn, monitor how well the PPB adheres to its own protocols for follow-through.

These implementation protocols and review practices have apparently improved Portland’s capacity for follow-through compared with its spotty record in 2003, though commitment to systemic reform remains inconsistent. By 2010, the OIR Group, which had conducted oversight studies for several other police departments, commended the PPB for quickly and decisively implementing the recommendations that emerged from the Chasse reviews: “The evidence of resolve by PPB executives to timely implement the recommended reforms is one of the most impressive that we have seen” (Gennaco, Miller, and Ruhlin 2010, 34). More recently, the OIR Group has raised concerns that the bureau’s ability to carry out recommendations for systemic reform has faltered once again, and it has called for further reforms to ensure consistent follow-through (Gennaco, Miller, and Ruhlin 2019, 3).

Forms of Learning

Third, Portland’s experience illustrates how incident reviews can contribute to organizational development even when they do not yield uncontroversial new rules and systemic reforms. In many cases, reviews have been unable to reach definitive conclusions about how well the agency handled a particular incident or how it should handle similar incidents in the future. Even so, they have drawn attention to previously unrecognized problems, and they have surfaced disagreement and uncertainty about practices that had previously been taken for granted or ignored. For example, detailed scrutiny of the factors that contributed to Chasse’s death made it clear that the transit division’s culture had gotten out of sync with policy and organizational culture elsewhere in the PPB, as transit officers interpreted the community’s complaints about disorder in the transit system as a justification for aggressive interventions in ambiguous situations. Debate remained: Was Transit’s approach appropriate given public concerns about drug activity and disorder in the transit system? Do community concerns about drug activity in the transit system justify the kind of aggressive proactive police work that led officer Humphreys to pursue James Chasse? Or did it violate core PPB values and damage public trust in the agency? This dispute was not resolved immediately; it may be the kind of difficult policy issue that has no final resolution. The review process itself exposed sharp disagreement, as training division managers and one force review board member criticized transit’s approach while the division’s manager defended it. Detailed incident review made a contribution not by resolving this controversy but, rather, by calling attention to its existence, prompting deliberation and further investigation about an important issue that had previously remained obscure. Once broached, this controversy became an ongoing focus of concern for PPB managers.

Questions like those surrounding the transit division’s aggressive anti-drug strategy recurred repeatedly in the PPB’s use-of-force reviews. Another fatal shooting of a man experiencing a mental health crisis took place inside the mental hospital where he was confined. Hospital staff had summoned the police when the man became agitated and, they felt, unmanageable, but a post-incident review concluded that the man had committed no crime and that police should not have gotten involved. The review concluded that police needed to establish and disseminate a clear policy outlining when they would and would not respond to requests for assistance from mental health facilities:

Dealing with garden-variety management of agitated patients is the responsibility of the hospital and its staff, not armed police officers who have not been trained to control persons with mental illness, particularly in a hospital setting. The PPB should have had a protocol in place that barred response to mental hospitals for routine patient management issues unless serious criminal conduct had been committed or threatened—a threshold not approached in the incident in question. (PARC 2005, 58–59)

After several months of deliberation, the PPB eventually adopted a new directive that clarified when police would and would not respond to calls from mental health facilities; stressed “the responsibility of mental health facilities to have the proper resources to manage people in mental health crisis”; and enumerated non-police resources (such as the Multnomah County Crisis Line) that mental health facilities could access. This directive was then revised several times over the course of the ensuing years.¹⁸

In cases like these, use-of-force reviews potentially become an important input into problem-oriented policing. They provide a new source of information for identifying substantive community problems that require more careful analysis than they have received to date, such as the problem of drug activity in the transit system or the problem of agitated patients in mental hospitals, and they provide an opportunity to rethink the appropriate roles that police and other community institutions should play to address those problems. When they serve that role, use-of-force reviews provide a unique perspective on the substantive business of policing by highlighting the level and character of force required by current responses and potential alternatives. This crucial feature of the police response to recurrent problems rarely figures into problem-oriented policing analyses, despite the fact that it may be one of the most salient factors that many community members use to evaluate that response.¹⁹

Provisional Learning

Fourth, even when incident reviews do issue discrete recommendations right away, those recommendations often remain provisional. They are tentative solutions that deserve further scrutiny and experimentation rather than definitive conclusions enshrined in inflexible policy; alternatively, they may represent options that officers and police managers can add to their repertoire of possible courses of discretionary action rather than mandates dictating when they must use them.

The provisional character of many recommendations is particularly significant, and it requires a sustained, rather than episodic, approach to review. Along with everything else that reviews question in Portland, they continually scrutinize the conclusions that have emerged from previous rounds of review. The PPB’s evolving efforts to improve the way officers handled mental health crises illustrate this iterative process. Officer Humphreys failed to recognize that schizophrenia rather than criminality explained Chasse’s behavior, and this mistake set off the foot pursuit that eventually killed him. The review of the incident made it clear (again) why it is so important for patrol officers to be able to recognize mental illness and respond to it in appropriate ways. The specific intervention that the PPB adopted to pursue this goal, however, ultimately proved problematic. The bureau’s decision to devote a full forty hours of crisis intervention training to all officers was an unprecedented sign of commitment to this issue and received universal praise across the country, but, in practice, the new training proved inadequate for a variety of unexpected reasons. Among other things, it was provided too soon in the officers’ careers (during the recruit academy rather than after sufficient time in the field to make it meaningful), it lacked sufficient engagement with local mental health providers, and it lacked the realism of live role playing. Moreover, the decision to provide all officers with crisis intervention training may have convinced PPB managers that the bureau did not need a dedicated around-the-clock crisis intervention team, but, in reality, the general knowledge and motivation that training could impart to a non-specialist officer could never reach the level that a dedicated unit could develop through training and experience.

All of these problems with the original proposal eventually came to light because of the deliberate scrutiny to which it was subjected. It would have been better to obviate those problems from the beginning, but proven models simply did not yet exist because Portland was trying to press beyond the level of mental health expertise that any other police department had provided. Recognizing that their plans broke new ground, Portland officials monitored how they played out in practice over time. In particular, the city enlisted its outside consultants to “consider if the earlier recommendations … made a difference when officers employed deadly force” in the subsequent cases that they reviewed (PARC 2009, 15). The OIR Group’s first report to Portland after its analysis of the Chasse case focused on six incidents involving people experiencing mental health crises. One of the stated reasons for focusing on that group of cases was to scrutinize whether the changes adopted in response to the Chasse reviews had actually been implemented and whether they really helped to improve how police handled people experiencing mental health crises. After a Portland officer shot and killed another man experiencing a mental health crisis in 2010 and the community once again erupted in protest, the city asked the Department of Justice to conduct a section 14141 review of the PPB’s use-of-force policies and practices. This sustained commitment to review explicitly treated new practices as provisional. Recognizing that weaknesses would emerge over time, it was necessary to continuously monitor whether the solutions adopted in previous iterations of review were helping or hindering officers in subsequent incidents. This iterative character of systemic change is an important aspect of learning from error in policing; an effective approach to systems change needs to recognize its own fallibility, relentlessly scrutinizing the upshot of previous rounds of review.

Learning about the Local Context

Finally, many of the lessons drawn from use-of-force reviews are intensely local, in that they involve adaptation to Portland’s local context more than the implementation of generalized best practices. To be sure, some lessons do seem fairly generic: best practices for the transportation of struggling suspects in full restraints, the supervision of a use-of-force scene where the responding sergeant has participated in the use of force, and mental health training may appear to be the same everywhere, even they all need to be tailored to the local organizational landscape.²⁰ Even in these cases, however, identifying the character and priority of these problems in Portland required close monitoring of the agency’s own body of experience. For example, the PPB’s mental health training became a special concern that warranted intensive investment at (and even beyond) the envelope of existing national best practices because of the series of tragic encounters between PPB officers and people experiencing mental health crises, which, in turn, arose because of the apparent spike in the local population of people struggling with severe mental health challenges, along with the weaknesses of Oregon’s existing mental health services infrastructure. In this respect, incident review is partly a mechanism for monitoring the evolving local context to identify priorities for scarce organizational attention. As Portland’s ambitious new mental health training illustrates, those priorities may include the hard and risky work of innovation, attempting to push beyond the envelope of national best practices to address the special urgency that a problem has at a particular time and place.

Other major lessons from the Chasse case were even more closely tied to unique features of Portland’s local context. Consider the role of the transit police in Chasse’s death (Gennaco, Miller, and Ruhlin 2010, 49–52). Transit policing in Portland involved a multi-jurisdictional team of officers from the PPB itself, the county sheriff, and a dozen other local law enforcement agencies; only a third of the division’s officers actually worked for the PPB. This unique structure posed several challenges that had remained invisible until the Chasse investigations surfaced them. Interorganizational policy differences posed obstacles to the investigation itself, as when the county attorney refused to allow the sheriff’s deputy to be interviewed until he had been deposed for a civil suit (the PPB itself allowed its own officers to be interviewed in the shadow of similar litigation). More importantly, the transit team’s structure contributed to a distinct culture and a distinct set of priorities that contributed directly to Chasse’s death. Working alongside officers from agencies that had very different policies and cultures under a complex supervisory structure, the PPB officers working for the transit division adopted an aggressive posture that the bureau itself explicitly discouraged. This conclusion raised both normative and organizational questions. Did the magnitude of current public concern about transit safety in Portland warrant a different approach to policing the regional transit system than the rest of the city? If not, what organizational obstacles had to be overcome to bring transit officers’ priorities in line with the rest of the agency? Both of these questions were profoundly local.

These examples illustrate how and why incident review permits an agency to monitor its inherited structures, cultural commitments, relationships, and practices in light of its evolving workload and environment to evaluate whether any of them ought to be altered. In the decentralized world of American policing, agencies often develop idiosyncratic structures and practices to manage the problems that their own communities currently expect them to manage. A police department’s own organizational history (for example, the unique multi-jurisdictional structure it has inherited for transit policing) and the institutional landscape of its local community (for example, the limitations of the county and state mental health infrastructure) shape the challenges and opportunities it currently faces in controlling the use of force. By continually scrutinizing a wide range of an agency’s practices—the way it trains officers about mental health, organizes special units, marks perimeters during hostage situations, recruits members of the rifle team, screens calls from local mental hospitals, holds sergeants accountable for effective supervision, and so on—the agency adapts its existing approach to policing to its evolving environment and priorities.

None of this is to deny the importance of input from the profession at large. Reviews of the way an agency has handled its own incidents allow it to refine its capacity to police the local environment, but new ideas about how to handle those problems and benchmarks for evaluating its current responses often come from other agencies. In Portland, the IPR’s consultants advanced this agenda to some extent. The OIR Group and PARC both had extensive experience working with other police agencies, and their reports regularly described practices used elsewhere that would improve on those that Portland officers were currently using (for example, PARC 2003, 175–78; Gennaco, Rosenzweig, and Ruhlin 2016, 19–20). Whatever else they do, consultants serve as a vehicle for transmitting ideas across organizational lines. However, the Portland police apparently did not take other systematic steps to stay abreast of ideas developed in other cities. PARC stressed the importance of this task early on, noting that the Washington, DC, Metropolitan Police Department explicitly charged its force’s investigation team with monitoring practices in other agencies and police industry publications (PARC 2003, 54). That effort is crucial because experiences elsewhere are a key resource for the open-ended judgments that experimentalist reviews have to make, allowing reviewers to ask whether an agency handled a particular situation as well or better than others in similar circumstances (compare Sabel and Zeitlin 2012, 170).

BEYOND PORTLAND

Portland’s experience echoes similar experiences in several other cities that have embraced the learning model (for example, PARC 2003, 53–54; Seattle Police Monitor 2015; Spokane Police Department 2015; Crime and Justice Institute 2016, 22–23; Hutson 2018, 4–5). While the details of each city’s approach to review differ, most resemble Portland’s process in that they conduct learning-oriented reviews routinely rather than episodically, usually by using chain-of-command and internal affairs reviews to gather evidence and then forwarding them to a standing committee similar to Portland’s FRB for deliberation. In most cities, a local or federal monitor (or both) audits these internal processes to maintain their integrity and encourage continual improvement, as the IPR does in Portland. The scope of incidents covered varies, as some cities only review deadly force incidents (for example, Crime and Justice Institute 2016, 22), while others also review a sample of less serious incidents (for example, Seattle Police Monitor 2015, 3).²¹

Routine incident review systems in other cities aim to advance many of the same goals as Portland’s, including systemic change, normative clarification, and local adaptation. First, all of them are expected in one way or another to identify the root causes of use-of-force incidents, considering how improvements in training, policy, supervision, equipment, communications, and other organizational infrastructure can reduce the likelihood that officers will use force in the future (for example, Stewart et al. 2012, 13, 23; Seattle Police Monitor 2015, 3). In Spokane, for example, routine incident reviews have led the agency to reconsider its foot patrol policy, add padding to the passenger compartment of patrol cars, and improve the description of de-escalation techniques during in-service training (Spokane Police Department 2015, 7). Second, reviews advance the normative work of identifying unrecognized problems and clarifying ambiguous priorities and standards. In Seattle, for example, the chain of command review and the FRB reached different conclusions about whether an incident qualified as a “pursuit” under existing State Personnel Department policy, and the controversy led the chief to refer the vehicle policy back to the policy unit for clarification (Seattle Police Monitor 2015, 13). Finally, reviews regularly identify idiosyncratic local circumstances that affect officers’ use of force and the organizational procedures for controlling it. In Albuquerque in 2016, the city’s new federally mandated FRB investigated a fatal shooting by an Albuquerque Police Department (APD) detective who had recently joined the Federal Marshal’s Task Force, and the review discovered troubling policy differences between the two agencies that may have contributed to the incident; the APD soon withdrew from the task force and initiated a review of its memoranda of understanding with other federal agencies.²² In Seattle, foot patrol officers who used pepper spray traditionally had not carried their own decontamination tools, as the department relied on quick response from the fire department when necessary, but incident reviews found that the city’s growing traffic problems increasingly caused unacceptable delays; ultimately, the department required all officers who carried pepper spray to carry their own decontamination tools as well (Seattle Police Monitor 2015, 18). Examples like these illustrate how idiosyncratic local conditions, from the major to the mundane, have been an important target for the reforms that have emerged out of incident reviews in these cities.

These cities’ experiences also echo the challenges Portland has faced. In city after city, it has proven difficult to develop and sustain the police department’s commitment to routine review, particularly in less serious use-of-force cases (for example, Mitchell 2012, 15–16; Seattle Police Monitor 2015, 14; Bromwich Group 2016, 33–68). This challenge is not unique to the learning model. The most intensive study of the long-term sustainability of use-of-force reviews in any American police agency found that the quality of both learning and compliance reviews in the Washington, DC, Metropolitan Police declined markedly after the federal consent decree that had mandated them came to an end (Bromwich Group 2016, 33–68). External oversight bodies similar to Portland’s IPR have repeatedly had to press agencies for renewed vigor in their incident review practices (Mitchell 2012, 15–16; Seattle Police Monitor 2015; Bromwich Group 2016). In Portland and elsewhere, the tenacity and professionalism of these oversight agencies contribute crucially to the learning model’s sustainability (Walker and Archbold 2014, 179ff).

A more distinctive challenge for the learning model has been the difficulty of ensuring that recommended systems changes are actually implemented. As noted above, Portland itself made great strides in this area over time, though progress in this area has been hard to sustain. Most police departments have found follow-through at least as challenging as Portland has. In Seattle, over half of the FRB’s organizational change recommendations essentially disappeared into the void, as the units responsible for implementation never reported back on follow-through. To address this systematic failure, the agency’s Compliance Bureau developed a formal process for monitoring implementation of the board’s recommendations (Seattle Police Monitor 2015, 4, 18). Las Vegas similarly struggled with implementation of the systemic change recommendations generated through incident review, and, eventually, it established a formal “Accountability Matrix” to track proposed reforms (Stewart et al. 2012, 27). Once again, external oversight from police auditors has played an important role in continually pressing for progress on this front.

CONCLUSION

The purpose of routine reviews of officers’ use of force seems deceptively simple: to make sure that officers follow law and policy. This role remains important, but we should recognize what it leaves out. Some of the most sophisticated recent research and practice related to the use of force of policing has stressed that the tactical choices made by individual officers on the frontlines are only one factor—and perhaps not the most significant one—influencing the likelihood that force will be used and with what results. Moreover, the tactical options available to officers and the standards appropriate for evaluating them continually evolve. Reviews that simply ask whether or not an officer’s actions comply with existing rules may miss the most important opportunities for controlling the use of force.

What else can be done to accomplish this crucially important goal? Recent scholarship and practice demonstrate that police departments and researchers can scrutinize how systems features like training, equipment, supervision, and communication protocols contribute to the use of force (Pickering and Klinger 2016; Hollway, Lee, and Smoot 2017; Schwartz 2018; Sherman 2018). Some authors hope to identify these features through systematic research about the impact of organizational design on use-of-force outcomes (Sherman 2018); others call for occasional reviews of exemplary incidents by special committees convened outside of existing organizational routines (Hollway, Lee, and Smoot 2017, 909–11; Doyle 2019; Browning et al. 2015; Shane 2013); and still others call for informal, face-to-face discussion of difficult incidents among officers during training sessions and similar forums (Alpert 2015, 3–4; President’s Task Force on 21st Century Policing 2015, 19). These compelling strategies for organizational learning deserve a place on the police reform agenda, but they do not exhaust the space of possibilities that the compliance model excludes. Cities like Portland illustrate a less episodic approach to organizational self-scrutiny that integrates learning into existing procedures for routine incident review. This approach allows them to identify emerging problems, continually question agency priorities, and adapt organizational practices to the constantly changing local environment. By that route, it contributes to the important work of normative progress and local adaptation.

Like other learning-oriented strategies for controlling the use of force, Portland’s approach pursues this goal in a more open-ended and potentially more demanding way than the compliance model does. The compliance model imposes a static and determinate standard of “excessive force,” insisting that officers must not use force in predefined circumstances. The learning model imposes a more dynamic standard, insisting that police agencies and individual officers have a duty not only to comply with known best practices but also to continually search for new ways to avoid the use of force; along the way, it requires them to continually scrutinize whether their existing practices can be justified in light of their evolving understanding of agency priorities. That more demanding ideal may provide an appealing way of understanding the legal and moral values implicated by the use of force in policing, replacing (or at least supplementing) fixed substantive rules with something like the “duty of responsible administration” observed in other areas of legal thought (Sabel and Simon 2016). As Egon Bittner (1990, 262) argued, the point of concentrating the monopoly on the legitimate use of force in a single institution is to ensure that it will be used less intensively and more responsibly than it otherwise would be. From this perspective, the mission of policing is to minimize the use of force in society by continually monitoring and refining practices for resolving those problems in which force may have to be used. As a species of experimentalist oversight, the learning model pursues this mission by striving to progressively expand the capacity of police organizations to resolve those problems in more restrained ways and to make the rationale for any remaining uses of force more transparent.

A skeptic might reasonably ask what review systems like Portland’s have accomplished. Has the learning model actually reduced the use of force in the agencies where it operates? The question is not easy to answer; the learning model is never implemented in one fell swoop to permit simple before-and-after comparisons, and it is invariably accompanied by other reforms and changing background conditions that may affect both the use of force and the way officers report it. Cities like Portland, Seattle, and Washington, DC, certainly cite evidence that the rate at which officers used various types of force declined substantially after they implemented packages of reforms that included the learning model (for example, PARC 2009, 22; Seattle Police Monitor 2017, 2–4; Bromwich Group 2016, iv; Prenzler et al. 2016).²³ This evidence, however, remains ambiguous since reporting practices, definitions, and even the significance of various types of nondeadly force are too variable to permit easy impact analyses of any reform that might affect them. Deadly force is measured and conceptualized more consistently, but it is used so rarely that even a large percentage change in the number of deadly force incidents may reflect nothing more than random variation; large post-reform declines may reflect regression to the mean after the elevated rates that prompted reform in the first place.

Even if we could overcome the technical obstacles that make it difficult to identify the impact of the learning model on the use of force, the experience of cities like Portland suggests that this way of framing the question is too narrow. While a major goal of the learning model is ultimately to reduce the use of force, it is not the only one. Review also aims to make police work more transparent and reflective, forcing a department to publicly confront and justify the implications of its existing practices and priorities for the use of force. Vigorous review practices can be understood, to borrow Sabel and Simon’s (2016, 201) words from a related context, “not just as instruments to greater productivity, but as intrinsic values that constitute a kind of organizational virtue”—as the embodiment of “a duty to examine rigorously the effects of conduct on civil rights values and to resolve ambiguity by articulating provisionally but reflectively the organization’s understanding of issues that have not been resolved externally.” To evaluate how well the learning model accomplishes this broader ambition, it is necessary to consider processes as well as outcomes and to make qualitative as well as quantitative judgments (see, in comparison, Sabel and Simon 2016, 200). Multiple investigations of Portland’s system by outside consultants have found that the agency has continually improved on these dimensions. Reviews have increasingly provided candid pictures of use-of-force incidents, scrutinized how officers and the rest of the organization can and should handle similar incidents more sensitively, and surfaced important, but previously neglected, questions about how police should understand their complex priorities; the review process has also monitored how well, and with what results, the agency has implemented the recommendations that have emerged from previous rounds of review (for example, PARC 2009, 61ff; Gennaco, Miller, and Ruhlin 2012, 2–3, 13ff; Gennaco, Rosenzweig, and Ruhlin 2016, 35–37). In these respects, Portland’s experience provides evidence that with sustained effort and external pressure, it is feasible to use routine incident review to make police agencies more reflective, transparent, and restrained in the way their officers use force.