Regulatory discretion: structuring power in the era of regulatory capitalism

Abstract

Discretion gives decision makers choices as to how resources are allocated, or how other aspects of state largesse or coercion are deployed. Discretionary state power challenges aspects of the rule of law, first by transferring decisions from legislators to departments, agencies and street-level bureaucrats and secondly by risking the uniform application of key fairness and equality norms. Concerns to find alternative and decentred forms of regulation gave rise to new types of regulation, sometimes labeled ‘regulatory capitalism’. Regulatory capitalism highlights the roles of a wider range of actors exercising powers and a wider range of instruments. It includes also new forms of discretion, for example over automated decision making processes, over the formulation and dissemination of league tables or over the use of behavioural measures. This paper takes a novel approach by linking and extending the significant literature on these changing patterns of regulatory administration with consideration of the changing modes of deployment of discretion. Using this specific lens, we observe two potentially contradictory trends: an increase in determining and structuring administrative decision, leading to a more transparent use of discretion; and the increased use of automated decision making processes which have the potential of producing a less transparent black box scenario.

‘In all advanced nations of the world, justice is administered more outside courts than in them’. ¹

Introduction

The expansion of state administrations in the twentieth century generated a degree of anxiety about the growth of administrative discretion. For Dicey, famously, administrative discretion was a threat to the rule of law in England.² However, discretion is an essential feature of delegation to government departments and agencies and became progressively more significant as the state took on more functions, notably those associated with the development of the welfare state. While the exercise of discretion by public officials is essential, it has raised concerns about compliance with norms that the largesse and coercive capacity of the state should be exercised only in accordance with the rule of law, and with the norms which are determined by elected politicians through recognised legislative processes.

Importantly, there has been a significant shift in European countries from governance through modes identified with the welfare state (which frequently implied substantial administrative discretion in such areas as welfare provision, health, housing, education and provision of public utilities and the potential to treat similar cases differently) towards more regulatory modes of governance, sometimes characterised as ‘the rise of the regulatory state’. The regulatory state mode of governing is said to involve a shift to greater use of arms-length agencies and with the effect of displacing some forms of discretion with rules and processes for monitoring and enforcing their compliance. The growth of such classic regulatory modes of governing may be thought to reduce or eliminate challenges of administrative discretion, through a more rule-based mode of governing. Nonetheless, even with such classic regulatory modes, agencies have substantial discretion, for example over when and how to enforce to maximise regulatory outcomes and to prioritise resources through choice of instruments.³ The influential theory of responsive regulation supplies a guide on how to exercise standard enforcement discretion through escalating from less stringent measures such as educating, advising and warning to more stringent responses such as civil and criminal penalties and licence revocation.⁴ However, from a rule of law perspective the discretion which is explicitly recognised as a component of regulatory enforcement is problematic.⁵

Beyond the regulatory state, contemporary thinking identifies a wider range of instruments and actors, public and private, as comprising a governance mode labelled ‘regulatory capitalism’. Regulatory capitalism sees pervasive regulation, going beyond agencies and rules, as embracing a wide range of regulatory actors and instruments, including non-state actors and both soft law and private law instruments.⁶ We suggest that the concept of regulatory capitalism entails the application of regulatory modes of governance that apply equally to delivery as to oversight functions of government, and may also be found in relationships with NGOs and firms cast both as regulators and regulatees. Thus, there is a diffusion of regulatory power associated with regulatory capitalism which takes quasi-public power further from elected officials and from legal rules with respect to core governmental functions both of public service delivery and oversight.

Furthermore, a certain disenchantment with command and control regulation as a mechanism for effectively changing behaviours has led to the search for new instruments which may be more effective but may also involve more extensive discretion. One set of newer instruments, informed by the behavioural sciences, and now mandated by a US Executive Order⁷ (but not in the EU better regulation package of 2015⁸), may create a new challenge of applying discretion to determine when to shape ‘choice architectures’ to promote better behaviours by citizens and others. The use of ‘nudges’ and other behavioural measures to steer behaviours towards better choices is particularly challenging. First, the initiation of behavioural measures may not require new legal rules, enabling the use of unregulated discretion. Secondly, the exercise of such discretion may not be apparent to those whose behaviour is targeted. Thus, nudging is open to a charge of manipulation.⁹ Other novel sets of regulatory instruments align around new technologies as further alternatives to command and control regulation. This is particularly the case with architectural or design-based forms of technological management.¹⁰ Furthermore, automated decision making processes are also increasingly utilised to assist regulators, whether on the standard setting, monitoring or enforcement level.¹¹

In this paper, we evaluate the challenge of discretion in models of regulatory capitalism. We ask what evidence there is that practices have been developed to address discretion both in government agencies and in the arguably more challenging discretionary practices of transnational and private regulators, who are further removed from elected government. It is significant that the measures which structure discretionary power operate ex ante and directly with control effects on primary decision making (and equivalent processes) as compared with much discussed mechanisms of accountability which operate ex post. The nature and scope of administrative discretion under regulatory capitalism, going beyond national state bodies, requires investigation. We initiate that inquiry and offer an evaluation of the nature of and motivations for the emergent measures within regulatory regimes, which structure and shape the exercise of discretionary powers. These include traditional forms of providing guidance to a regulatory decision maker, the application to increasingly significant behavioural measures, as well as newly emerging forms relying on technological assistance.

1. Setting the scene: the regulatory state and regulatory capitalism

The welfare state established itself as a primary mode of governance in much of Western Europe in the twentieth century, underpinned by legislative frameworks, which, to varying degrees assigned discretion to public officials over such matters as housing, health, education and welfare payments, but also extended into other areas such the use of government capacity to offer utility services and to support businesses.¹² Pulling back from the welfare state, addressing fiscal challenges whilst also meeting ideological challenges to large state models, the rise of the regulatory state was one of the key policy trends throughout most of the OECD member states and beyond from the 1980s. This shift draws on ideas and institutions of the US regulatory state which emerged in an earlier period, initially in the late nineteenth and early twentieth centuries. The independent regulators of the US originated in administrative tribunals such as the Interstate Commerce Commission (established 1888) and were modelled on British court-like structures, determining rights through adjudicatory models. The gradual shift of the ICC from a reactive adjudicator to proactive regulator set a trend that was followed by the wave of new deal regulators established in the 1930s and then the social regulators established since the 1960s.¹³

The trend to establish regulatory agencies to take responsibilities at least for applying (if not making) regulatory rules spread to Europe, with many social and economic regulators established from the 1960s and the pace accelerating from the 1980s with the twin tendencies towards liberalisation of once publicly operated and monopolised markets and the promotion by the EU of independent regulators to insulate decision making from politics and risks of national favouritism.¹⁴ This trend, captured in the idea of the regulatory state, is characterised as including a shift towards more arms-length agencies, the separation of policy making from regulation (and delivery) of public services and, highly relevant to this paper, from discretionary to more rule-based instruments.¹⁵

The shift to more rule-based modes of governing within the regulatory state has a number of dimensions. Amongst the most obvious of these is found in the provision of public services, where discretions to ministers or what are or were publicly owned service providers have increasingly been displaced by regulatory rules setting down expectations on service providers in instruments akin to contracts.¹⁶ In the UK such contractual-type instruments are found not only in respect of the licences issued to private or public service providers, but also in service-level agreements governing intra-state relationships between policy making ministries and delivery agencies, and between public bodies and companies delivering services under both procurement contracts and more specific legislation for delivery of public services such as refuse collection.¹⁷

Whilst the regulatory state trend is identified as being a move away from welfare state modes of governance, it has not eliminated such modes of governing nor the government imperative to deliver welfare services. Indeed, there are lively debates about how changes in thinking about governance have led to both less and more rule-based approaches to welfare payments¹⁸ and also considerable debate about the extent to which regulatory modes might support rather than undermine the redistributive ambitions of welfare regimes.¹⁹

Furthermore, regulatory modes of governance, whilst more rule-based than the discretionary provisions of welfare state modes, frequently invoke principles as the basis for regulation. Principles-based regulation (PBR) recognises that it is not always efficient and effective for the legislature to anticipate and set down detailed expectations of regulatees.²⁰ The setting down of principles challenges regulatees to work out how to achieve the broad objective. Examples are found in the common law systems which provide defences to criminal regulatory conduct for those who can demonstrate that they exercised due diligence (a principle which can be satisfied by demonstrating that, at the discretion of the regulatee, a system to comply was put in place and implemented).²¹ Such systems exemplify the use of principles as the core standards for the regime, introducing significant discretion as to implementation both for regulator and for regulatee.²² As a consequence of assigning considerable discretion to regulatees, PBR and other meta-regulatory ways of governing require strong oversight capacity.²³

Alongside the discretion associated with relatively open-textured regulatory standards, regulatory enforcement also typically involves, at least implicitly, fairly extensive discretion in deciding how to monitor for compliance and how to address infractions when detected. The older literature contrasts punitive or deterrence-based approaches, where formal enforcement actions were normal and routine, with compliance-based approaches which seek to encourage regulatees towards compliance with education and advice.²⁴ It has been widely noted in empirical research that enforcement agencies adopt differing approaches to how they use their discretion around enforcement.²⁵ The seminal contribution of Ian Ayres and John Braithwaite suggests that these two polar opposite approaches to enforcement are frequently combined within an ‘enforcement pyramid’ in which regulators initially seek to persuade and educate regulatees into compliance and only when such approaches are ineffective follow up with the more punitive methods of fines, prosecution and licence revocation.²⁶ The pyramidal concept has both an empirical basis, in that many regulators across numerous jurisdictions recognise it as consistent with that they do, but also a normative basis in that it offers means to address both the willing compliers (who generally respond well to education and advice) and the ‘amoral calculators’ who will comply only where it is in the interests of their business to do so. For this latter group, the carrying of the big stick (that is the threat of the more punitive sanctions), if credible, enables them to speak softly and secure compliance at the lower level. With a third group, the incompetents, the low level sanctions may not be appropriate and it may be better to take them straight out with such sanctions as licence revocation or closure of premises (where available) or prosecution.²⁷ The theory of responsive regulation suggests that a high level of discretion is necessary for regulators to be able to match the sanctions to the regulatee so as to use resources most effectively to promote compliance. In the next section of this paper we proceed to look in more detail at the complexities of discretion and the challenges presented in the context of regulatory state modes of governing.

The ink was hardly dry on the concept of the regulatory state in Europe when scholars began to articulate a somewhat broader conception of regulation which observes that regulatory governance characteristics are not limited to relationships amongst state bodies and between state bodies and businesses but extend to relationships in which NGOs and businesses are cast both as regulator and regulatees. This wider conception of regulation as involving more actors and more instruments has been labelled ‘regulatory capitalism’.²⁸ Whereas the welfare state model tended to place considerable emphasis on the discretion of state bodies in delivering public services, the emergence of the regulatory state model saw the separation of ministries, regulators and delivery bodies (the latter including public, privatised and private organisations) and increasing governance through rules.²⁹ Regulatory capitalism maintains a continuing emphasis on rules, but involves a decentring of the state, and the supplementation of legislative rules with contracts as ministries, firms and NGOs are each potential principals for regulatory governance and similarly state, market and NGO actors are each potentially both regulators and regulatees. Within businesses, we see firms regulating themselves in respect of such matters as corporate social responsibility and environmental obligations.³⁰ Businesses also regulate others, for example through the specification of requirements in supply chain contracts, and collectively regulate each other through setting and applying regulatory standards.³¹ Moreover, NGOs have a key role in regulatory capitalism setting, monitoring and enforcing, through a variety of mechanisms.³² The beneficiaries of regulation are conceived of as extending beyond citizens, to consumers, customers and also employees. All sectors are increasingly affected by indications that we are experiencing a fourth industrial revolution with both work and decision making increasingly being shaped by advanced information technologies.³³ We suggest that the concept of regulatory capitalism can be appropriately extended by recognising that modes of regulating through legislative rules and contracts are supplemented by behavioural measures (including taxation and the setting of default rule such as auto-enrolment in pensions) and algorithmic decision making, the latter with a tendency to reduce day-to-day discretion at street- or screen-level, arguably substituting a more hidden systems-level discretion.³⁴ EU and international organisations are also increasingly important actors in regulatory regimes, creating a further dispersion of authority. Thus, regulatory capitalism is somewhat diffused, involving regulation by state agencies of business, but also the growth of regulatory relationships based on a wider range of actors and instruments beyond rules.

Contemporary regulatory capitalism thus raises significant challenges for the debate concerning law and discretion, whether we are thinking about classical rule making, monitoring and enforcement by state regulatory agencies, or of the increasingly diffuse regulatory modes which draw in a wider range of actors and instruments.

2. Changing conceptions of discretion

(a) Changing conceptions of discretion within state administrations

Administrative discretion emerges in settings where those charged with implementing an administrative regime have scope to make a number of different decisions, often with reference to criteria or objectives. Davis states that ‘an officer has discretion whenever the effective limits on his [sic] power leave him [sic] free to make a choice among possible courses of action or inaction’.³⁵ When thinking about ‘effective limits’, the discretion may be explicitly stated within the governing regime (de iure discretion) or may be implicit as an aspect of implementation, for example because of a lack of effective oversight (de facto discretion).³⁶ This latter form of discretion is captured by Dworkin's memorable image of the hole in the doughnut, where the fried dough forms a belt of control, while the space in between gives de facto discretion.³⁷ For social scientists it is said that the interest here is not so much the risk of discretion undermining the law, but rather the fact that the implementation of meaning of law is fundamentally shaped by the taking of decisions by administrators.³⁸ Similarly, regulatory scholarship is typically more interested in how decision making shapes a regulatory regime and its effects.

The presence of administrative discretion both de iure and de facto is a source of anxiety to many lawyers.³⁹ We elect legislators to make decisions as to how the financial resources and coercive power of the state are to be deployed and we hold them accountable for the exercise of that power. Once discretion is delegated, implicitly or explicitly, to unelected officials the democratic source of legitimacy for public spending decisions and decisions to coerce others is diluted or removed. The decisions are now the decisions of ‘street-level bureaucrats’ rather than of elected politicians. How may we know that the capacity of the officials to make discretionary decisions is being made for the public purposes set down in the legislation as compared with some private purpose, for example to assist a friend, to support the values of the official (as opposed to those of the elected politicians) or to provide the official with a somewhat easier job, quieter life or self-enrichment? All these, and other potential deviations from the public will are said to be possible where discretionary power is exercised and are characterised in the principal-agent literature as representing aspects of shirking or drift.⁴⁰ There was a lively debate concerning UK welfare policy in the 1970s and 1980s between those who asserted that welfare payments should be anchored in (justiciable) rights, on the one hand, and those who suggested that discretion was inevitable and helpful in seeking to meet the needs of welfare recipients.⁴¹

The growth in judicial oversight of administrative discretion, through the setting down of procedural norms, both legislatively, as with the US Administrative Procedures Act, and judicially has been a key part of the response to the recognition of widespread administrative discretion. Judicial review has been supplemented by the establishment of specialist tribunals in some common law jurisdictions.⁴² However, the focus on judicial supervision of administrative decisions is generally narrow. Writing about administrative justice, Davis suggested that ‘American and English administrative law has been devoted to the 10 or 20 percent of the administrative process that involves either formal proceedings or judicial review, and it has almost completely neglected the 80 or 90 percent that involves informal or unreviewed action’.⁴³ For our part we think that the estimate of the informal or unreviewed administrative actions may be too small at 80–90%, and is more likely in the band 90–100%. Whilst judicial oversight contributes to street-level decision making, much of this contribution is reactive. The reactive tendency has been supplemented in some cases by a more proactive effort to give effect to administrative law norms within administrative decision making.⁴⁴

Within a regulatory setting significant objections to the varied approach to enforcement discussed above have been expressed in the language of the rule of law.⁴⁵ Leighton McDonald sets down the beginnings of a template which recognises both external supervision of regulators through courts and also internalisation of institutional checks on discretion within agencies, seeking a way to ‘to render governance calculable in advance by reference to generally applicable rules (or principles)’.⁴⁶

The English courts have not generally been attracted to this injunction to constrain administrative discretion with rules, explicitly adopting a norm against the fettering of discretion.⁴⁷ The effect of this norm is to prevent those who are subject to administrative decisions relying on a public agency to enforce its own rules, since it must generally have the discretion to meet the circumstances of the case before it, ‘subject only to general constraints of legality and reasonableness’.⁴⁸ A recent analysis suggests that case law on the no-fettering doctrine allows the courts to regulate the degree of structuring of discretion carried out by public bodies, with considerable variation in the kinds of cases where a hard structuring of discretion will be rejected and those where it will be accepted.⁴⁹ There may also be a legitimate expectation that public bodies will follow their own norms, lest there should be an unfair balance in the sense that those subject to the agency decision are expected to follow the agency's rules whereas the agency is not.⁵⁰ Furthermore, there may be circumstances where it would be unreasonable for an agency exercising discretion not to develop some rules to structure its decisions and to make the basis for its decisions more transparent.⁵¹

In the context of regulatory capitalism we consider it helpful to think of discretion in an extended manner. Whilst the literature traditionally conceives of discretion as arising in the context of administrative implementation in state agencies, this now requires elaboration in two directions. First, research on regulation has gradually eroded a bright line between setting and implementing norms, with increasing recognition that – either formally or informally – norms may be set through such practices as issuing guidance, setting down enforcement strategies, devising behavioural interventions or narrowing or broadening the range of outcomes in an algorithm. Accordingly, we think of the problem set which may be addressed by structuring discretion as extending from the implementation practices to include rule making or norm-setting also. Secondly, to the extent that power and capacity is held beyond state agencies, whether in associations, firms engaged in making and implementing contracts, devising behavioural measures or algorithmic decision making, at both national and transnational level, we should review also their discretion and the measures through which it is structured.

(b) Administrative discretion and new regulatory instruments

The shift towards more rule-oriented governance may be conceived of as reducing discretion (but with the qualification that enforcement practices, in particular, frequently involve quite wide discretion, as noted above), but the increasing recognition of other kinds of instruments which focus on changing the behaviour both of delivery organisations and consumers and citizens directly appears to import a further layer of discretion in policy making and implementation.⁵² The fact that such behavioural instruments are frequently largely self-enforcing may be positive, in the sense that there is no discretion on the part of the regulator at the enforcement stage, but this calls for us to focus more centrally on the policy making stage. Nudges appear often to be adopted as experimental measures with quite a high degree of official discretion. Critique of behavioural instruments has tended to focus on the ethics of changing (or more pejoratively manipulating) the ‘choice architecture’ in ways of which those whose behaviour is affected may be unaware. Such instruments are thus open to two objections: first, that they are not transparent (and thus may be manipulative);⁵³ and secondly, that they may rob citizens of agency if they are steered towards making better choices rather than making them of their own volition.⁵⁴ Given the nature of nudges, that they change choice architectures rather than mandate behaviours, they frequently will not require legislative authorisation or oversight and have the potential to extend the discretion of those exerting regulatory power. This is not the case for all nudges. Default enrolment in pensions schemes, for example, requires legislation to change the settings, but reorganising the display of food in a school canteen does not.

Besides behavioural tools, another increasingly common phenomenon is the extended use of new technologies in regulatory processes. One way in which this happens is that regulators use automated decision making based on machine learning to inform or determine administrative processes.⁵⁵ Automated decision making ‘mines data to identify patterns that are then turned into predictive analysis, which is in turn applied to new data’.⁵⁶ There are several steps involved in such a process. Usually an algorithm is first applied to training data, from which it learns certain patterns based on correlations in the data. Based on this a model is developed which, after further tests, can be applied to new datasets and allows for predictions that can inform also administrative decision making processes.⁵⁷ There are already numerous applications of machine learning to assist in administrative processes.⁵⁸ Examples range from New York City using machine learning to determine where to send building inspectors to detect illegal conversions into additional rental units;⁵⁹ to Estonia using automated processes to help decide on subsidies for farmers;⁶⁰ or Dutch welfare authorities determining the risk for social security, labour law and tax fraud.⁶¹

Another regulatory strategy based on new technologies is to create architectural barriers that directly steer and limit regulatees’ behaviour. Roger Brownsword termed this latter approach technological management.⁶² Here rule-based regulation is replaced through the implementation of technological fixes that are design-based.⁶³ A classic standard example is password protection, which allows only those users duly registered and supposed to enter a certain domain to do so. There is also the possibility for a combination of different types of regulation based on new technologies. For instance, the City of Los Angeles uses machine learning in its traffic signalling system. Even though in this example we find an automated decision making process which puts in place rules without direct human engagement, we can easily imagine the combination with physical or virtual barriers, such as a gate.⁶⁴ Karen Yeung developed a taxonomy consisting of eight different algorithmic systems, which distinguish between whether standard setting is fixed or adaptive; whether the system functions in a reactive or pre-emptive/predictive manner; or whether it provides a recommender system or is automated.⁶⁵ Importantly, these novel regulatory tools are not only applied by public regulators. In most cases they were developed and first implemented in the private sector and are therefore also often widely used in private regulatory settings.⁶⁶

However, particularly in their application by public administrations, the use of new technologies brings new significant challenges to administrative decision making and discretion. We distinguish between two ways in which automated decision making and technological fixes impact administrative discretion: first, there is the question how much discretion public officials should have in applying and relying on these new regulatory tools. Secondly, when structuring discretion, we may ask how much this may be facilitated and limited through the application of new technologies. Regarding the first issue, there is an increasingly lively academic debate, with a strong focus on the rule of law implications of these new regulatory tools.⁶⁷ In practice, we can also observe the emergence of sets of requirements for engaging automated decision making in the public sector, such as the UK, Office for Artificial Intelligence's Guide to using artificial intelligence in the public sector.⁶⁸ The second issue has received particular attention in the area of preventive policing.⁶⁹ Yet, also in areas beyond policing, automated decision making is increasingly assisting and determining the exercise of administrative discretion across a variety of areas beyond policing. In section (c) we will further depict this development, by showcasing examples of how new regulatory tools assist in the structuring of administrative discretion.

(c) Administrative discretion and new regulatory actors

As mentioned above, regulatory capitalism not only involve new instruments but is in particular also characterised by new types of regulators with no direct link to the traditional administrative apparatus. Due to the significant expansion of regulatory activity by non-state actors, we can observe efforts to build procedural frameworks both in legal scholarship, as well as in regulatory practice. A notable example is the Global Administrative Law (GAL) project.⁷⁰ The GAL project conceptualises the development of administrative procedures and principles as emerging global administrative law.⁷¹ GAL principles themselves provide a limitation to regulatory discretion. Thus, requirements such as reasoned decisions, proportionality, means-ends rationality, avoidance of unnecessarily restrictive means, and legitimate expectations, are – unsurprisingly – well established principles and procedures to curb and scrutinise the domestic exercise of discretion. Beyond theory, transnational regulatory practice is indeed characterised by increased proceduralisation. The development can be found in both international governmental organisations such as the UN Security Council or the World Bank,⁷² as well as in privately controlled organisations, whether they are NGO-⁷³ or industry-driven.⁷⁴

Furthermore, we can witness the establishment of so-called meta-regulation.⁷⁵ An often-cited example here is the ISEAL Alliance, which develops codes for standard setting, implementation and impact assessment practices, largely to support the market credibility of the labelling-based NGO-driven standards regimes.⁷⁶ These codes significantly structure the practice of participating private regulators regarding standard setting and implementation processes. Thus, the standards setting code provides requirements for standards development and revisions such as how to identify stakeholders, how to conduct public consultation, etc.⁷⁷ Similarly, the ‘Compliance Code’ sets out standards for monitoring and enforcement procedures.⁷⁸ Both instruments do limit the exercise of discretion. Thus, a decision to deny certification, for instance, needs to be based on a pre-specified certification process, must be based on pre-determined, openly accessible criteria, and, importantly, is open to review.⁷⁹ The OECD supports similar trends towards proceduralisation in the governance of regulators for making, operating and reviewing regulation.⁸⁰

3. Structuring discretion across regulatory regimes in the regulatory capitalist era

In the previous section we noted challenges of discretion beyond state actors to include both transnational private regulators and inter-governmental bodies, as well as through the application of novel instruments. We turn in this section to consider the character of administrative discretion in regulation and the extent to which those involved in regulating have recognised these challenges. Discretion has the potential to affect both the legitimacy and the effectiveness of regulatory action. We show how these challenges have been addressed within regulatory regimes through measures which tend towards structuring the discretion inherent in contemporary regulatory capitalism.

(a) Structuring discretion in regulatory regimes

Scholarship which focuses on the challenges of regulation frequently has a strong focus on regulatory agencies, as delegation to agencies tends to deviate from norms that require executive power to be exercised by elected government.⁸¹ However, the recognition of regulatory capitalism, that regulatory modes of governing engage a wider range of actors and relationships, calls for a broader focus on regulatory regimes, comprising the actors, instruments, and norms for the application of regulatory power.

Thinking about regulation as a form of social ordering or control, it is helpful to break down the components into three elements:⁸²

1. (1) the setting of norms, standards or rules;

2. (2) feedback, monitoring or detection to discover the extent to which the targeted actors in compliance with (1);

3. (3) actions to correct behaviour deviating from (1), detected in (2), which may include formal enforcement or less formal mechanisms to promote changed behaviour.

A key observation of contemporary regulatory scholarship is that these three components are not necessarily located in single organisations. Rulemaking may be the responsibility of elected parliaments, with monitoring carried out by specialised agencies and formal enforcement reserved to courts. In practice, such neat compartmentalisation is problematic and these three core regulatory tasks are liable to be the overlapping responsibilities of several organisations which together may be characterised as regulatory regimes.⁸³ It is nevertheless helpful to keep them distinct for heuristic purposes: (a) to understand better what challenges for discretion they are generating; and (b) how such challenges are addressed through structuring of discretion. Some forms of regulatory activity are not at all well captured within this heuristic, notably design- or architecture-based interventions, of which nudges provide a key example, where a single act of changing the context of choice for an individual will frequently encompass the assertion of the norm that one choice is better than another and also be self-enforcing, thus comprising all the elements of regulation in one act. Furthermore, in our research and our discussion we have not found significant examples of the kind of discretion that is exercised with a nudge being subject to structuring (hence it is an empty set in the discussion which follows).

The structuring of regulatory discretion occurs within these components, in part externally to the regulatory bodies, and in part as an aspect of their own administrative processes. Key examples of the external aspect can be found in the promulgation of general administrative and legal rules governing the exercise of discretionary power, decisions of courts engaged in reviewing administrative decisions and which set down norms for the future,⁸⁴ and also the activities of oversight agencies such as supreme audit institutions and ombudsmen which, through their general reporting work, set down procedures, for example for demonstrating value for money or for handling complaints.⁸⁵

Turning to the internal dimension of the structuring of discretion, it has been noted that technological change has tended to support the routinisation of decisions and the stripping out of discretion to some degree.⁸⁶ The effects of such routinisation may have greater impact in welfare decisions. Yet also in other regulatory activities they are rising, even though with the latter it is more challenging to strip out individualised judgements about how to proceed in a particular instance.

We have discussed regulatory capitalism as arising within both state and private regulatory regimes and in respect of both the setting of norms and the monitoring and enforcement of norms. Turning to address in more detail how this affects the structuring of discretion in practice, we have distinguished external or meta-forms of control, which structure regulatory discretion, from the structuring of discretion, which is led and managed internally and might be thought of as self-control. Bringing these components together, we have created a six by two matrix, with examples which are amplified in the text that follows.

Table 1. structuring and limiting discretion under regulatory capitalism

This matrix captures the ways in which discretion is structured in a regulatory capitalist environment. As outlined above, structuring occurs externally and internally, as well as targeting state and non-state actors.⁸⁷ It is important to note that although this matrix captures the structuring of discretion in different contexts and on different levels, it fails to account for the relationships that emerge between these levels. Thus, as the examples below will show, we can find cases where private actors are provided with discretion in implementing regulation but there is external control by the public regulator. Private regulators might also align their processes with international standards set by other state or non-state actors, as can be seen in the context of GlobalGAP certification and ISO conformity assessment for certifying bodies.⁸⁸

(b) Structuring discretion – examples from regulatory practice

(i) Standard setting

Within many parliamentary systems of government, rule making is the quintessential function reserved to parliament to ensure the linkage between elected politicians and key decision making. This notwithstanding, it is common to delegate rule making powers to ministers. In some presidential systems, such as that of the US, there is greater willingness to delegate rule making functions to independent executive agencies such as regulators. Beyond national governments, rules are made by intergovernmental organisations (of which the most developed are the institutions of the EU) and by private bodies, including firms, self-regulatory organisations, NGOs, and technical standards bodies.

The discretion over rule making has increasingly been regarded as challenging to the legitimacy of those with power or capacity to make rules. This sense has resulted in a progressively higher degree of proceduralisation around rule making, early examples of which include the US Administrative Procedure Act.

In the UK, a key instrument for structuring discretion around setting standards and rules are principles on consultation. Whilst not very prescriptive, the principles encourage extensive engagement:

Consider whether informal iterative consultation is appropriate, using new digital tools and open, collaborative approaches. Consultation is not just about formal documents and responses. It is an on-going process.

The principles support sharing details of who has been consulted and promptly publishing the response to the consultation (within 12 weeks).⁸⁹ The EU similarly places considerable emphasis on consultation on rule making, with the new Road Map approach in the 2015 Better Regulation package.⁹⁰ Many regulators go well beyond minimum standards for consultations, developing roadshows to bring their consultations out to the community more generally, publishing comments so that participants can see what others say. These practices constitute both a support for good decision making but also a constraint on the discretion to make decisions.

The UK Communications Regulator, Ofcom, for example, publishes its own consultation principles and, includes a statement on how it publishes and responds to comments:

We think it is important that everyone who is interested in an issue can see other people's views, so we usually publish all the responses on our website as soon as we receive them. After the consultation we will make our decisions and publish a statement explaining what we are going to do, and why, showing how respondents’ views helped to shape these decisions.⁹¹

These practices are not as extensive as those operated by one of Ofcom's predecessor organisation, Oftel, which used to build in time for stakeholders to comment on the published comments of other participants – comments on comments – an exemplar of good practice.⁹² It has been noted that the UK regulators tended to develop engagement practices which were more extensive than those of government partners, perhaps conscious of the need to bolster legitimacy through transparent and structured exercises of discretion.⁹³ More generally there has been a trend towards consulting on strategic plans of regulators which both aids transparency and tends to constrain regulators to doing what they said they would do, following certain priorities, developing and deploying certain instruments and so on.

Increasingly new technologies are applied to inform, structure or even alleviate administrative decision making and discretion. One example for automated rule making is the aforementioned Los Angeles traffic control system, where the Automated Traffic Surveillance and Control Center automatically adjusts traffic signalling based on real time traffic measurements.⁹⁴ As mentioned earlier, the interplay of the use of new technologies with discretion is twofold. On the one hand, new technologies and especially automated decision making may limit or structure discretion. On the other hand, there is a question regarding the degree of discretion a regulator should have in applying such technologies. Here we can observe the proliferation of guidelines that inform the public sector on whether and how they can implement AI systems in their organisational processes. An example is the UK Office for Artificial Intelligence's Guide to using artificial intelligence in the public sector, which was briefly introduced above.⁹⁵ The Guide provides information on when the use of AI is the right solution for a public sector organisation as well as advice on the implementation and management of such technologies, and finally, information on ethics and safety. It has been noted that academic thinking is increasingly driving the concern to regulate both public and private sector actors for ethical use of AI, at both national and supranational level.⁹⁶ The EU is currently working on a comprehensive approach to address AI and new technologies both within the private and the public sector.⁹⁷ Furthermore, both ISO/IEC as well as the Institute of Electrical and Electronics Engineers (IEEE) are developing extensive standardisation specifically with regard to new technologies applying AI.⁹⁸ Though these standards are not directly targeting the public sector and will most likely see initial uptake by industry actors, they nonetheless contribute to a more general framework that delimits the application of AI in decision making processes. What these and other approaches have in common is that they provide a basis for structuring the use of automated decision making which may well inform the application of such technologies in discretionary processes.

Regarding new actors, we found that for international organisations, the Global Administrative Law project has tracked the increased tendency to promote participation, to give reasons for decisions, and other forms as ways in which discretion is structured. We see similar trends in private standard setting bodies. Particularly worth mentioning in this regard is the WTO's TBT Annex 3,⁹⁹ which has been complemented by the ISO's Code of Good Practice for Standardisation. This Code is used by (particularly industry related) standard setting bodies and prescribes procedures of good practice for standard setting, such as participation of concerned stakeholders.¹⁰⁰ For private actors engaged in sustainability regulation, the ISEAL Alliance, mentioned earlier, provides a Standard Setting Code, which outlines a comprehensive set of compliance criteria for private standard setting bodies in the environmental and social issue area.¹⁰¹ An example for private internal procedures regarding standard setting can be seen in the private food safety regulator GlobalGAP's Standards Development Policy, which also focuses on stakeholder participation and included public consultation.¹⁰²

(ii) Feedback, monitoring and detection

Mechanisms for detecting deviations from regulatory norms are highly varied and include the receipt of complaints, inspections, audits, whistleblowing, and gatekeepers. The receipt of complaints involves considerable discretion for regulatory bodies in deciding whether to respond, how to respond, the extent to which they channel complaints to the complained of organisation, and the use of complaints data in wider decision making. There is a trend towards regulators setting down documentation as to how they manage such matters as inspections, audits, complaints and so on, which amplify their statutory duties and powers.

With respect to inspections, a more reflective approach has been stimulated by an official UK inquiry (Hampton),¹⁰³ and a statutory code introduced in the Legislative and Regulatory Reform Act 2006. The Regulators’ Code (2014), as it is now referred to, is essentially a meta-regulatory instrument which requires regulators to set down their approach to inspection and enforcement. The approach is close to the prescription of responsive regulation with an emphasis on encouraging regulators to promote compliance responsively rather than punitively. The Food Standards Agency, for example, publishes a statement of compliance with the Regulators’ Compliance Code which sets down in considerable detail how it manages inspections and in particular how it applies a risk-based approach to determining frequency and stringency of inspections.¹⁰⁴ Much of the emphasis of the Hampton inquiry was on reducing compliance costs, and this is reflected in the FSA Statement with a commitment to joint inspection with other agencies, where possible and appropriate, rather than multiple inspections from different agencies.

Automated processes based on machine learning are particularly relevant in structuring discretion in the monitoring context. Thus, regulators either use technology directly for monitoring purposes or as a way to determine the probability of a violation and thus as assistance to decide on the allocation of scarce resources for inspection. An example for the first case is the Securities and Exchange Commission's (SEC) MIDAS (Market Information Data Analytics System) which monitors the stock market by collecting about ‘about 1 billion records from the proprietary feeds of each of the 13 national equity exchanges time-stamped to the microsecond’. This approach allows the SEC to analyse thousands of stocks over periods between six months and one year, ‘involving 100 billion records at a time’.¹⁰⁵ Coglianese and Lehr predict that this technology could eventually be used to detect and automatically stop insider trading activities.¹⁰⁶ In such a scenario, administrative discretion on where to allocate inspection resources becomes superfluous as technology provides a way to overcome resource constraints. Examples for the second case include the New York system, mentioned earlier, which uses automated decision making to determine where to send building inspectors next;¹⁰⁷or the Internal Revenue Service in the US, which uses automation processes to predict tax fraud and thus decide where to allocate resources for review.¹⁰⁸ In The Netherlands The Hague District Court recently decided that the so-called Systeem Risico Indicatie (SyRI) legislation, quickly introduced above, is breaching the European Convention on Human Rights. SyRI is an algorithm put in place to help the Dutch government detect several types of fraud such as social benefits or tax fraud.¹⁰⁹

These are just a few examples for the application of new technologies in the context of monitoring compliance with regulation. Given the limited resources that the public sector has to dedicate to monitoring, technology is used to inform and structure regulatory action. It thus limits, and potentially even replaces discretionary processes.

Other forms of feedback to regulators, beyond these techniques, as mentioned, include the receipt of complaints, whistleblowing and reporting by gatekeepers such as banks and professional services firms. With respect to complaints, the trend has been towards using complaints processes proactively to encourage regulatees in both public and private sectors to develop their own effective complaint handling and, in the best systems, to use complaints data to enhance how they perform. There is wide discretion both with regulators and regulatees to decide at what level to use complaints data, treating complaints as matters to be disposed of quickly and cheaply, or conceiving of the data as wider information about the performance of the system as a whole. The discretion to ignore whistle-blowers has been a major source of challenge for public and private bodies as well as for whistle-blowers themselves. There are more systematic protocols around the treatment of information provided by gatekeepers such as banks and professional services firms, notably in respect of money laundering regulations.

With regard to private norm systems, we can observe the professionalisation of monitoring, often through the use of certification or auditing guidelines and requirements. Thus, the ISEAL Alliance has developed an Assurance Code which sets down requirements for certification bodies and their operations.¹¹⁰ Similarly, the ISO has developed requirements for certification bodies.¹¹¹ Thus, certification bodies need to ensure impartiality and confidentiality as outlined in these (external) documents. They furthermore need to implement appeals procedures and ensure proper qualification of personnel regarding terminology, metrics and other area-specific requirements.¹¹² An internal procedure can be found with GlobalGAP, which sets standards for agricultural practices. Those standards are monitored by certification bodies, that are under close scrutiny of GlobalGAP based on its General Regulations.¹¹³ GlobalGAP furthermore developed the Certification Integrity Program (CIPRO), which monitors and assesses GlobalGAP approved certification bodies, to ensure that they operate in line with GlobalGAP's standards and guidelines.¹¹⁴

(iii) Implementation, corrective behaviour, and enforcement

We noted above that even the enforcement of criminal law by police officers involves considerable discretion as to when and how to respond to a detected infraction. This is even more true in respect of other types of regulatory enforcement. Julia Black has noted the wide range of factors shaping the scope and exercise of discretion for enforcement officials, including

the legal framework, the characterisation of the regulatees, the relational distance between regulator and regulatee, the type of firm, the nature of the breach, organisational structures and norms, personal backgrounds and attitudes, and the broader political, social and moral context in which the regulation sits.¹¹⁵

Much attention has been paid to the structuring of enforcement discretion, concerned with what we might think of as traditional regulatory enforcement. As the discussion of examples below will demonstrate, a good deal less attention has been paid to actions of regulators and others which fall outside traditional enforcement approaches. Many regulators have wide discretion to use advertising and advocacy campaigns and to promote nudges, all aimed at influencing behaviour. Where such measures require statutory powers, they are more likely to receive scrutiny. But in the absence of such statutory underpinnings such measures may be far less visible. If it is correct that such behavioural measures can be very significant in changing behaviours, as many proponents believe, the relative opaqueness of such measures from a structuring discretion perspective is a matter of concern.

In the context of enforcement, new technologies and automated decision making processes are starting to play a significant role in structuring and limiting discretion. Here particularly design-based forms of technical management are relevant. They provide a technical infrastructure which prevents regulatees from taking any action other than the one intended by the regulator.¹¹⁶ Thereby they shift discretion forward to the point of implementation of the technology. As mentioned earlier, the easiest example is a password-protected website which only allows those persons to enter that are duly registered and eligible for the service. Roger Brownsword predicts that in the near future road traffic rules maybe entirely replaced by technical management, such as scenarios where a self-driving car would automatically be slowed down once it exceeds the speed limit.¹¹⁷

Examples where discretion is informed by automated decision making processes are also on the rise with many of them causing particular controversy. One area of controversy is preventive policing, where new technologies are applied to pre-empt crime.¹¹⁸ Using large amounts of data, specific AI tools identify suspicious places and individuals.¹¹⁹ Automation is increasingly used to identify places where crimes are committed by matching furniture or noise of a crime scene picture or video to existing data that reveals the actual location.¹²⁰ Chatbots may also be used as a way to fight crime, by identifying perpetrators engaging in grooming or webcam-based sexual exploitation of children.¹²¹ The criminal justice system also makes use of AI, for instance by determining pre-trial behaviour and thus determining bail.¹²² Automated decision making in the context of policing and criminal justice is, as mentioned, controversial, particularly because of its impact on human discretion. Though it might increase the efficiency of police work and help reduce the problematic uses of human discretion, it might also just be a way of disguising discretion behind apparent objectivity or, worse, it might even eliminate good discretion that takes into account the specific circumstances of the individual or the neighbourhood concerned.¹²³ Thus, new technologies do, and will in the future to an even larger degree, shape discretion on the enforcement level. What it does not do is eliminate the concerns that arise with such discretion.

Beyond these cutting-edge forms, governments also still apply traditional ways to manage discretion in enforcement. Key examples are provided by the UK measures to regulate enforcement practices first through a soft law instrument, the Enforcement Concordat (1998) and subsequently through legislation in the Regulatory Enforcement and Sanctions Act 2008. The Enforcement Concordat set down six principles of regulatory enforcement and required enforcement officials to seek informal resolution to enforcement actions where possible and, where taking formal action, to provide written reasons for taking the action within ten days to the regulatee.¹²⁴ Black notes that the complexity of regulatory enforcement combined with the number and variety of principles may make it difficult for enforcement authorities, even where they seek to comply, to fully implement the objectives of the Enforcement Concordat. An alternative approach is found in subsequent legislation, which in respect of the new regime of civil sanctions, requires regulators to prepare and publish guidance as to how the offence is enforced.¹²⁵

1. (2) The guidance must include guidance as to—

2. (a) the sanctions (including criminal sanctions) to which a person who commits the offence may be liable,

3. (b) the action which the regulator may take to enforce the offence, whether by virtue of this Part or otherwise, and

4. (c) the circumstances in which the regulator is likely to take any such action.

This approach does not specify the exact content of the guidance to be prepared and published but rather the broad range of matters to be addressed. The Act also creates a duty on regulators to consult with those affected before publishing such guidance. In this way, the UK government has provided a model for requiring agencies to structure their discretion. Similar requirements exist under other legislation, such as the Enterprise Act 2002 and the Competition Act 1998. Thus, for example, the Competition and Markets Authority published a statement on its approach to administrative penalties in 2014.¹²⁶ This statement sets down in detail the agency's statutory powers, its objective in enforcing the statutory rules and how its approach to applying sanctions advances the objectives. It states also the factors influencing the agency in whether to impose penalties and if so in what amount.

Beyond the state, private actors also engage in structuring discretion in enforcing their own regulatory instruments, eg along supply chains.¹²⁷ For instance, the industry-based regulator GlobalGAP has developed a hierarchy of sanctions in cases of non-compliance. Thus, depending on the type of non-compliance first a warning is issued; if the non-compliance is not resolved then a product may be suspended and finally cancellation of the contractual relationship (which involves a prohibition on using the GlobalGAP trademark) can be issued in cases of fraud, lack of trust or inability to show evidence of compliance.¹²⁸ A further example is the ISEAL Assurance Code.¹²⁹ Though as outlined above, it is often easier for private regulators to enforce their regulation where they possess sufficient economic power (eg through their location in the supply chain), NGO-driven regulators are also concerned with compliance and enforcement. The Assurance Code provides rules regarding certification processes and thus covers monitoring and enforcement mechanisms. It focuses on issues such as impartiality, clear and sufficient operating procedures, or stakeholder input.¹³⁰

Conclusion

The challenge of administrative discretion has changed with transitions from welfare state to regulatory state as the key governance mode within contemporary states. Even with the enforcement of classic hierarchical regulation there is a good deal of discretion both in when and how to enforce. As we have demonstrated, a further shift towards regulatory capitalism has both broadened the scope of actors we identify as involved in regulation as well as the instruments they apply. Discretion over key aspects of regulatory activity has been addressed across public and private bodies with instruments and approaches which seek to limit, structure, consult on and, in some cases, make transparent approaches to regulatory management more generally. Within this approach, regulatory management becomes a central concern, and in particular the measures taken by regulators, public and private, to demonstrate their accountability and to bolster their legitimacy and effectiveness.¹³¹ This concern has led to a whole-of-agency approach to developing regulatory strategies and plans and engaging stakeholders in consultations in such a way that the whole discretion of the agency is embraced by the published plans. On the other hand, we also observe a trend towards applying new technologies as a way to inform and structure decision making processes or to frame regulation more generally. In these cases, there is a certain risk of loss of transparency and accountability. New technologies might often appear like black boxes to non-experts.¹³² For this reason, ongoing efforts to define and limit the application of such tools are important. There are also distinctive questions about managing discretion with respect to behavioural measures which frequently do not require rules for their underpinning. We found little evidence of attempts to structure discretion with respect to these kinds of instruments and no evidence of leaders in this practice.

In this paper we suggest the need for a renewed understanding of the significance of discretion in a regulatory capitalist context. In order to fully comprehend the exercise of administrative power we need to go beyond traditional constraints and take a holistic approach that includes the interplay of a variety of actors and instruments. Forms of regulatory structuring need to be scrutinised for their ability to facilitate and to curtail regulatory power both from external and internal perspectives. The external dimension involves legislators, courts and other oversight bodies setting down norms for structuring discretion. Equally significant, but less visible, is the internal dimension of public and private regulators managing their discretion in order to enhance their legitimacy and effectiveness in undertaking their tasks.

For courts, a more reflexive or meta-regulatory approach might seek to assess the appropriateness and effectiveness of internal procedural structuring. Courts might also take a view on the extent to which public and private regulators are bound to follow the procedures which they have set down for themselves. Put another way, this involves the courts in understanding and assessing not only the procedural norms they have set down, but also those that have been set down by actors whose behaviour they are reviewing. There is wider research to be done here. Such research might extend beyond the oversight of internal structuring of regulatory discretion, to ask also what motivates those public and private regulators who take more of a lead in structuring their discretion and with what effects on legitimacy and effectiveness.