Managing uncertainty: Principles for improved decision making

Abstract

Effective management of uncertainty can lead to better, more informed decisions. However, many decision makers and their advisers do not always face up to uncertainty, in part because there is little constructive guidance or tools available to help. This paper outlines six Uncertainty Principles to manage uncertainty.

• Face up to uncertainty

• Deconstruct the problem

• Don’t be fooled (un/intentional biases)

• Models can be helpful, but also dangerous

• Think about adaptability and resilience

• Bring people with you

These were arrived at following extensive discussions and literature reviews over a 5-year period. While this is an important topic for actuaries, the intended audience is any decision maker or advisor in any sector (public or private).

Disclaimer

The views expressed in this publication are those of invited contributors and not necessarily those of the Institute and Faculty of Actuaries. The Institute and Faculty of Actuaries do not endorse any of the views stated, nor any claims or representations made in this publication and accept no responsibility or liability to any person for loss or damage suffered as a consequence of their placing reliance upon any view, claim or representation made in this publication. The information and expressions of opinion contained in this publication are not intended to be a comprehensive study, nor to provide actuarial advice or advice of any nature and should not be treated as a substitute for specific advice concerning individual situations. Not all the material in this paper necessarily reflects the views of all of the authors. Unpaid volunteers have produced this publication to promote discussion in the public interest. You are free to redistribute it or quote from it (with acknowledgement) without further permission or payment but you should quote from the work accurately and in context.

Introduction

This report, and the working party that has produced it, arose from a specific concern that, by quoting “1 in 200” numbers for capital requirements under Solvency II, actuaries were, perhaps unwittingly, being party to giving false confidence about the financial strength of insurers.

This issue naturally led on to broader reflections on uncertainty, looking beyond the actuarial profession and even beyond the insurance industry.

There is a tendency, amongst advisors/experts, and also most decision makers, to want to impose certainty on very uncertain situations. Actuaries, for example, may be called on to quantify, or assume away, problems by setting (fixed) assumptions about how the future could evolve. This can be at the expense of rigorous consideration of factors that are not readily quantifiable or, indeed, in any way knowable.

The failure to face up to issues surrounding uncertainty is a threat to good decision making, with issues arising in a number of areas:

• The scope and nature of analysis undertaken by experts (such as actuaries); are important questions ignored because they are “too uncertain”?

• The positioning and communication of such analysis and advice; is the fact that some difficult questions are out of scope highlighted? If uncertainties were highlighted, that might lead to the issues being thought about more.

• The understanding of experts and their work by decision makers; “we paid for the advice, so let’s use it without worrying further”.

• The intelligent recognition of uncertainty when decisions are made.

So how can uncertainty be managed to make better decisions?

This report is aimed at both decision makers and their advisors. While the context for much of the thinking of members of the working party is insurance, the ideas discussed are relevant to decision making across industries and government. Consideration is given to both technical and social aspects with an emphasis on being practical and constructive. A collection of anecdotes regaling past uncertainty calamities may be interesting but would not, in itself, offer support to decision makers often facing difficult circumstances and unenviable challenges.

The report is in two parts. The first part introduces six high level Uncertainty Principles for use in supporting, making and critiquing decisions:

• Face up to uncertainty

• Deconstruct the problem

• Don’t be fooled (un/intentional biases)

• Models can be helpful, but also dangerous

• Think about adaptability and resilience

• Bring people with you

The intention is for these to be both catchy and memorable, while being immediately meaningful and useful. They are offered as an overarching framework for further research and guidance. A summary explanation is provided for each principle followed by more in depth discussion including illustration through a number of uncertainty “vignettes”.

The second part explores three insurance-related case studies, highlighting situations where there is a high level of “unknowability”. The intention is to show how uncertainty is often handled, to illustrate some challenges to professionalism and the realities of improving decision making through applying the principles:

• Managing uncertainty after being catastrophically wrong. An insurer facing key decisions following the 2017 catastrophe losses

• The dividend question. An insurer exploring options to reduce solvency capital requirements in order to free up capital to meet shareholders’ steady dividend expectations

• I disagree! Exploring strategies an advisor might consider when responding to a valuation request, with various professionalism challenges to navigate

Part 1: Uncertainty Principles

1) Face up to Uncertainty

When making decisions, we are often confronted by uncertainty: the inherent unpredictability in future outcomes; a lack of information about the dynamics of the problem at hand; often compounded by the unknown, or unknowable.

Yet people crave certainty in what is a highly uncertain world. In practice, there is a tendency for people to downplay or ignore uncertainty when deciding on a course of action. Or, more dangerously, there is a tendency to impose certainty on a situation where there is little or none.

So, why focus on uncertainty when it is only the combination of luck and time that will tell if a particular course of action is the “better” one? We believe a “better” decision is one that is made with everyone’s eyes open – so that there is a more realistic and informed view of the degree of uncertainty in any situation, the potential impacts (where known) of these uncertainties crystallising and any mitigating actions (taken or not taken) to manage uncertainty within acceptable levels.

Facing up to uncertainty leads to more informed and better decision making. To help do this, we offer guidance through five further principles:

• Deconstruct the problem

• Don’t be fooled (un/intentional biases)

• Models can be helpful, but also dangerous

• Think about adaptability and resilience

• Bring people with you

There is no single neat answer, but these principles offer practical and constructive ideas to help decision making in the face of uncertainty. Where uncertainty cannot be mitigated, the use of the principles will help ensure decisions are made with your eyes wide open and the eyes of others.

Facing up to uncertainty is at the heart of this paper. The aim of this principle is to encourage us all to “tune in” to uncertainty, with all its messiness and unpredictability, and in spite of our deeper instincts to turn away.

In order to be able to face up to uncertainty, you need to be confident that you have a strategy to deal with it. For this reason, in this section focusing on the first of our six principles, we introduce the other five before going on to discuss each of them. We would regard the first and the last of our principles: “face up to uncertainty” and “take people with you” as the most important of the six.

A degree of uncertainty is present in any decision. As is detailed in the Deconstruct the problem section, uncertainty can take a number of forms: inherent randomness, lack of knowledge, modelling limitations, ambiguity, errors, people factors and broader social and ethical factors. It is important to appreciate that where there is uncertainty there may be opportunity, just as much as there are risks.

Recent history shows us that uncertainty is a game with high stakes, and that the costs on society of failing to “face up to” and better manage uncertainty are high.

Examples include:

• The global financial crisis of 2007–2008: in the 10 years up to 2006 US banks had delivered average returns of over 13% RoE; and yet, despite ever more sophisticated approaches to managing financial risks (including use of experts and financial models), the complexity within the system had obscured underlying exposures to loan defaults. See inset (Katzenstein, 2014)

• Climate change: where global efforts to slow increasing temperatures and reduce the build up of greenhouse gases have not yet been successful – and uncertainty over the causes and effects of global warming has created a vacuum in which the risks and potential effects can be downplayed or denied

• Nokia and Kodak: companies which, in the face of rapidly evolving technology, went from market leaders to potential insolvency in a short period of time

• Deepwater Horizon in 2010: a disaster involving the loss of 11 lives and large complex pollution claims, with decision making (before and during) challenged by the inherent uncertainty of deep water drilling

• The near collapse of Lloyd’s of London in the late 1980s/early 1990s: unmanageable exposure complexity (exacerbated by the “Excess of Loss Spiral”) combined with under anticipated losses (catastrophic events such as Piper Alpha and longer term asbestos and pollution claims)

Case study: Financial Crisis of 2007–2008: The near collapse of the American financial system in 2008 wiped out more than $11 trillion in household wealth. The peak-to-trough decline in real GDP was 4¼% and the decline in payroll employment even larger at 6.3%. The global financial crisis is considered by many economists to have been the worst financial crisis since the Great Depression and reshaped the world of finance and investment banking. In hindsight, it is clear the crisis had multiple causes but that at its root was a systematic failure to “face up to” the uncertainties inherent within the financial system and failure to build resilience into the system to protect against these.

The years before the crisis saw a flood of lending to “subprime” borrowers with poor credit histories, who subsequently defaulted on their loans, creating unprecedented defaults across the wider US and global economy. Financial institutions had become highly leveraged, reducing their resilience to losses of this scale. Much of the leverage was achieved using complex financial instruments such as off-balance sheet securitisation and derivatives, particularly credit default obligations (CDOs), which acted to repackage loans and pass risk on to third parties. Additional complexity had made it difficult to monitor and reduce financial risk at an institution and industry wide level.

“Elegant” (sic) financial models used to price CDOs were found to have been based on historic data from periods of low credit risk, as well as highly material and sensitive assumptions about the relationships between different credit risks. In the financial industry and academia, whilst a few had realised that the assumptions underlying these models were unrealistic, they were unsuccessful in highlighting the risks to those who could act. Warning signs were in general disregarded by industry practitioners who were enjoying profitable returns, and who wanted to retain market access. Regulators and rating agencies bore some responsibility too, for failing to keep economic imbalances in check and for failing to exercise proper oversight of financial institutions.

Focusing on human behaviours around uncertainty helps us understand how intelligent people, in an environment of copious information and complex analysis, adhered to social conventions that led them to believe that they were making decisions in a world of risk, when, in reality, those very conventions led them to the cliff’s edge.

Sources:

1. [1] https://pdfs.semanticscholar.org/d6a2/26e54cff14f38df986f09773b063629602a5.pdf

2. [2] https://en.wikipedia.org/wiki/Financial_crisis_of_2007%E2%80%932008

3. [3] https://media.economist.com/sites/default/files/pdfs/store/Managing_Uncertainty.pdf

4. [4] http://faculty.wcas.northwestern.edu/~scn407/documents/NelsonandKatzensteinIOApril2014.pdf

5. [5] https://www.princeton.edu/ceps/workingpapers/243blinder.pdf

6. [6] https://mitsloan.mit.edu/LearningEdge/CaseDocs/09-093%20The%20Financial%20Crisis%20of%202008.Rev.pdf

7. [7] https://www.ifw-members.ifw-kiel.de/publications/the-financial-crisis-and-the-systemic-failure-of-academic-economics/KWP_1489_ColanderetalFinancial%20Crisis.pdf

8. [8] https://academic.oup.com/cje/article/33/4/563/1730705

There are many examples where uncertainty is managed well, albeit such cases naturally attract less attention. These underpin the thinking behind the proposed principles, which are highlighted throughout the paper.

It is worth noting that uncertainty is present as much on a personal, as a corporate or national level. You may have grappled with the decisions involved with buying a house and experienced how hard it is to do so without a full understanding of your current and future requirements, the motivations of the seller or having all of the facts to hand. The potential for financial (and emotional) gain can be significant and so is the potential for loss. Whilst additional research (e.g. a surveyor, talking with neighbours, preparing an analysis) might reduce some of the uncertainty, it will not remove it completely, nor will it always bring uncertainty within an acceptable level.

1.1 Why is it so difficult to “face up to” uncertainty?

The following elements combine to make overlooking uncertainty such a common failing:

1. 1. We (and the people around us) are biologically programmed to seek certainty with our brains constantly trying to apply memories or experience to predict what will happen next.

   These elements are driven by the notion that our brains are “pattern-recognition machines”. To solve problems our natural instinct is to try to apply memories or experience to predict what happens next:

   “The brain is a pattern-recognition machine that is constantly trying to predict the near future. To pick up a cup of coffee, the sensory system, sensing the position of the fingers at each moment, interacts dynamically with the motor cortex to determine where to move your fingers next. …. If it feels different, perhaps slippery, you immediately pay attention.

   Even a small amount of uncertainty generates an “error” response in the orbital frontal cortex (OFC). This takes attention away from one’s goals, forcing attention to the error. This is like having a flashing printer icon on your desktop when paper is jammed – the flashing cannot be ignored, and until it is resolved it is difficult to focus on other things. Larger uncertainties, like not knowing your boss’s expectations or if your job is secure, can be highly debilitating.” (Rock, 2008)

2. 2. There may often be some urgency to make a decision; so patience to entertain issues which are not readily resolvable will be limited. It is also possible that the more serious consequences of overlooking uncertainty may have a relatively low likelihood of occurring. A decision maker down playing uncertainty will often be lucky. Short-term reward or personal risk are not necessarily aligned to effective uncertainty management.

   Thinking about uncertainty and taking any sort of practical action is inviting additional workload and hassle, and can also delay matters – it is not the easy route. This is often exacerbated by little support from others or quite possibly active resistance.

3. 3. Uncertainty is messy and difficult; we do not know what to do about it.

1.2 Uncertainty and decision making

In financial textbooks and applications, there is often the implicit or explicit assumption that risks are known and can be modelled. Those risks that are unknown or unquantifiable often do not receive the proper attention of the decision maker, who may be relying on the model to inform the decision and who may well ignore any caveats or limitations presented on the analysis.

There is the further challenge that there can be a tendency to analyse or model a situation without being fully clear on the purpose. In particular, what is the scope of the problem and what decisions are under consideration?

The following thought process illustrates the challenge.

Is the problem well defined?

Firstly, consider the clarity of the question or problem that you are trying to solve (see Figure 1). It is essential to understand this as clearly as possible upfront otherwise this will lead to additional uncertainty later.

How much of the problem can be quantified?

The next step is to assess how much of the problem can be looked at using modelling techniques. It may be that there is a sufficient quantity of data and understanding of the dynamics of the system that the problem may be considered a “modelling challenge”. But for many real-world problems, one should recognise substantial additional uncertainty that cannot be readily captured by a model and instead turn to a different toolkit to analyse the problem. We refer to this as an uncertainty challenge, where the focus should be on adaptability and understanding the constraints of what is known about the problem (see Figure 2).

Figure 1. Question clarity.

Figure 2. The scope of quantification.

In practice, most problems exist between these extremes, and a consideration of where they fall is a helpful first step in working through any analysis.

It is also important to check whether the expert analysing the problem and the decision maker agree on this perspective, which will help later on (see Figure 3). Aligning perspectives is something that will be discussed further in the Bring people with you principle.

Figure 3. Perspectives on question clarity and quantifiability.

1.3 Introducing six key principles for “better” decision making

Our work has identified six key principles for managing uncertainty in any decision and supporting tools in each area. These have been developed primarily with experiences of decision making within the insurance industry in mind, but which are in our view sufficiently generic to apply to other situations.

We believe that to successfully drive improvements in decision making, these principles are relevant to all parties involved in the decision: experts as much as non-experts and decision makers as much as advisors.

Having faced up to uncertainty (#1), it is beneficial in our experience to take a broader view – to deconstruct the problem (#2) by considering the way the question itself has been framed, the dynamics and motivations at play, the stakeholders involved and the potential consequences or knock-on impacts of any decision.

In the absence of certainty, or facts and data, biases can become more powerful (#3, Don’t be fooled). Think of the forceful CEO pushing through a “buy” price higher than the analyst’s valuation; it is easier for the analyst to give way when there is a lack of information about the target company, than when he/she has the full facts and figures to play.

We put forward the idea that models can be helpful, but also dangerous (#4). It is true that much of traditional financial economics favours modelling of quantifiable risks over a better understanding of the unknown, unmodellable and human behaviour.

We do not use a complex model to catch a ball (or famously, a frisbee (Haldane, 2012)); our minds automatically interpret the flight and speed against the background and use prior experience to project where it would land. We do, however, fly in aeroplanes designed, developed and piloted using many models (not least, Newton’s equation for universal gravitation).

Experience, judgement, rules of thumb (“heuristics”) and modelling/analytics are all central to decision making under uncertainty. However, it is also important to have thought through the situations where these work well and vitally where they might break down.

As is ensuring that you control potential downsides and consider the value of optionality within any decision strategy – a principle this paper refers to as “Think about adaptability and resilience” (#5). Finally, “Bringing people with you” (#6) is a pre-requisite for bringing about any change.

So, in what type of situations are these principles useful?

Uncertainty in practice

Uncertainty is a fact of life – and it is not just outcomes that are uncertain. So, often there is uncertainty around the question being asked, the context of the issue being touched on and the dynamics of the problem at hand.

When making an offer on a house, it is not only future house prices that are unknown – so is the rationale behind the seller putting the house up for sale, the extent of work/repairs required, what you and your family value in a home, how other people might value the property in future (proximity to schools, parks, new transport links), your ongoing ability to make mortgage payments, as well as future economic conditions. On a personal level, you may have grappled with the decisions involved with buying a house and experienced how hard it is to do so without a full understanding of the process or having all of the facts to hand. Whilst additional research (e.g. a survey, talking with neighbours, preparing a spreadsheet analysis) might reduce some of the uncertainty, it will not remove it completely, nor will it always bring uncertainty within an acceptable level. Indeed, a degree of uncertainty is present in any decision.

Something as apparently simple as booking a holiday can be fraught with uncertainty about the true service standards of a hotel or the reliability of website reviews. There will be information available that can be used, but often a clear recognition and consideration of what is unknown can be just as valuable in making a good decision.

A “better” approach to decision making

So why focus on “uncertainty” when it is only the combination of luck and time that will tell if a particular course of action (in this case, the offer made on the house) is the “better” one? There are two key objectives of our work:

1. 1. To promote “better” decision making where there is uncertainty and provide a toolkit for making “better” decisions;

2. 2. To drive awareness and management of adverse outcomes, where uncertainties crystallise.

We believe a “better” decision is one that is made with eyes open – to the degree of uncertainty in any situation, the potential impacts (where known) of these uncertainties crystallising and any mitigating actions to manage uncertainty within acceptable levels for that organisation.

2) Deconstruct the Problem

Some decisions involve situations that are clearly complex with wide ranging uncertainties. Others can appear deceptively straightforward and mislead a decision maker into overlooking important perspectives, assumptions and uncertainties. In both situations, more constructive insights can be achieved from deconstructing the problem into more manageable elements.

Simply breaking a problem into parts can be very helpful (so long as this is not at the expense of also considering the whole). However, three particular deconstruction perspectives are worth considering:

• The decision-making process: the context and framing of the question; the decision support analysis and modelling and the results communication and interpretation

• The decision stakeholders: the different people involved with, or impacted by, the decision

• The assumptions (explicit and implicit) and types of uncertainty involved with the problem and the decision analysis. A structured framework is suggested to support this type of analysis

These perspectives are brought together with the Assumption and Uncertainty Onion.

The other uncertainty principles can be used to provide insights and inform strategies for both the problem as a whole and its component parts.

When looking at a specific decision, there are often uncertainties around the issue itself as well as further uncertainties around the process of analysing the problem.

A starting point may be to look at the different elements of a problem. For example, if buying a house you might consider your requirements (neighbourhood, size, utilities, transport, schools, etc.) and the financing (deposit, mortgage, bills, stamp duty, cost of improvements). Or a government considering a new infrastructure project may look separately at the costs, direct benefits, broader economic benefits, technical challenges and political challenges. However, some of the most insightful, but less immediately obvious issues, can be uncovered by looking at the decision-making process; the people involved and the different underlying assumptions and types of uncertainty.

2.1 The decision-making process

Given our focus on decision making, the deconstruction of the decision-making process is helpful:

1. Framing

What is the question and its context and are both understood properly? The other Uncertainty Principles can provide helpful prompts to provide a further deconstruction:

• Don’t be fooled: What is the motivation for the question being asked? Are there any known or potential biases? What are the information and understanding gaps (questioner versus advisor)?

• Models can be helpful, but also dangerous: Is the question amenable to analysis? What are the expectations regarding how modelling might be used and are these informed and realistic?

• Bring people with you: to what extent is there a shared perspective on the framing of the question, including expectations over how a subsequent decision will be influenced?

As part of the framing, it is also important to consider the broader background, both specific to the actual question and wider “big picture” issues.

2. Analysis and modelling

Is the proposed work understood, both in terms of the approach and the key uncertainties or limitations? The Models can be helpful, but also dangerous and Think about resilience principles offer further assistance here, as does the deconstruction of assumptions and uncertainty types discussed later in this section.

3. Results communication and interpretation

What are the results and how should they be interpreted (in light of the question and analysis)? Communication and appropriate understanding highlight the importance of the last principle, Bring people with you.

There is also the risk of misinterpretation of results which is covered in more detail with the Don’t be fooled principle

2.2 The decision stakeholders

It is useful to consider the different individuals, groups and organisations with a connection to the decision:

• Who will directly influence the decision maker?

• Who has opinions or expertise that can provide support to the decision maker?

• Who will be impacted by the decision?

Understanding all these perspectives helps develop a richer understanding of a problem leading to more informed decision making. The Bring people with you and Don’t be fooled principles both offer guidance to help with this.

It is possible to use simple plots to identify and prioritise attention on relevant stakeholders and unknowns. For stakeholders, the relevant dimensions are “power/influence” and “stake” (respectively, how much that party can affect what happens, and how much they are affected by what happens) – see Figure 4.

Figure 4. Stakeholder identification (French, Maule, & Papamichail, 2009).

2.3 Assumptions and types of uncertainty

One feature of poor decision making is a lack of rigour in distinguishing between fact and assumptions and the tendency to overlook some assumptions entirely. A particular problem arises with implicit assumptions, those which everyone takes for granted as being both valid and shared by all parties. All too often people are not on the same page, despite everyone’s initial instinctive belief that they are. Such assumptions may only come to light later as the analysis and decision-making process progress, causing frustration and ill-informed views on the most appropriate way forward.

A related but different issue is the challenge in understanding where uncertainties exist when considering a particular problem and decision. The University of Chicago economist Frank Knight (1885–1972) distinguished between risk and uncertainty in his 1921 work Risk, Uncertainty and Profit (Knightian Uncertainty) (Knight, 1921). In essence, he defined risk as where you know the probabilities of different outcomes and uncertainty as where you do not. Some things are fully known, e.g. probabilities of outcomes from throwing an unloaded dice, while others are, to all intents and purposes, unknowable (or at least unknowable in the timeframe of the decision). In practice, there are many variations and levels of uncertainty in between which in turn require different approaches to management.

It is possible and useful to deconstruct the assumptions and types of uncertainty in a problem separately. However, greater value is possible by looking at the two together.

Assumptions

People have a natural tendency to focus on outcomes. This can mean important assumptions regarding the starting position are overlooked. There may also be a lack of rigour in considering whether a given approach will be effective in leading to the desired outcome.

In deconstructing assumptions, it is suggested a problem can be looked at as a journey:

1. 1. Where are we now? (starting point)

2. 2. Where do we want to get to? (end point)

3. 3. How are we going to get there? (effectiveness of the solution to get to the end point)

In all cases, particular attention should be paid to identifying implicit assumptions. What is being assumed to be true that is not being discussed and has the potential to undermine the validity of a decision?

Types of uncertainty

Uncertainty is a broadly defined and applied term; different types require appropriate responses. Merton (1936) considered this aspect as early as 1936 and, incorporating a number of other perspectives and influences (French n.d.) ¹ , we offer the following categorisation of types of uncertainty:

• Limited knowledge due to the unpredictability of fortuitous outcomes. This is also referred to as stochastic or aleatory variability (or in layman’s terms, randomness)

• Limited knowledge due to ignorance. This may arise where there is inadequate information or where something is effectively unknowable. This is also referred to as epistemic (or knowledge) uncertainty

• Modelling limitations, including compromises in capturing the reality of a real-life system (both model design and parameterisation judgements)

• Ambiguity, where lack of clarity over the problem framing and analysis assumptions may lead to unintended consequences

• Errors and other operational uncertainty. These can occur at any stage and take many forms, for example, the use of incorrect data, a flaw in a computational process or a typographical mistake

• People uncertainty, arising from a lack of awareness of possible hidden agenda or unintentional biases. These are discussed in the Don’t be fooled principle section

• Social and ethical uncertainty. Linked to people uncertainty but taking a broad societal perspective. Examples include self-fulfilling and self-negating prophecies – in which public statements change social views so substantially that the prophecy is no longer valid. Or basic values – in which the realisation of one’s fundamental values is so important as to preclude any consideration of the consequences

Where a model is known to have limitations this might be labelled as “uncertainty” (and is included in the above list) but is in fact deliberate and perhaps may be considered as relating more to “assumptions”. Similarly ambiguity is entirely avoidable. This overlap with assumptions illustrates why there are benefits in looking at assumptions and types of uncertainty together.

The above does not differentiate between degrees of uncertainty and this is also a valid and important potential deconstruction. The benefit of using complex models to help inform optimal strategies is greater in risk (known probability) situations. Conversely the relative simplicity of certain heuristics (rules of thumb, intuitive judgement, etc.) will come to the fore insituations with greater uncertainty and unknowability. This is discussed further with the Models can be helpful, but also dangerous principle.

Combining assumptions and types of uncertainty

The inter-connection between assumptions and types of uncertainty suggests there is value in considering both together. This is illustrated below (Figure 5), where assumptions can be identified and their uncertainty categorised:

Figure 5. Assumptions and types of uncertainty.

For example, we might consider moving house, as discussed in the Face up to uncertainty principle:

• Unpacking assumptions

  • ○ Where are you now? Existing space adequacy, current location suitability, current house value (if applicable), current finances, career prospects

  • ○ Where do you want to go? Perceived requirements (space, location criteria, etc.), time constraints, financial position (including career progression), future flexibility?

  • ○ How are we going to get there?

    • ▪ Property availability (location, size, features, price, etc.)

    • ▪ Selling existing property (demand, price, pre-sale improvements?)

    • ▪ Raising deposit and mortgage financing

    • ▪ Research (reviewing areas, surveyor’s report, improvement costs, bills, etc.)

    • ▪ Financial analysis (spreadsheet of costs and future finances)

    • ▪ Relationship management: estate agent, buyer(s), seller, parents/family?

• Categorising uncertainties

  • ○ Limited knowledge due to the unpredictability: future house prices, interest rates

  • ○ Limited knowledge due to ignorance: hidden problems (partly mitigated through survey, search, etc.), future space needs, future job changes

  • ○ Modelling limitations (financial analysis): sale price, buying price, cost of essential or desirable improvements, future bills, earnings, interest rates

  • ○ Ambiguity: how clear are your criteria?

  • ○ Errors: overlooking costs, spreadsheet analysis errors

  • ○ People: buyer, seller, agent, family

  • ○ Social and ethical: crime levels, environmental factors

It is of course possible to create significant complexity through such an approach. However, a rigorous deconstruction can identify key assumptions and uncertainties that may otherwise by missed, or at least not be properly understood or managed.

2.4 The assumption and uncertainty onion

To help combine the decision process, stakeholder, assumption and uncertainty type deconstructions, we suggest the use of a vegetable analogy: the Assumption and Uncertainty Onion (Figure 6).

Figure 6. The assumption and uncertainty onion.

Consider a problem as an onion with the outer layers containing information regarding the context and key aspects of the approach to be taken to analyse the problem. The inner layers contain more technical detail which either requires specialist expertise or is simply less important than information in the outer layers. A key point about the onion analogy is that many assumptions are likely to be below ground and hidden from the non-inquiring mind. By deconstructing the problem it becomes easier to identify the implicit assumptions and highlight those that are sufficiently important that they belong in the outer layers.

Engagement with the outer layers is essential for everyone (see the Bring people with you principle). A decision maker not prepared to contribute to the outer layer activities is unreasonably abrogating responsibility to others and cannot expect to make well-informed decisions. Likewise “experts” need to do their part by helping identify the outer layer information for a given problem, and do their part to make it as relevant and comprehensible as possible.

3) Don’t Be Fooled (Un/Intentional Biases)

Things are not always as they seem. Without careful thought it is easy to find decisions (or support for decisions) being corrupted by deliberate agendas or a range of unintentional biases. Understanding where a decision maker or an advisor can be fooled is essential to reducing human-derived uncertainty. The subsequent communication strategy for each party should be adapted accordingly, recognising both perspectives.

Real-life situations often involve the questioner and expert having different knowledge and perspectives. An open declaration of all information and issues may be encouraged from a formal professional perspective, but may not be an optimal nor reasonable approach for one or both parties. In some situations, understanding “the game” is important so as not to be fooled, but also to ensure intelligent practical application of professional standards.

Unintentional biases can also be powerful corrupters of good decision making. The awareness of the manner in which our brains are designed to instinctively take decisions helps us to identify common biases and traps that can undermine the quality of these decisions. We categorise some of these cognitive effects and suggest some of the ways in which we can manage our approach to avoid the dangers presented.

This principle covers two different but important elements: the challenge of managing uncertainty where there are deliberate efforts made to mislead; and the wide ranging ways unconscious bias can corrupt well intentioned decision makers and their advisors. Often (possibly always) these two factors will arise at the same time in any given situation, with the decision maker and advisor deliberately or sub-consciously biasing their judgement in order to align with expectations and agendas. For the purpose of describing and explaining them in this section, we have kept them distinct.

We first consider deliberate agendas and information asymmetries and the role of communication. In particular we highlight the importance of two-way communication and make observations on how to “play the game”.

We then look at the wide range of unintentional cognitive biases, looking at when they can arise and how awareness itself is perhaps the main technique for countering their potential harm.

It is worth noting that while professional ethical codes require an individual to act in a manner that is free from undue influences and bias, the reality is that such codes can only meaningfully be interpreted quite narrowly. The full range of potential sources of influence and bias is diverse and clearly addresses a much wider class of circumstances than envisaged under professional ethical codes.

3.1 Intentional biases and playing the game

The classical way of thinking about communication – perfecting the one-way presentation

Clear communication is often interpreted as a one-way activity, particularly in the context of professional advice. Hence, professional standards are crafted in terms of setting out information in “a clear and comprehensible manner” and examinations assess the ability to present concepts clearly to lay-readers. In the context of uncertain estimates, the need to explain probabilistic concepts in an intelligible fashion is key, particularly where probabilities are small, and underlying model and parameter choices may be significant. Hence professionals develop standardised approaches to expressing ideas, often building on precedents developed over time, and the evolution of certain terms of art.

In this case the advisor imparts information and the client ² absorbs it. The advisor wants the client to benefit from the greatest possible understanding in the most efficient fashion and the client has an objective of obtaining the advisor’s best advice given the information available. Constraints and incentives are aligned with these goals, so can be ignored as they do not distort the position. For example, the exercise is seen as a one-off, with neither sides’ actions taking into account the effect of prior nor subsequent work.

Game-theoretic approach – 2-way communication strategies

Real-life communications involve two or more parties, each with different pieces of information and with their own incentives, objectives and payoffs.

The advisor may:

• Know the implicit assumptions underlying the work

• Constrain the amount of time or effort that they wish to put into the work (e.g. because their remuneration is not linked to the amount of work done)

• Conversely seek to increase the amount of work they do in order to increase their remuneration

• Wish to defend a previously held view and avoid embarrassment to themselves

• Wish to break with a previously held view (probably of another) or store up margins for recognition in future

• Be aware of how far they might be prepared to modify their advice to accommodate or support a desired outcome by their client sponsor

The client may:

• Know the consequences of different outcomes

• Know the opinions and estimates of others, including other advisors

• Want a particular outcome to avoid embarrassment to themselves now

• Want a particular outcome to provide a cushion against potential embarrassment in future

• Know what information has not been provided to the advisor

• Have a view regarding whether a particular advisor tends to be more or less conservative with their advice; has a good or reliable reputation (based on their own experience or that of others); tends to be flexible in response to challenge

These influences may also link in with relationships between advisor and client, either at a corporate or a personal level that may be sustained over a number of years, with mental accounting of favours granted or owed providing a successive link between interactions.

In addition, further complexity may arise as there may be multiple client stakeholders, multiple advisors and potentially agents and layers of sub-ordinates to decision makers who may introduce a form of “friction” to the communications.

As a result, communications need to be seen as a form of negotiation or transaction, often broken down into a series of sub-transactions or negotiations.

Theoretically optimal strategies for 2-way communication

How then does the advisor determine the most appropriate strategy for navigating this scenario, where each party has information that the other lacks, and there may be characteristics of the behaviour of the other that are known or unknown?

While classical theory might suggest that the optimal approach is to be both predictable and transparent with the information possessed, a game-theoretic view suggests that this may not be ideal. Indeed, many experienced advisors will have experienced or observed cases where clients have taken advantage of a naïve advisor, and a few will have war stories to tell. But equally, an inappropriately cautious or aggressive approach from an advisor can prove inefficient and counterproductive.

In practice, we often observe a complex ritual during which each party seeks to use their own information advantages to elicit desired information from the other party. The advisor and client may not initially have the same concept of what is a proper request; so the initial exploratory language adopted by both sides may be deliberately ambiguous or cryptic, augmented by non-verbal signals and other indirect means. These serve to allow retractions to be made and positions altered under the guise of innocent misunderstandings.

Such a process is not unique to advisor–client relationships, but pervasive through human interactions. Psychologists have studied such behaviours in other contexts such as courtship rituals and interactions between citizens and law enforcement agencies.

Here are some observations on the suitable tactics and strategies to adopt in this complex 2-way communication environment:

1. 1. Unilateral full transparency can be a weak tactical approach for an advisor as the client can take advantage of the advisor by persistently seeking to influence advice in one direction.

2. 2. Full opacity can therefore be equally unhelpful. While it might provide an initial tactical advantage, it can undermine trust and therefore weaken the strategic position over the longer term.

3. 3. Some ambiguity can be helpful, particularly during early stages of communications, while each side builds trust with the other. Excess or persistent ambiguity however can undermine trust and frustrate meaningful progress. Too much ambiguity can confuse others to your true position, inhibiting their ability to share information, while persistent ambiguity impedes progress towards decision making and action.

4. 4. Consistency of approach over time builds trust and credibility with the counterparty, so yields strategic benefits to the relationship and the communication framework.

In summary, deliberate ambiguity or adoption of a position may have some justification and may be a practical necessity in active communication, but needs care in its application to avoid straying over the line into what might be considered unprofessional or unethical behaviour. The context of the decision, the decision support being sought and the relationship of the questioner and the advisor all need to be considered. Whether justified or not, being alert to potential intentional bias is important for all parties.

3.2 Unintentional biases and traps

There is a rich body of research covering cognitive bias. In the context of decision making and uncertainty it is helpful to categorise the different biases and heuristics (rules of thumb) into three groups (an example of another uncertainty principle “deconstruct the problem”):

• Latent framing: biases and heuristics that influence the perception of a problem and expectations of the outcome

• Traps: general biases and heuristics that can deceive the decision maker and advisor

• Over-interpretation: biases and heuristics relating to reading too much or too little into data

Daniel Kahneman’s book, Thinking Fast and Slow (Kahneman, 2011), provides a helpful introduction to this field, setting out the results of many years of behavioural research. In essence, he identifies that the human brain has a fast, instinctive decision-making capability, necessary for survival and to manage the many decisions that we must take each day. In addition, it is also capable of a much slower, deliberative and logical manner of thinking. The former, which he terms “System 1”, is always on, whereas the latter “System 2” requires effort and is often not deployed because our brain is programmed to conserve energy (or is “lazy”).

What is clear is that certain tools and techniques can be effective at countering specific biases, and to achieve this there is an over-arching theme of stimulating “System 2”. The very act of identifying the potential presence of a particular bias stimulates this more structured approach to thinking about a problem and can go some way towards mitigating the impact of these biases on a decision.

Although there are no simple routes to avoid the tricks presented by the manner in which our brains operate, an awareness of the various hazards that exist can help us to be on our guard. In a professional advisor/decision-maker relationship context, we recommend taking steps to turn on our System 2 approach to problems.

Some examples of ways of doing this might be:

• Independent challenge of work

• Forcing yourself to document and explain your thinking

• Avoiding rushing to conclusions

• Use of checklists

• Asking the counter-factual questions – what if this was wrong, or what would need to happen for this to be very wrong?

Note that some instinctive biases and heuristics (rules of thumb) can be useful tools insituations involving uncertainty and unknowability, avoiding being fooled by the spurious complexity of approaches which are only valid with greater underlying knowledge. This is discussed further with the Models can be helpful, but also dangerous principle.

We conclude this section with a brief summary of some of the common biases and traps identified in preparing the paper and recommend further reading around this topic for a more in-depth understanding. Two notable sources were Daniel Kahneman’s book Thinking Fast and Slow (Kahneman, 2011) and the 2012 Lloyd’s paper “Cognition Minding risks: Why the study of behaviour is important for the insurance industry” (Weick et al., 2012). This list is not intended to be exhaustive, but we recommend readers familiarise themselves with them. Reviewing these biases and traps when giving advice or taking decisions may provide a helpful means of avoiding them.

Latent framing

• Affect heuristic: the tendency for people to use their personal likes and dislikes to form beliefs about the world

• Anchoring: the process of using a starting point for evaluating or estimating unknown values

• Confirmation bias: tendency to seek evidence that is compatible with a given view

• Halo effect: the tendency to like (or dislike) everything about a person, including their opinions

• Myopic loss aversion: a phenomenon whereby investors are particularly concerned with the potential for a short-term loss, even in the context of long-term investments

• Trusting intuition: the tendency for people to have a lot of confidence in their intuition

• Status quo bias: the preference for things to stay the same

• Sunk cost bias: costs incurred in the past are used as a justification to continue investing in suboptimal projects or strategies in the future

• Survivor’s Curse: tendency for the lucky to survive and have misplaced optimism

Traps

• Gambler’s fallacy: the tendency of decision makers to underestimate the probability of a repetition of an event that has just happened

• Illusion of validity: the use of evidence to make confident predictions even after the predictive value of the evidence has been disproved

• Law of Least Effort: the tendency for people to seek the easiest way possible to complete a task

• Mean-reversion bias: when decision makers assume that over time, a trend has to return to the mean

• Planning myopia: the tendency to consider consequences over a too restricted time horizon.

• Priming: purposefully triggering thoughts or ideas

• Temporal discounting: the greater the delay to a future reward, the lower its present, subjective value

• Winner’s Curse: tendency for winning bidders to overpay where there is incomplete information

Over-interpretation

• As if bias: ³ the potential to be optimistic when restating historic behaviour due to exposure revisions or past misfortune

• Availability heuristic: the tendency for people to respond more strongly to risks when instances of those risks are more available to them (from memory, imagination, media, general social discourse, beliefs about the world)

• Causal thinking bias: tendency for people to seek patterns and explanations rather than believe in chance

• Hindsight bias: the false belief that events are more predictable than they actually are

• Illusion of skill: the tendency for people to mistake good luck for skill

• Small probabilities: a group of biases that can arise when people reason about rare events. Small probabilities tend to receive too much, or too little weight depending on the decision context

4) Models Can Be Helpful, but Also Dangerous

“All models are wrong, some models are useful” George E. P. Box

Where true uncertainty exists, complex models can be dangerous. Instead the use of simple models or rules of thumb in the hands of an experienced practitioner can be a better approach. Simple rules can offer an understandable and clear approximation to the problem, and the experience of the expert helps to gauge where the model will work well and where the assumptions might fall down and alternatives need to be considered.

Understanding the context of the problem and the influence a model may have on a decision is key. Models can still be useful insituations where there is significant uncertainty, or even unknowability. For example, value may still be obtained from comparing options or through a better understanding of a process or system.

Simplified models or rules of thumb will have inevitable limitations. The uncertainty can be mitigated through our next principle: Think about adaptability and resilience.

The practical challenges of working with complex models are illustrated by looking at the use of Economic Capital Models and catastrophe models.

Imagine trying to direct air traffic into Heathrow airport without the tools used to project the future movements of aircraft. There may be situations that arise where these are inadequate, but there are safety processes in place where manual intervention can take over to avoid a disaster. Or imagine trying to plan for the safe transfer of perishable goods across a large network of supermarkets without a model to predict demand and to optimise the transport routes.

Models can be very useful and allow us to do things that would otherwise not be possible.

A model might be defined as an abstraction of a system that makes appropriate simplifying assumptions for the purpose of understanding the underlying process. The output from a model, or the associated understanding of the system, may directly or indirectly influence decision making.

Taking the broadest perspective, a model can range from a complex statistical tool with a number of dependency assumptions to a simple rule of thumb or heuristic.

Models can be useful in a number of ways, for example:

• Estimation of values (or patterns), e.g. finding the “best fit” by analysing observed data

• Prediction of future values, e.g. forecasting the future claims costs for an insurance contract

• Comparison and calibration: better/worse, bigger/smaller, etc., e.g. has an insurance exposure increased or decreased since last year; how much will the claims costs of an insurance contract increase if additional coverage is included; or is company A more profitable than company B

• Optimisation: what is the preferred option for a given set of objectives and constraints

• System understanding, e.g. how changes in exchange rates will affect the financial results of a company; or how might a cyber incident impact an insurance portfolio

Given the broad way in which models can be used, sweeping statements about the usefulness (or danger) of a given model are not possible. It is vital to understand the context of a decision, the quality and appropriateness of the model and the way in which the model might influence the decision.

The use of a poor model, or the inappropriate use of a good model, can give misplaced confidence or insight to a decision maker and be dangerous. These issues are greater where there is uncertainty.

4.1 Risk versus uncertainty and the use of models

With the Deconstruct the problem principle we discussed types of uncertainty and, in particular, Knightian Uncertainty. This makes the distinction between “risk” where you know the underlying probabilities and “uncertainty” where you do not. The insights of Gerd Gigerenzer, ⁴ the renowned German psychologist, are helpful in understanding the usefulness of models.

He argues that the concept of “optimising” is only truly possible where you know the underlying probabilities, i.e. in risk situations. However in most real life situations this is not the case. Even where we have a lot of statistics we do not know the true probabilities. The further away you move along the risk-uncertainty spectrum towards complete “unknowability”, the less useful complex models become.

Gigerenzer highlights the value of heuristics in uncertain situations. Wikipedia defines heuristics as “any approach to problem solving, learning, or discovery that employs a practical method not guaranteed to be optimal or perfect, but sufficient for the immediate goal. Heuristics can be mental shortcuts that ease the cognitive load of making a decision, for example using a rule of thumb, an educated guess, an intuitive judgment, guesstimate, stereotyping, profiling, or common sense”.

Heuristics are what most people use in the real world, but this is not to say all are good. There can be bad heuristics (just as there can be bad complex models), and good heuristics used inappropriately given the context of the decision or the lack of experience of the decision maker. In situations involving uncertainty they should not be seen as second best, rather their merits should be considered alongside more complex modelling alternatives.

Gigerenzer was involved with a Bank of England project on the use of simple heuristics which led to Andy Haldane’s The Dog and the Frisbee paper referred to in the Face up to uncertainty principle (Haldane, 2012). This paper highlights the benefits of simplicity and how, for example, a simple heuristic restricting bank leverage ratios would have been at least as effective as the complex (and expensive) model-based rules that are promoted in banking regulation.

Key takeaways are:

• Simple heuristics may often be as effective as complex models where there is high uncertainty. “The simplest solution tends to be the right one” (Occam’s razor ⁵ )

• If one (or two) heuristics work, then there is huge benefit in simplicity (the marginal value of additional heuristic and complexity may be small so not useful)

• If the problem is one of “risk”, or in games that are fully defined, then complex modelling is much better, e.g. if playing roulette or black jack at a casino then do your calculations – don’t use heuristics!

It is important to think about modelling and heuristic options as a toolkit. The specific situation and context are key to the choice of tool(s), hence the experience and wisdom of the decision maker (and adviser) is paramount.

4.2 When to use a model

The Face up to uncertainty principle introduced the distinction between problems that are largely quantifiable (“modelling problems”) and those that are largely unquantifiable (“uncertainty problems”). It is important to understand what is unquantifiable and hone your “unknowability radar”:

• Identify limits to knowledge

• Spot bad (actuarial) science

• Spot hard problems

Your unknowability radar can be used to distinguish between smooth and knotty problems – those that are easy to solve and those that require additional care. It will also enable us to identify unknowable problems – those that are ultimately impossible to resolve with any confidence.

Table 1. Examples of distinguishing problem unknowability

As outlined at the beginning of this section, models can be useful in different ways. If a problem is unknowable there may still be valuable insights from comparison exercises and system understanding through modelling, but the potential for inappropriate and dangerous use is obvious.

Do not be afraid to say where modelling cannot contribute to a decision: Face up to uncertainty. More constructively it is important to consider how uncertainty can still be managed through the use of good “rules of thumb”. However your rules of thumb may turn out to be poor so it is also important to Think about adaptability and resilience, our next principle.

An illustration of unknowability

In their paper “Ersatz model tests” (Jarvis et al., 2016), the authors consider the following problem. We have ten historic observed losses 26, 29, 40, 48, 59, 60, 69, 98, 278, 293 (listed in ascending order). The mean of these observed losses is 100. What is the value of a 1 in 100 loss (i.e. there is only a 1% chance of the next, 11^th, loss being bigger)?

Consider three scenarios:

• Green scenario. The losses come from an exponential distribution with a mean of 100.

  The 1 in 100 (99%ile) of this distribution is 461 (to the nearest whole number).

• Amber scenario. The losses come from an exponential distribution with an unknown mean.

  The observed mean is 100 but the true underlying mean could be higher or lower (we could have been lucky or unlucky). We could estimate a distribution of the mean (formal parameter uncertainty) and calculate (or simulate) the 1 in 100 number. This will be more than 461.

• Red scenario. We know neither the underlying distribution nor the mean

  The observed mean and standard deviation are both exactly 100, consistent with an exponential distribution. But we don’t know either. We could select a distribution, or an ensemble of distributions, and assume the mean has some parameter uncertainty. This could give a range of results but it is likely the 1 in 100 loss will be greater than for the amber scenario.

The extra uncertainty in the red and amber scenarios relates to a lack of knowledge, rather than inherent randomness (referring to our list of uncertainty types in Deconstruct the problem: “Limited knowledge due to ignorance” rather than “Limited knowledge due to the unpredictability of fortuitous outcomes”).

It is rare to find practitioners assuming they are operating in anything other than a green scenario world (even though they might acknowledge this assumption). Distributions are fitted to observed data and results from the tail of these distributions are used with limited challenge. More sophisticated approaches may appear to counter this by incorporating, for example, parameter uncertainty. However, such methods may be argued to be somewhat spurious as decision makers can be misled by the implied accuracy with the unknowability being apparently addressed.

The real world is always red. However, this does not mean model use is always inappropriate or dangerous. But it does require careful consideration of the context and the influence model results will have on a decision. In practice complex models can be used intelligently by experts along with other heuristics to ensure limitations do not detract from good decision making.

4.3 Example: economic capital models

It is increasingly common for (re)insurers around the world to use an Economic Capital Model to help inform business strategy. This is particularly common in Europe where there are potential regulatory benefits from it (such as lower capital requirements). A typical Economic Capital Model will be a probabilistic model that looks to quantify the capital buffer that the company needs to hold to be able to withstand potential losses to a specified tolerance level. The tolerance is usually quite extreme. Under the European Solvency II regulatory regime, for example, the tolerance is set at a 0.5% probability level over a 1 year time horizon. That is, the model is attempting to quantify a 1 in 200 year loss scenario for the company.

The calibration of Economic Capital Models is reliant on experience data. Availability and consistency will vary, but it is unusual to have more than 15 to 20 years of data that can be considered to be relevant for modelling the company as it is today. The models attempt to extrapolate from this experience to what a 200 year level loss could look like. This extrapolation is clearly very uncertain.

Some model components use elegant techniques to try to get around this, using methodologies that can be based on better datasets. Natural catastrophe models are an example, which are typically based on physical models. These can make use of extensive weather datasets, such as for wind speeds and rainfall which have been captured over many years. However, with climate change and other trends, even this is flawed.

Then there are unknown unknowns, things that we have not considered and that have not occurred yet. Some allowance may be made by introducing loadings for Events Not In Data (ENID). However, this is inevitably highly subjective.

Unsurprisingly, the overall 1 in 200 estimate is extremely uncertain and taken literally is an “unknowable” problem as highlighted by our red scenario.

It is worth noting that regulators use a “rule of thumb” to sense check capital model results. It is called the Standard Formula and is a set of leverage factors relative to premium, reserves and assets. This is a useful tool in the hands of an experienced practitioner to make sure that the results of an Economic Capital Model seem sensible relative to peers and market norms.

The unknowability issues are well known and it is well recognised that the 1 in 200 estimate produced by an Economic Capital Model is not very accurate. Despite this, many insurers have invested heavily in developing Economic Capital Models and use them extensively in running their business. This is because the ways in which the models are used are not always reliant on the 1 in 200 being precise. Often it is the movement in this risk measure is what really matters, rather than the absolute number.

Change of business strategy illustration

Your model might indicate that pursuing a certain business strategy might increase your 1 in 200 risk measure from $500 to $600 million. Instead of fixating on the dollar number, it may be better to view this as the risk increasing by about 20%. Though the absolute number, $600 million, might suffer from the accuracy problems described above, the relative movement, 20%, is likely to suffer less and be more accurate.

Now let’s consider an alternative strategy, which is expected to lead to a similar increase in profitability. We see from testing this in our model that it would increase the 1 in 200 from $500 to $700 million. This is a 40% increase. Our model is suggesting that the first strategy is likely to be better from a risk versus return perspective. We cannot guarantee for certain that the first strategy is the best as it is still based on a model, but using the model in this way should be far more resilient than trying to anchor on the precise number.

Investment optimisation illustration

A real-life example was seen with an insurer who was looking to optimise their investment portfolio. They set up a model and it suggested a portfolio that would give the highest return for a specified level of risk. The optimal portfolio was seen to give a significant increase in return for broadly the same level of risk as the current portfolio. On inspection, it could be seen that the existing portfolio was over concentrated in certain sectors and was not as efficient as this new portfolio.

However, the model was seen to be very sensitive. A series of sensitivity tests were carried out where the model was based on alternative economic assumptions. Worryingly, each alternative test resulted in a different optimal portfolio. The alternative assumptions were all plausible, so it was not clear which portfolio would be best. But it was seen that the proposed new portfolio did perform reasonably well under all the alternative scenarios and in all cases looked to be far better than the existing portfolio. So whilst the company could not be certain that their new portfolio was optimal, they could be reasonably confident that it would be a significant improvement on their existing portfolio. Despite its uncertainty, their modelling had guided them to a better portfolio than they would have had without it. Using a model in this way links in closely with the ideas in the “Think about adaptability and resilience” section.

So despite their clear limitations, models can be useful if used right. Using risk models like this can help reinforce a risk culture and drive appropriate behaviours, leading to companies that are better able to manage risk.

4.4 Example: natural catastrophe models

There are well-publicised failures of the catastrophe models that are used by the insurance industry, such as their failure to allow for the possibility of flooding in New Orleans from the levees breaking in Hurricane Katrina or very recently with the rainfall from Hurricane Harvey.

However, it is well accepted that the use of these catastrophe models has given insurers a better understanding of their exposures and accumulations and the number of insurer failures from natural catastrophe losses has declined markedly since their adoption.

Knowledge of engineering and science is continually expanding. For example, the US Geological Survey has recently helped develop a better understanding of earthquakes in the US – which has been incorporated in recent commercial catastrophe models. This and other developments are expected to lead to more accurate modelling, or improved insights, over time.

For rare events, by definition, there is always limited data and this can be interpreted in different ways. For example, the New Madrid earthquakes of 1811 to 1812 could be treated as either three unrelated earthquakes spaced a few months apart or as a series of connected events. The view taken will make a significant impact on your modelled distribution – and understanding these uncertainties is key to making the best use of the historical data.

A similar example is catastrophe modelling that previously considered a point estimate on the likelihood of flood defences failing under particular stressed conditions. These were later developed to be based on a range of likelihood estimates (e.g. “cautious”, “mid-range” and “optimistic”), as assessed by a focused group of experts. This led to a much wider range of uncertainty estimated in the modelling results – allowing decision makers to better understand these inherent uncertainties.

This uncertainty can be problematic if the model is needed to give a point estimate, such as for calculating a price for an insurance contract. In this situation we need to establish protections so that we can be resilient to the possibility of the model being inaccurate, for example, by using prudence margins or purchasing reinsurance. This links to the ideas from the Think about adaptability and resilience principle.

A key positive feature of catastrophe models is that they break the problem down into a series of areas – each of which is focused on a single domain of knowledge. This links in with the ideas in the Deconstruct the problem principle. This structured approach helps with model conceptual understanding and allows individual parts of the model to be developed in isolation as new research is available.

As users of catastrophe models, our role is to assess the overall appropriateness of the output and to clearly communicate the known limitations and uncertainties.

5) Think About Adaptability and Resilience

In the face of uncertainty it is important to adopt resilient thinking and forward looking planning. This builds preparedness and adaptability to deal with the consequences of decisions that may not turn out as hoped. While such surprises can be unwelcome they may also present opportunities.

Thinking about adaptability and resilience is not just about risk mitigation or avoidance, but also the ability to take advantage of new circumstances.

We Deconstruct the problem to suggest an Adaptability and Resilience Toolkit covering the things you can control and those you cannot:

Things you can control

• Have a clear strategy and approach to uncertainty

• Think about what can go wrong? “How much we can afford to lose”, “what will hurt/kill me”?

Things you cannot control

• Build in strength and options. “What makes us stronger?”, “What might make us re-think the strategy, and what options do we have?”

• Plan for outcomes (not causes). The potential causes may be too complex, uncertain or unknowable to make a complete analysis realistic

5.1 Introducing the adaptability and resilience toolkit

In the face of uncertainty, it is important to be resilient to adverse outcomes (see Figure 7). It is also valuable to be able to adapt and capitalise on opportunities following changing circumstances and expectations.

The traditional ERM framework focuses on risks that can be identified, controlled and quantified. Unfortunately, the world does not divide neatly up into risk and sub-risk categories, and it may be that the residual risks are overlooked, which is where the “adaptability and resilience toolkit” comes in. It helps us identify and capitalise on upside risks, when treading in uncertain waters, as well as protect against downside risks, when navigating uncertain waters.

5.2 Clear strategy and approach to uncertainty

In his book, “Good Strategy/ Bad Strategy” Richard Rumelt (2011) explores successes, and failures, in corporate strategy. Quoting John Kay, writing in the FT (Kay, 2012):

“For Prof Rumelt, the kernel of a strategy is the diagnoses of a situation, the choice of an overall guiding policy and the design of coherent action. A guiding policy is an element of strategy, but is not a strategy until it is translated into specific actions. One of the silliest remarks in business is “strategy is easy, implementation is difficult”. But strategy that lacks a clear path to implementation is not strategy at all, just wishful thinking”.

The nature of the strategy that should be adopted in the face of the uncertainty will vary according to the decision and its context. In a commercial context, the chosen strategy will depend on the organisation’s commercial (or other) objectives, market opportunity, stakeholder expectations and available resources. It will also depend on the organisation’s risk appetite and attitude to uncertainty. A key point in Rumelt’s thinking is that a strategy alone will not determine success – it is critical to understand the issues at stake, and have a set of actions to support your overarching goal.

When it comes to uncertainty, as time progresses the situation and outcomes will change as new information comes to light. It may or may not be possible to predict such changes, but it is important to build optionality into your game plan – actions you will or won’t take to reflect on the new circumstances and adapt where appropriate.

In a negotiation, for example, it is important to understand the situation (see 2-way communication in the Don’t be fooled section) and decide on a strategy regarding the information you are prepared to reveal and when, the outcomes you consider acceptable, and your overall approach to making and responding to proposals.

Some situations present additional options and challenges.

In Dr Keith Bickel’s book “Mastering Uncertainty: The three Strategies You Need to Know”, (Bickel, 2011) he proposes three alternative strategies:

• Pioneer strategy: High Risk/ High Reward, Market Shaper

• Pouncer strategy: Medium Risk/ Reward, Marketing Powerhouse

• Hedging strategy: Medium-Low Risk, Willing Contrarian

In the 1990s demand for personal handheld phones was uncertain. Nokia (a pioneer) moved early to “make” the market. Ultimately, they were not as successful as Apple or Android (pouncers) who redefined the product, once demand had stabilised and investment costs were lower, creating change in consumer demand that Nokia could not keep pace with. By contrast, hedgers keep options open by investing in one or more areas that run counter to their current strategy, for example, Tesco marketing mobile phones as well as acting as a distribution channel.

When faced with areas of significant uncertainty, for example, Cyber Risk (see below), a fourth strategy is also available to you: “avoid”. However, this strategy can result in an extremely destructive outcome, e.g. Kodak’s avoidance of the digital imaging market which turned out to be the winner.

To put the above strategies into perspective, we consider these with regards to “cyber” risk and how various players in the market have adopted one of the above strategies.

5.3 TO cyBEr or NOT TO cyBEr?

While cyber risks affect all organisations, our focus here is on insurers who have considered and chosen their commercial strategy to take on (or to avoid) cyber risk.

Financial Losses from cyber attacks (an attack from one or more computers, on others) are significant and have been increasing rapidly in recent years ($5 billion estimated damages in 2017, projected to increase to $1 trillion by 2021).

The scale of economic vulnerability to cyber attacks is not only material but also very highly uncertain: for example, Lloyd’s recently estimated that the potential economic impact of a malicious hack on a cloud service provider ranges from $15 to $121 billion (Maynard & Ng, 2017), highlighting factors such as the nature of the industry sectors exposed and duration of the outage. This is confounded by contagion risk – such that it is not clear where an attack might arise from, or why. For example, experts believe the “NotPetya” attack in 2017 started out as a politically motivated attack on Ukraine, targeting power stations (Chernobyl), banks, ministries and transport systems. However, it was designed to spread quickly and did – with companies such as Cadbury’s, Maersk, DHL, DLA Piper and a British advertising company all finding they were affected.

In a capital rich and highly competitive global insurance and reinsurance market, cyber risk and the extent of the associated uncertainty presented a market opportunity. Some players, such as AIG, Beazley, Hiscox and Swiss Re amongst others, adopted a “pioneer” strategy and moved early. This allowed them to start to develop the expertise, frameworks, tools and partnerships required to manage exposures and the associated uncertainty. Others, such as Berkshire Hathaway, have chosen to avoid or strictly limit exposures: Warren Buffet said that “cyber is uncharted territory” and the risk is going to get “worse, not better”.

At this early stage, it is not clear which of these strategies will ultimately prove to be the better one. In 2016, globally insurers writing cyber insurance ran average combined ratios of over 75% (Aon Benfield Analytics, 2017) for Cyber. It remains to be seen whether this apparent level of profitability is sufficient to compensate insurers for the risks being run, and it may well be it does not.

Conversely, all insurers (pioneer or otherwise) are finding they are exposed to Cyber losses:

1. (1) “through the back door” – through more standard property and casualty insurance, as organisations they insure look to recoup cyber losses wherever they can; and

2. (2) “through the front door” – through vulnerabilities in their own data and technology systems. It remains to be seen which insurers will ultimately prove to be the most resilient to cyber risk?

5.4 What can go wrong?

In the Deconstruct the problem principle we described the benefit of unpacking the assumptions underpinning the decision: where are we now, where do we want to get to and how are we going to get there? In order to understand what can go wrong, it is helpful to start with the end in mind.

“All I want to know is where I’m going to die so I’ll never go there.” Charlie Munger

In the face of uncertainty, it is fundamental to know how much downside risk you are exposed to at any point (however remote) and balance this against how much you can afford to lose. A resilient organisation will typically understand these exposures and build redundancy into the system to allow strategies to fail, without breaking the organisation, or at least allowing it to recover.

Yet, as we discuss in other sections of this paper, quantifying potential losses can be hard and the degree of difficulty may be a significant driver of the strategy that is finally adopted. In the context of a “going concern” it is often difficult to step away from a natural bias towards the organisation’s current business plan and recent experience. Even when it is possible to envisage future scenarios, the uncertainty attaching to a forecast (e.g. initial estimates of the global vulnerability to climate change) can be so great as to hinder any reliance being placed on these estimates in decision making (in this case, government policy and public behaviour have been slow to react).

However, one may be able to make a decision by considering potential downside scenarios, their financial impact, and the limits on the amount of loss the organisation is willing to tolerate (risk appetite).

It is worth noting that Solvency II (European Union, 2009) has encouraged many insurers to develop a range of scenarios to assess downside risk and its potential impact on their organisation. Good practice is for scenarios to be developed collaboratively by risk management and business experts and to consider external and internal threats (e.g. extent and responses to a cyber attack). A key benefit is that these scenarios can be used as an independent challenge to more complex models used to quantify risk. The more sophisticated insurers have extended scenario planning further to recoverability assessments; considering not only the “biting” scenario (i.e. what will kill them) but also the corrective actions (pre and post event) that will be available to them to respond. Scenario planning forms a key part of business planning and management within insurance, and is bringing about a cultural change in the insurance industry to actively focus on corrective actions and recovery plans. However, resilient thinking highlights two key challenges to scenario planning that organisations should consider:

• Firstly, these scenarios test a continuum of more or less extreme outcomes but do not necessarily test their sensitivity to the assumptions underlying them.

• Secondly, the Solvency II directive has steered most organisations towards focusing energy on developing, and contingency planning for, quantifiable scenarios. Consequently, the non-quantifiable uncertain scenarios do not get appropriate level of focus but can be equally detrimental to the business.

It is not our view that the above two pitfalls are ignored by the insurance industry, but merely an observation that these have the potential of not getting their due importance within business management and can lead to unexpected shocks.

Example: uncertainty from specific events

Rewinding to 1994, the Northridge Quake surprised insurers who struggled to provide cover to the market. The US State formed the Californian Earthquake Authority (CEA) which would reinsure the market, thereby helping insurers provide cover to homeowners and other property owners. However, CEA wanted to obtain retrocession in the market – no one wanted to take it.

Berkshire Hathaway took 100% cover for 3 years, starting 1 April 1997. Buffet’s rationale was as follows:

“Large as these coverages are, Berkshire’s after tax “worst case” loss from a true mega-catastrophe is probably no more than $600 million, which is less than 3% of our book value and 1.5% of our market value … I have mentioned that a mega-catastrophe might cause a catastrophe in financial markets, a possibility that is unlikely but not far-fetched. Were the catastrophe a quake in California of sufficient magnitude to tap our coverage, we would almost certainly be damaged in other ways as well. For example, See’s, Wells Fargo, Freddie Mac could be hit hard. All in all, though, we could handle this aggregation of exposures.

In this respect, we try to “reverse engineer” our future at Berkshire, bearing in mind Charlie’s dictum: “All I want to know is where I’m going to die so I’ll never go there”. … If we can’t tolerate a possible consequence, remote though it may be, we steer clear of planting its seeds. That is why we don’t borrow big amounts and why we make sure that our super-cat business losses, large though the maximums may sound, will not put a major dent in Berkshire’s intrinsic value.”

Example: evolving uncertainty

Many infrastructure projects entail significant cost commitments and assumptions which in turn are justified by long-term benefits over many decades. For example, consider the decision-making process behind projects such as the Channel Tunnel, airport runway development (e.g. Heathrow versus Gatwick) and HS2.

The rationale behind these decisions is often based on long term forecasts of, for example, economic growth and passenger behaviour. The range of assumptions and their uncertainty characteristics is considerable. However much “science” is used it seems obvious that many issues are unknowable over such long time horizons.

A characteristic of such problems is that the project uncertainty develops over time with new information becoming available. It is important to stress test assumptions at the time of the original decision: what can go wrong? However, it also important to reflect on the next tool in our Adaptability and Resilience Toolkit.

5.5 Build in strength and options

What makes us stronger? What might make us re-think the strategy, and what options do we have?

It is often difficult to step away from a natural bias towards the organisation’s current business plan and recent experience. Hence there is an element of over-confidence in the underlying assumptions and knowledge of the outcomes. Even when it is possible to envisage future scenarios the investment in exploring those scenarios is often seen as a cost burden rather than making the organisation stronger. An example where certain players in the industry took the view that the investment was worthwhile and introduced optionality into their business models arose when the survival of the Lloyd’s market was potentially in doubt. Although fully supporting the Lloyd’s model and expecting it to survive, some managing agents took the view that setting up an alternative regulated entity provided good optionality. Even though Lloyd’s survived, this optionality created additional value in that those players had a larger capacity and could tap different market segments through these new entities.

An additional aspect to consider in order to make the business stronger is to continually question and reassess the assumptions on which the key decisions are based and in particular if certain outcomes are binary (as in the above case). For example, and with hind-sight, had the directors of the Equitable Life Assurance Society brought in additional independent challenge to their approach to guarantees and bonus allocations they may have changed their strategy or explored mitigation actions that may have been useful should their assumptions about the future behaviour of interest rates proved to have been incorrect. Even when the problems with their approach became apparent had they been more agnostic as to the likelihood that the court may disagree with their approach to bonus allocation amongst its various cohorts of policyholders then they might have identified certain options for the survival of the business at an earlier stage.

On the flipside there are organisations like Royal Dutch/Shell, who in the late 1960s/early 1970s, developed a technique called “scenario planning (and storytelling)” (Shell n.d.). By listening to planners’ analysis of the global business environment, they brought their management team around to an understanding of the risks inherent in its business model, bringing in optionality/preparedness to deal with certain scenarios.

This was seen in October 1973, when Arab oil-producing countries imposed an oil embargo on Western governments – a response to US support for Israel in the Yom Kippur war. Within weeks, the price of crude oil soared from around $2.50 a barrel to $11, restricting economic activity in the West and causing a slump in the stock market. Shell’s scenarios work had allowed it to foresee and, to some extent, prepare for this shock. They were able to recover more quickly than their competitors.

And again in 1981, when other oil companies stockpiled reserves in the aftermath of the outbreak of the Iran-Iraq war, Shell sold off its excess before the glut became a reality and prices collapsed.

5.6 Plan for outcomes (not causes)

The Models can be helpful, but also dangerous principle highlighted the limitations of using complex models insituations involving uncertainty, or indeed unknowability. A warning sign of inherent uncertainty (and unknowability) is where the system is so complex to the extent that underlying exposures are opaque and their size can no longer be assessed.

When faced with “black swan events” or “unknown unknowns”, it may be more relevant to plan for the outcome rather than the specific event in question.

The financial crisis of 2007/2008 taught us that the packaging and re-packing of default risk on sub-prime mortgages within mortgage-backed securities (MBS) or collateralised debt obligations (CDOs), made it impossible for investors and institutions to grasp the extent of their exposures. Indeed, a very similar scenario played out within the London Market in the 80s/90s, where exposure to natural catastrophe events was packaged and re-packaged via insurance, reinsurance and retrocession such that some insurers took risk back onto their books (via inwards reinsurance) that they believed they had passed on to others (via outwards reinsurance).

As the “NotPetya” cyber attack showed, it is not always that relevant for an organisation (say DHL, or Maersk) to specify the source and nature of individual events (in this case, a political attack on Ukraine).

A recent example where this has been implemented at an insurance industry level is the “Market Turning Event” study (The London insurance market - various, 2017) where 28 market participants, Lloyd’s, The Prudential Regulatory Authority and the Financial Conduct Authority took part in a simulated exercise to examine how well the market can deal with some catastrophe events. The introduction from Robert Childs, Chairman of Hiscox sets out the key benefits derived from the exercise and we consider these types of benefits can also accrue to organisations that buy into managing uncertainty.

“In such an exercise it is very easy to become fixated on financial performance, but assessing solvency was not the focus of this project, which is precisely why we went further than measuring the markets financial muscle in a market-turning event. It encouraged participants to think about how to get a handle on the loss, pay claims swiftly, and at the same time seize the opportunity in a hardened rating environment. We also focused on two other very important aspects of managing such a major crisis: how to communicate clearly and candidly, and how to provide leadership when there may seem to be disarray. This exercise taught us many things, which we outline in this white paper. Perhaps the most valuable, though, is that a catastrophe does not need to be a crisis. The London insurance market is well established, highly regarded, and robust enough to withstand shock losses. We have deep underwriting and management expertise, a leading position in many lines of business and, most importantly, a commitment to pay claims. We have a vital part to play in the City of London as well as the wider UK economy. Contingent on a surefooted regulatory response when the worst happens, this exercise shows we have all the ingredients we need to not only survive a market-turning event, but to thrive. We look ahead with confidence. The London Market needs to prepare itself for the next major market-turning event. The last time we were really tested was during the tragic events of 9/11, and much has changed since then.”

The tragic events of 9/11 also led other organisations, and many federal and national government bodies to consider their readiness to deal with catastrophic events. The lessons learned from the “dress rehearsals” led to a very positive outcome when the impact from the terrorist attack at the Boston Marathon was greatly reduced (United States Federal Emergency Management Agency, 2013).

In Massachusetts, the Federal Emergency Management Authority (FEMA) convened a summit for the emergency management and response communities from multiple jurisdictions to plan for an unknown disaster. The focus included effective leadership and coordination of multiple government and non-government agencies; building trust, defining roles and responsibilities and lines of communication between agencies that would be effective in a period of stress; as well as response planning and training, including multiple large-scale dry-runs.

When two improvised explosive devices (IEDs) detonated near the finish line of the 117th Boston Marathon on 15 April 2013, confusion ensued. The source, extent and nature of casualties were unknown, and it was hard to reach injured runners through volumes of runners and spectators present at the finished line.

At this point the disaster plan kicked in. Very sadly, 3 people died, 16 lost limbs and 264 were injured. Yet, every one of the wounded who were alive when rescuers reached them, survived.

News of the blasts travelled quickly through twitter, text, news and smartphone apps. Medical staff at the hospitals put all scheduled surgery on hold and prepared rooms for emergency vascular and orthopaedic procedures (they expected wounds from shrapnel, fractured bones, etc.) as well as one or two for neurological and thoracic injuries (in case). Off shift medical staff heard the news and headed straight to the hospitals.

Two co-ordination centres were in constant communication with the cities and hospitals along the marathon’s 26 mile route, facilitating exchange of information and “near real-time situational awareness”. The National Weather Service switched from determining weather reports, to monitoring toxins or hazardous chemicals. Extensive “dry-runs” of disaster events had institutionalised roles, responsibilities and unity of effort.

“Talking to people about that day, I was struck by how ready and almost rehearsed they were for this event.” – New Yorker

6) Bring People with You

There are often many stakeholders involved in a decision. You, as the decision maker or an expert advising the decision maker, may be enlightened to face up to uncertainty and have various ideas having deconstructed the problem:

• You understand the real question being asked? Don’t be fooled

• You understand how the problem should be analysed (use of models? what if things go wrong?): Models can be helpful, but also dangerous, Think about adaptability and resilience and Don’t be fooled

• Your eyes are open to the uncertainty that exists

However your efforts will be frustrated if your knowledge and perspectives are not shared by others. It is vital to bring people with you and it is vital their eyes are also open to the implications of uncertainty (and unknowability) when the decision is made.

We look at three areas:

1. 1. Resistance: the importance of recognising it, understanding it and managing it

2. 2. Building trust: the importance of engagement and understanding at the time of an important decision and the benefits of developing knowledge and trust ahead of such situations

3. 3. Communication: understanding what to share and how

You may Face up to uncertainty, but if others involved in the decision are not engaged, then your best efforts will be limited in their effectiveness. Do others:

• Understand the problem?

  • ○ Deconstruct the problem

  • ○ Don’t be fooled: understand any agendas (and potential latent framing biases)?

• Understand the way the problem can be analysed?

  • ○ Models can be helpful, but also dangerous: what can be quantified, how will models be used, what is unknowable?

  • ○ Think about adaptability and resilience: what can go wrong, and what options exist if it does?

• Have sufficient knowledge and engagement to take appropriate responsibility for the implications of uncertainty on the decision?

The last point is key. If a decision maker genuinely lacks the knowledge or awareness of uncertainty, or is not sufficiently engaged to take proper responsibility for the uncertainty (i.e. just blame the “expert”), then there is an inherent problem. Similar issues rest with an expert adviser: they need to understand the knowledge and perspective of the decision maker and judge what should be communicated and how. Bring people with you is the last principle and one that runs alongside all the others.

6.1 Resistance ⁶

It is natural for us to feel that if we can present our ideas clearly and logically, and if we have the best interests of the stakeholder at heart, then our expertise and ideas will be accepted and followed. We soon discover that no matter how reasonably we present information and recommendations, we can be met with resistance.

Resistance can be puzzling and frustrating. We begin to view the person resisting as stubborn or irrational. The key is to understand that it is an emotional process. It is not a reflection on the issue at an objective, logical, rational level.

Resistance is a natural reaction against the process of having to face up to uncertainty and difficult problems. The stakeholder is defending against the fact that they are going to have to make a difficult choice, take an unpopular action and confront some reality that they have emotionally been trying to avoid.

When you encounter resistance, you are seeing the surface expression of more underlying anxieties (often fear of loss of control or vulnerability, both very common where there is uncertainty). Two things are happening: the stakeholder is feeling uncomfortable and is expressing this discomfort indirectly. To overcome it, the resistance needs to be expressed directly before a stakeholder will be genuinely ready to move forward and engage with the decision.

The skill is to:

• Identify when resistance is taking place (e.g. anger, withdrawal, procrastination, flooding with detail, brainwashing, repetition of a stock response, etc.)

• View it as a natural process

• “Name the resistance” by articulating what is happening in a neutral way (e.g. you seem tense, could you explain why?)

• Listening to the stakeholder’s response carefully

• Not take it personally or as an attack on you or your competence – you are uncovering valuable information

The best approach is to help resistance express itself, so that it becomes less powerful and may even blow away, like a storm. The worst approach is to fight it head-on. Feelings pass and change when they are expressed directly. The most effective way to encourage someone to be open is to be open yourself. You cannot talk someone out of resistance because it is an emotional process. Behind it are certain feelings, and you cannot talk people out of how they are feeling. If you fight the resistance and feel you have to conquer it, all you will do is intensify the resistance.

6.2 Building trust

You may have worked out how to manage uncertainty in a given situation, but will anyone listen to you? If you are an adviser to the decision maker then to what extent will they own and take responsibility for the issues you have identified? If things do not turn out as expected (and they might not – they are uncertain and may even be unknowable), then will you be blamed? Is this fair when it was not your decision?

For a given decision it is important to understand the wider context and consider all the stakeholders. A stakeholder deconstruction was shown with the Deconstruct the problem principle, repeated here for ease (Figure 8). It is important to identify the main decision maker as well as others with power and influence. For simple situations this will be straightforward but can become highly complex and political in some scenarios.

Figure 7. The Adaptability and Resilience Toolkit.

Figure 8. (Original Figure 4).

The Uncertainty and Assumption Onion (see Deconstruct the problem) illustrates the value of identifying what is in the outer layers: the problem context and the approach to be taken at an appropriate level (scope, data, use of experts, use of models, etc.) – without overloading everyone with irrelevant or avoidable technical detail. Engagement with the outer layers is essential for everyone and a responsibility of both the decision maker and the expert(s).

Possible techniques for stimulating engagement and encouraging ownership (of the whole Onion) include:

• Ask rather than just tell. Questioning helps make people think about things (and may get their attention if they seem not to be listening)

• Put in broader context. Highlight the importance and consequences of the decision: what are the upside and downside implications?

• Seek input on key judgements. For example seek advice on use of experts

• Bring out “carrots” and “sticks”. What are the potential positive benefits, and what negatives should be avoided?

Successful engagement in a given situation is of course good. But trust takes time. If there is an urgent decision to be made, triggered by a particular problem, it is unreasonable to expect full engagement and understanding in the outer Onion layers if certain important issues have not been discussed previously.

Effective trust and understanding benefit from longer term thinking and planning. Time invested in engagement during periods of relative calm will give a stronger base for decision making in times of crisis.

Illustration: a big bold idea

You have a big bold idea, which is quite radical but has the potential to be very lucrative. So what is the best way to sell the idea to your superiors? Bring them with you on a journey.

Going into a board meeting and putting it out there in one go could easily scare people off. They may be impressed with the idea and at the time may not have any immediate concerns, but will probably feel uncomfortable at not having the opportunity to think through the idea and consider the wider implications. Particularly if the idea is contentious, people could become defensive and dismiss the idea. This approach is unlikely to overcome inertia.

A better way is to warm people up to it. This may be by informal chats, one on one, and often repeated. This gives each individual the time to grasp the idea and also express any concerns in a more comfortable environment. This then gives you time to deal with these concerns and bring people round to the idea. The aim is that by the time of the board meeting, all would have a good understanding and have formed their own views.

Illustration: blip or trend?

Consider an example of an actuary performing a year-end reserving exercise for the reserving committee, where the actuary is estimating the performance of the business written for the purposes of preparing statutory accounts and regulatory returns. The year in question has seen poor initial results and an unexpected increase in very high valued claims.

If the performance of the business has been worse than expected the actuary’s job tends to be much harder, and the actuary may find some of the following happen:

• There is a lot more challenge to the actuary’s numbers and the actuary needs to allow time to explain results to stakeholders;

• The owners of poorly performing lines of business may change behaviour:

  • ○ There may be hostility towards the actuary (see Resistance earlier)

  • ○ In extreme cases it may be hard for the actuary to obtain clear information to understand the results

  • ○ The actuary may be asked to carry out extra studies under tight timeframes to investigate and allow for good news stories (e.g. to prove claims are speeding up, so results are not as bad as they look)

  • ○ The actuary’s methodology or data may be discredited (e.g. “it is not possible to use paid claim data any more as the patterns are no longer reliable due to changes in process”)

• The actuary may start to doubt herself or himself as there may be people from different areas of the company all saying that the actuary has misunderstood

Are the poor results a blip or do they signal a worsening trend? The answer is probably unknowable and the actuary is in an impossible position as if wrong will be blamed as “the expert”.

A better earlier strategy would be to have invested time with the reserving committee so there was a broader understanding and appreciation of the different areas of potential uncertainty. By the time the unexpected increase in very high valued claims occurs it is too late.

This would not change the problem but would help ensure a more informed appreciation and ownership of the unknowable issues. The actuary may still be wrong in the judgement of blip or trend, but the decision maker (the committee in this case) would understand the issues and rationale for the decision themselves and not seek to cast blame.

6.3 Communication

Presenting technical analysis is one thing, what is important is to engage and influence.

It is important to avoid jargon and to try to keep things simple when presenting to a non-technical audience. But to be really effective, we need to adapt a lot more than just that. The way we present things may change depending on the character of the individuals. Some individuals are detail focused, others like things kept at high level. Some like charts, others like tables of numbers. Knowing the audience is key.

Strategic communication: climate change

In their 2011 paper in Nature (Pidgeon & Fischhoff, 2011), the authors put forward the concept of strategic communication. Communication of uncertainty amongst scientists is very different from the needs of lay people and policymakers. Indeed, scientists’ very familiarity with the issues can impede their ability to communicate with people outside their field, to the point that even excellent scientists can be poor communicators.

Unless people receive the information that they need, they must guess at it. Useful considerations and approaches include:

• Consider the perspective of the audience

  • ○ What are their objectives and risk appetite?

  • ○ Seek to understand and “fill in” their understanding of the model

• Understanding risk requires more than just knowing risk estimates. People also need cognitive representations (or “mental models”) of the processes creating and controlling the risks, and thus causing uncertainty about them

• Communicate in ways that are as accessible as possible

  • ○ Keep it simple: cover the basics as well as areas that you wish to focus on, and use clearly defined terminology

  • ○ Talk about what is known and agreed

  • ○ Allow the listener to engage in the model and its results

  • ○ Explain what is modelled and what is not

  • ○ Explain how the model has been “validated” against the real-world

  • ○ Avoid vagaries (“probable”, “likely”, “unlikely”) in favour of scenarios, numbers and probabilities

• Engage a range of specialists and non-specialists in the issue and the way it is communicated. Seek feedback

While these points are highlighted in the specific context of the challenges of communicating climate change modelling and analysis, the broader relevance to more general situations is obvious.

Managing uncertainty for the board

Clearly the numbers will be very important to the board, but the way that the actuary behaves in the situation can help the board to do what needs to be done in the situation even if the results are not as desired. In this way bringing people with you can help the actuary and the board work together to achieve the best outcome for the circumstances.

In times of stress the board will expect the actuary to be competent and credible as well as responsive to the situation that presents itself. Table 2 shows some common pitfalls and suggestions for actuaries presenting results that contain bad news:

Table 2. Presenting bad news

Part 2: Case Studies

Case Study A: Managing Uncertainty After Being Catastrophically Wrong

The case study is based on an insurer, ModelRe, where the 2017 catastrophes losses were materially greater than the board expected. ModelRe’s strategy was to use catastrophe models to build a portfolio that maximised profit compared to risk. This strategy had led to a disproportionate market share in Puerto Rico and so losses from Hurricane Maria.

This case study applies the managing uncertainty principles to deciding: (1) the 2018 plan for the business that generated the losses and (2) the associated capital requirements.

A.1 Background

• The losses from Maria were generated by a single MGA, StampDown, representing 10% of ModelRe’s premium; StampDown renews at 1/1.

  • - The prospectus for 2018 proposed unchanged exposure and rate change of +50%.

  • - ModelRe is currently using 95% of its catastrophe risk appetite.

• Model Re purchases a single catastrophe reinsurance programme. The programme is material; if it was removed the net risk would double.

  • - The programme renews at 1 April

  • - The general reinsurance market rate change is assumed to be 20%.

  • - ModelRe’s RI brokers are suggesting the market will load the price given the experience by 50%-100% on top of the general market movement.

• ModelRe was well capitalised in 2017 with capital above the regulatory requirement; the Maria losses mean that the available capital has dropped to equal the 2017 requirement. ModelRe must recalculate its capital at the end of 2017. Model Re is required to notify its regulator and re-agree its plan if its capital requirement increases by more than 10%.

• Initial indications are that actual losses are likely to be greater than the model losses.

  • - While uncertain, the loss estimates range between the 1 in 250 and 1 in 10,000 scenarios.

  • - Losses of this size would fail the model validation back test threshold.

  • - The reasons for the model failure are unclear; low damage factors, low data quality and BI exposure have been mentioned.

A.2 Question (1) – StampDown renewal

Face up to uncertainty and deconstruct the problem

There are two key sources of uncertainty: (1) model and (2) rate change uncertainty.

• (1) Loss and model uncertainty. There will not be clarity over the ultimate level of loss and the extent to which model risk needs to increase until after the key business decisions have been taken. Four options for how the modelling could change are set out below:

  • - Has the world changed so that new model/controls are required?

    • ▪ This may be the case but is not clear at this stage. There is no evidence at this stage of incorrect exposure use or other control failure.

  • - The model/controls are ok, but parameters need to be reset in light of new information?

    • ▪ Given the size of loss, the central view is that it is likely that the model risk will be increased if the actual loss remains above the previous 1 in 250 view.

  • - Nothing has changed so all ok, just bad luck, so renew and take whatever the market gives you and more.

    • ▪ While this could be the case, it is difficult to demonstrate when the risk is a material part of the portfolio.

  • - New uncertainties have emerged which makes modelling difficult and the adaptability and resilience principle even more important (e.g. after WTC this feature came into play, a current example being cyber)

    • ▪ There is no evidence at this stage that the risk is fundamentally different to previously assumed.

• (2) Business and rate change uncertainty

  • - The uncertainty over rate change is material.

  • - Given the amount of uncertainty, particularly over the change in reinsurance prices, it is likely that different rate scenarios lead to different optimal choices with respect to renewal. For example, if reinsurance rates increase materially more than gross rates a strategy that reduced gross exposure and reinsurance could be optimal.

Don’t be fooled (un/intentional biases)

It is in many stakeholders’ (StampDown’s in particular) interests to support scenarios where underwriting continues.

• There is a risk that reports (e.g. of gross loss) are presented in a positive light.

• There is a risk of over-optimism with respect to anticipated gross and reinsurance rate change. Testing alternative rate change scenarios is important to understand and communicate the level of risk.

• It would also be useful to understand historic loss and model movements post event (e.g. post 2005 Katrina/Rita/Wilma (KRW) and the global financial crisis) as well as historic gross and reinsurance rate changes.

Some managements would instinctively underwrite the risk, if they were able to, given it would align with (1) “it will be better rated than it has ever been” and (2) managements incentives to take risk.

• Given this instinct it may be useful to ensure formal discussion of the profitability of different business plan scenarios.

Models can be helpful, but also dangerous

The model can continue to be used as a currency for communicating the amount of risk being taken between the MGA, Insurer, Reinsurers and Regulators but the model fail must be clearly communicated to all stakeholders.

Think about adaptability and resilience

There is material uncertainty at the point of renewal. Options include:

• Extending the renewal to 1/4 or 1/6

• Changing StampDown’s risk appetite authority – e.g. give 50% of appetite and then introduce triggers (e.g. rate or agreement of model change) to allow greater deployment as the uncertainty reduces.

Under either option this would formalise the flexibility to change, and potentially reduce, StampDown’s risk appetite to allow for changes including:

• The gross loss deteriorates

• Gross rates increase less than plan

• Reinsurance rates increase more than plan

• Regulators insist on de-risking or increased capital

Bring people with you

Key stakeholders to consult include:

• Board/Non-Executive Directors given the materiality.

• Reinsurer and regulators.

A.3 Question (2) – Year End Capital Requirement?

Face up to uncertainty and deconstruct the problem

There is uncertainty over the regulatory capital requirement following events of this type in addition to the model and rate uncertainty highlighted as part of the renewal decision. The key risk is that the regulatory capital requirement is ultimately greater than initially proposed by ModelRe.

Don’t be fooled (un/intentional biases)

It is in management’s interests to support arguments that limit the increase in capital requirements and so it could be useful to include a range of options for the change in capital requirements including ones which increase.

Models can be helpful, but also dangerous

The increase in model risk to allow for the additional experience will not be finalised before the capital requirement needs to be set. It is likely that the key drivers of the agreement of the level of model change and so capital requirements will be previous case studies (e.g. the model increases post KRW and the global financial crisis) and other regulator objectives (e.g. preventing model drift).

Think about adaptability and resilience

Approaches to manage the key risk that capital requirements increase more than plan including:

• Flexibility to reduce StampDown’s risk as discussed as part of the renewal decision.

• Increases in available capital requirement (e.g. can Letter of Credit facilities be increased)

Bring people with you

• Understand the board/NED requirements to sign-off the SCR.

• Understand regulator perspective and requirements to agree the updated capital requirement.

Case Study B: The Dividend Question

B.1 Setting the scene

You are heading the capital team of a large listed general insurer, with shareholder expectation ⁷ of steady annual dividends. Having had recent losses, the capital position of the company is strained. The CFO comes to you asking for recommendations/solutions to protect the dividend payout the market has come to expect, for the given year and over the next 2–3 years. He asks you to explore options that reduce the 1 in 200 year solvency capital requirements, thereby freeing capital to pay the dividend.

The company manages its business with a risk appetite of having a solvency ratio between 130% and 160%. At last year end, the company declared a solvency ratio of 145% (after declaring the dividend payout). Following poor underwriting experience leading to higher losses, changes to the Ogden rate and higher weather losses from US hurricanes, the company’s current solvency ratio is 135% (before declaring any dividend). What would you, as the Capital Manager, do?

Structure of the Case Study: This case study is set out to put the uncertainty principles in context of the current thinking mechanism. We talk about solutions that can be employed and then consider them through the uncertainty principles to highlight how these principles provide a structure to “Face up to uncertainty” and make appropriate decision in light of it.

NOTE: The reference to principles below is not meant to be an exhaustive application of the principles to the solutions considered, rather a demonstration of how these can help informed decision making in the face of uncertainty. We encourage the readers to consider the other principles (not discussed for the section) to get a broader sense of their day-to-day applicability.

B.2 Understanding the situation

The company has a strained capital position as a result of poor experience in the last year. How well do we understand how the company ended up where it is? What are the underlying drivers for this situation and do we as actuaries have the expertise to judge them?

You will rely on information provided by other teams, for example, the baseline financial plan. You may well ask yourself if the bad experience in the previous year hints at elements not sufficiently incorporated into the previous baseline plan. You know that prospective dividend payments do not just depend on the 1 in 200 number but also on expected future surplus generation. For example:

• Is there an inherent problem in the company’s underwriting approach? What does the Chief Underwriting Officer and his senior underwriters think about this issue? What do you think and why do you think it given other people’s views?

• Are the company’s claims handling and operating expenses too high? How do you judge this?

• Is the provisioning/reserving process robust? Was there an allowance for a legislative shock? Has the degree of prudence in reserves been altered given last year’s results?

• Was there a robust exposure management system in place, to be aware of the losses that may occur?

Understanding how you have reached the present situation and if you have been reading some signals inaccurately in the past, in conjunction with other management actions to correct perceived weaknesses, may also help you frame what can be expected in the future. By acknowledging that the business may have got some things wrong whilst trying their best helps introduce the concept that there is uncertainty and to face up to it.

Uncertainty principles in action:

B.3 Resisting the lure of optimisation

Once armed with a view of the current situation and with a number of lessons learnt, you will attempt to project forward the financial results of the company and the implications on a number of performance measures that different capital management options might entail. In coming up with a projection of future performance and exposures, the following will need to be addressed:

• Is the company’s current predicament a one-off/a trend/an emergence of a trend?

• What are the short-/medium-/long-term solutions being implemented to avert similar scenarios in the future?

• Is the problem likely to be rectified over 1/3/5 years?

The answers to these questions are factored into the base line projections. Of course, they will be educated guesses, but guesses none the less. For example, will the management actions to be implemented to resolve known weaknesses (e.g. expense levels, pricing strength, more effective exposure management, etc.) be effective? And, will the assumed level of interaction between actions and future levels of exposure and income be as anticipated?

The uses and purpose of financial plans are not exclusive to the modelling exercise. In fact, financial planning precedes the practice of financial modelling. Financial models now take financial plans and interpret them as the “best estimate” of what can happen in the future. One of the purposes of financial plans outside their new role as statistical means is to engender commitment and accountability from different departments in the company. For example; to cut costs, to decline badly priced risks, to develop more profitable business.

Being realistic about the nature of the uncertainty underpinning financial plans and complex capital models that take such plans as their “best estimate” is essential to put the suggested dividend protection options into context. Their effectiveness cannot be guaranteed based on modelling results, even though detailed projections and the level of “scientific” appearance will lend them high credibility to non-experts. Stress testing of the dividend protection options under different base line scenarios is encouraged. In fact, the Solvency 2 regulation makes it necessary.

Uncertainty principles in action:

B.4 Capital management solutions

Not all capital is the same

At present your company makes very little use of Tier 2 capital. So one solution could be to issue subordinated debt and return the equivalent amount of Tier 1 capital to shareholders as dividends.

Now, whilst you have a good understanding of the flaws in the assumptions of the Modigliani- Miller theorem, ⁸ your board may not. In particular, there is the risk that if you show the company being at the same solvency position before and after performing the above, some may think that nothing has really changed from a risk perspective. But the fact is that whilst sub debt protects policyholders it does not protect the shareholders and may increase liquidity and other risks in the business. So the cost of returning capital today is that the shareholder returns are more risky. Whether this is an issue or not is another question.

Short-term capital solutions

Your ORSA process models solvency over 3 years. You have always, very sensibly, taken the view that given the difficulty in assessing the 1 in 200 year risk now, projecting it too far into the future becomes a futile exercise.

However, there is the risk that actions taken to support dividends in the coming 3-year period, which seem to increase the value of the company, lead to longer term value destruction. For example, a Quota Share/working XoL reinsurance arrangement can be a useful capital management tool, making use of cheap reinsurance capital. However, there is the risk that if this is taken to extremes, a significant proportion of the company’s profit goes to the reinsurers and it becomes hard for the company to generate sufficient capital organically to wean itself off the reinsurance in the long run. So optimising in the short run can lead to implications in the long run.

Uncertainty principles in action:

B.5 Modelling solutions: “optimising” the model

The 1 in 200 year risk number, by nature, is highly uncertain and there is a tendency to put high reliance on the result for business decision making. The final result from the model is only as good as the inputs feeding into it and you may be tempted to “optimise” the capital result by “managing” the inputs. The intention of the model is to capture extreme risk, and actuaries could be perceived as being prudent when trying to estimate the inherent risk. Arguably, it is possible that capital required for the business is prudent, resulting from the aggregation of small amounts of “bottom-up” prudence in calibration.

As the capital manager, you may be tempted to recognise that prudence at the overall level and may look at pulling some levers to “cut down the fat” in the capital requirement. There may be an argument of “fat” previously recognised in the model which was documented and kept in as a buffer for a rainy day. You may leverage on that to improve the solvency position.

On the other hand, an optimistic business plan for the future may be the easiest and quickest change that could benefit the capital requirement and improve the solvency position of the company, thereby enabling the promised dividend payment. Even if the optimism may be exposed just 1 year down the line with another worse than expected result; now the board, stock analysis and credit rating agency could see a trend underpinned by what they deem to be bad management.

Are these really professional courses of action? Given the high degree of uncertainty that exists in quantifying the extreme risk, coupled with the lack of credible experience to justify it – what is the right amount of solvency capital a business really needs (regulatory/ORSA/published Solvency and Financial Condition Report)? Is it reasonable to assume improvements in profitability within the business plan, which are less likely to be achieved than in previous years? Presumably, the modelling/calibration approach has been used for several years and adapted over time. Given the level of uncertainty inherent in any estimate of solvency capital, the model’s value is often to illustrate changes in relative risk over time and relative risk associated with different business strategies. Indeed, changing the level of prudence or optimism in the model can be looked at as a surreptitious change to the organisation’s risk appetite, by the back door.

As a professional, is it really possible to justify changing the approach, for example, the level of prudence in the model or the business plan, driven by the motivation of improving the capital coverage ratio? Is this a professionally sound decision, from a policyholder and shareholder perspective? Is this good for the long-term sustainability of the business?

Uncertainty principles in action:

B.6 Actuary’s bias

Over-pessimism

You are typically by nature conservative. You like to keep something in reserve for a rainy day. So you present solutions to the board, solutions which you know do not “push the envelope”. Unfortunately, your presentation to the board uses words like “optimal”. So the board decides that they want to be a bit more prudent and only partially implement the advice.

It is not beyond the bounds of possibility that the implications unfold as follows:

• The next dividend is below market expectations. Analysts change their recommendation from hold to sell. The share price falls materially!

• A large group, seeking to grow in a stagnant global market, sees you as an attractive target, and even more so, when they see you as undervalued given the hidden capital in the business.

• Your company is taken over.

Over-optimism

You wake up one morning having had the nightmare set out above. You know the board is conservative, so you push your advice to the limits of what you can do professionally. Unfortunately, the board has noticed the M&A activity, and likewise do not wish to see the business acquired. They also know from years of experience, from when you were the reserving actuary, that you have a tendency to be conservative. So they assume that there is some “fat” hidden in your advice and they push harder than you suggest, paying out more dividends.

Unfortunately, the next year is a difficult one, with weather losses, changes in legislation, investment losses and the like. At the end of the following year, you find your solvency below your peers and your rating under threat. You find yourself the target of a takeover by a large group seeking growth.

Uncertainty principles in action:

B.7 Over-arching concerns: professionalism and governance

The above capital management and modelling “solutions” may sound lucrative and support the CFO’s requirements, gaining some “brownie points” with the boss. It may sound reasonable to leverage on the known prudence in the model, which the CFO may already be aware of. Or may seem minor given that the “1 in 200” number is inherently uncertain, with justifications building from information on more frequent return periods and expert judgement. Hence, a minor “management” of the number may seem harmless. However, the following over-arching concerns should be considered against the proposed changes.

• Risk Appetite: The above solutions have an underlying theme of changing the risk appetite of the business without actively wanting to do so. The business risk appetite will have been set considering various analyses including, but not limited to, the likelihood of burning the 30% margin between 130% risk appetite and the strict requirement to hold 100% of the capital requirement. Any change to the model will inadvertently lead to a change in this probability which should be well understood in light of the changes.

• Model Change & Validation: How will the proposed changes interact with the model change policy? Will they need a validation review? The model change policy is designed to ensure that proper process and justifications are followed for model/parameter changes.

• Checks & Balances: Does the capital manager have OR should the capital manager have the power to agree and implement modelling changes in a way that artificially reduces SCR? With an intention to support a dividend in a scenario where the business is (or can quickly be) strained for capital in the short/medium term?

• Professionalism: As actuaries, we are required to work in a more transparent way through appropriate communication to senior management and the board, and not to hide changes, especially changes not in the best interest of policyholders.

• Regulatory: The regulator may simply construe this as breach of the overall capital policy. The inherent optics of a weaker capital position following a poor experience and a dividend decision supported by a weaker capital requirement may not be well received by the regulator, credit rating agencies, etc.

Uncertainty principles in action:

B.8 Shareholder expectations

You look into a variety of alternatives to protect the shareholders’ dividend and come to the conclusion that, based on the numbers you will present, the board will find it difficult to support a divided payment in line with the initial expectation. You should start warming the CFO to the possibility of not recommending a dividend payment as large as previously suggested. The ultimate judge of what gets paid is the board, with the CFO tasked with recommending the dividend. Discussing the issue with the CFO as early as possible is beneficial from the point of view of managing external and internal expectations of the company’s performance and the dividend outcome for the year.

With time to mull over the consequences the CFO can look for ways to frame the situation and reasons for the decision. Perhaps the CFO’s plan is to work from a position of strength that “despite a poor year for the company (and in the market), the company is paying some dividend…” An alternative framing may be that company and market responses to a bad year should improve profitability and meet the shareholder targets in the medium term. The intention of the exercise is to instil confidence in the long-term stability and profitability of the company.

Effectively, the target of the exercise is to prepare the markets/shareholders for what is to come, come results day, so as to protect a steep fall in the share price following a bad year; manage rating agencies’ view of the company and protect the current rating, to ensure stock analysts do not change their view from hold to sell! Managing shareholder income expectations from the dividend so as to keep their confidence in the company.

Is this the right way to interact with shareholders?

Uncertainty principles in action:

B.9 Conclusion

The problem at hand is a highly complicated one with numerous moving parts, and how they interact with each other. Capital problems, by nature, are more complex as they feed off the challenges on both assets and liabilities and there is never really a simple solution.

In reality, the inherent confidence in the 1 in 200 year risk may be driven by alignment/comparison with peers; pseudo confidence resulting from being consistently right (or wrong!) when the capital requirement is benchmarked against the market. A review of the model on its merits compared to own business profile may be useful. However, the validity of that may be challenged when done with an underlying motive of relieving the strain on the capital position.

When it comes to models, there is an element of unquantifiable uncertainty inherent in the real world, which cannot be reflected in the model. Over and above the capital requirements and solvency ratios, one should also consider what those are and, potentially, what the financial consequence those can have on the business. Perhaps, it is important to factor those into the decision-making process and not place sole reliance on the 1 in 200 year capital number.

The framework above is just an idea generation tool to constructively influence decision making and aims at bringing out a variety of factors/concerns to be considered. What is a viable solution would depend on the company’s philosophy and approach. The final answer depends on how all the moving parts interact with each other and can be reasonably justified within the realms of regulatory/business requirements and the over-arching professionalism requirement to protect the policyholders (first) and shareholders (second) in the long term.

We also hope that this problem helps readers to challenge certain aspects of business practices and attitudes.

Case Study C: I Disagree!

You are an advisor who has been asked to provide a valuation to assist with decision making.

There are three salient aspects of the exercise:

• It is the first time you have provided this advice;

• There is a deadline; and

• You disagree (materially) with existing advice or estimates. ⁹

The question to consider is how, as a professional, should you address the difference?

This case study seeks to explore some of the possible consequences for the different strategies that you might consider, and shows how the principles set out in this paper can be applied to help navigate the situation.

C.1 Why does this matter?

The reason for asking the questions is that while 9 times out of 10 the problem might go away, or never matter, when things go wrong you will inevitably be judged by others with the benefit of hindsight.

It is important for this case study that there will be insufficient time to achieve certainty, nor can you assume that there will be a repeat opportunity to address the problem. ¹⁰

So, while it is easy to respond that you would take some time to review the analysis, discuss the underlying assumptions with the person responsible for the estimates that you disagree with, an unstructured approach may not get to all of the issues in a timely fashion, and you may find yourself going round in circles, cause confusion, increase costs and give advice with so many caveats as to materially diminish its value.

Structure of case study

We start by outlining some bad and unstructured strategies, highlighting some of the pitfalls that might arise from adopting one or more of them. Then we set out how the principles in this paper can apply to the scenario, as a means of providing a more considered response that will enable the professional advisor to provide something of genuine value to decision makers.

While we think that all of the principles in this framework apply to this scenario, we think that the strategy that should be adopted as an advisor should be mindful of the potential long-term damage that can arise if a short-term or “path of least resistance” approach is followed. The Don’t be fooled principle explores some of the challenges that we can face as advisors, persuading ourselves that a particular course of action is appropriate.

C.2 Managing uncertainty, but without professionalism…?

Here is a list, possibly incomplete, of sub-optimal strategies that we identified have been adopted by others faced with this sort of situation. We describe them and include some observations ¹¹ regarding the issues associated with applying each in practice.

• “Conduct a limited review/methodology and assumptions/quick sense check review”, “Perform some agreed procedure tests”

  Don’t be fooled: Is this just code for “Look the other way”, “Don’t look too hard”?

  This approach basically accepts the existing work, or work performed by the other person as good enough.

  A variant of this is accepting arguments put forward from time to time that a piece of work “has been audited” or “looked at/benchmarked” by the auditors. While reliance upon audit is clearly important for advisors in many roles, often it is used to duck difficult decisions, or applied in inappropriate situations. Great care is therefore needed when advancing or responding to such an argument; we strongly counsel readers to proceed with caution when considering such an approach.

  Sometimes it is used as a non-confrontational solution, particularly where there are strong or dominant personalities involved. As a result, it is important to reflect upon the motivations of different parties involved: are they steering you away from a problem area, or legitimately trying to avoid needless repetition of recently performed work.

  This case can arise where the data is inadequate or substandard. Advisors may include caveats that there are issues with the data, but this can be highly dangerous where bad practice is going on, or a naïve approach to new areas of business.

  Advantages

  • ○ Minimum effort required.

  • ○ Limited direct personal responsibility.

  • ○ Don’t need to lie if you don’t believe the answer.

  Disadvantages

  • ○ You don’t get an answer that you can defend

  • ○ You aren’t adding any value

  • ○ You might be being pushed around/seen as a soft-touch where there are strong or dominant personalities

  • ○ You risk anchoring to the existing results, and not providing true independence of thought to the question at hand

  • ○ If it is wrong, you may still have to carry the can

  • ○ If you have a stake in the result being right, you lose out.

• Trade through it

  Think about adaptability and resilience: If the decision that is taken is essentially gambling on good news round the corner, then it is even more critical that decision makers understand the consequences of the bet not coming off.

  This is, wilfully or naively, hoping that future profits will offset known losses before the bad news actually becomes public. You know that your estimates are not really best estimates, but you think that you are smart enough to get away with it. This is, in our view, a highly risky approach, and one that can result in significant career damage if it goes wrong.

  Advantages

  • ○ Avoids a disagreement/fight over which estimate is right.

  • ○ The issue might not need addressing.

  Disadvantages

  • ○ It can go badly wrong (major career damage) if it comes out that you were covering up an issue that you were aware of.

• “Blow the whistle”

  Bring people with you (Part 1): Whistleblowing is an important option to have on the table, however its extreme nature means that having it as an option provides you with greatest influence. By its nature, blowing the whistle means that you have failed to bring people with you.

  Here you call out the issue and escalate it in a somewhat extreme fashion. This could result in demanding that a lot of additional work is performed (with possible cost consequences) or in a more extreme scenario, some form of whistleblowing activity.

  Advantages

  • ○ Seen to be doing the right thing.

  • ○ Default position of acting “with professionalism”

  • ○ Calling out incompetent or inadequate work is important professional principle.

  Disadvantages

  • ○ Highly confrontational approach.

  • ○ Whistle-blowers can find that there are adverse impacts to being seen as the one rocking the boat.

  • ○ False alarms/crying wolf can risk damaging your reputation.

  • ○ Might find yourself having to pick up the additional work (or cost) involved.

• “Quietly leave” or “Decline to act”

  Bring people with you (Part 2): Similar to scenarios where you blow the whistle, you find yourself unwilling or unable to bring others with you.

  Here you walk away, or find another excuse why you cannot take responsibility.

  Advantages

  • ○ Avoid taking the blame for an issue if it emerges subsequently.

  • ○ Low conflict approach

  Disadvantages

  • ○ May still get drawn in if major problems emerge; can you really walk away and claim no duty of care?

  • ○ Open to the challenge of “Why didn’t you tell anyone it was that bad”

  • ○ May require changing your role, job or losing an important client.

  • ○ Doesn’t really address the problem, just leaves it for someone else to deal with.

C.3 Better options – managing uncertainty with professionalism

Here we use the principles set out in this paper to provide a route-map, both to think through the situation, and to work with others to navigate the problem ( Bring people with you ).

First, you need to work out why the situation has arisen. There can be several possible explanations, so we think the first of our principles to use in this scenario is Deconstruct the problem.

1. (a) Different facts

   For example, one party has additional data, benchmark, analysis, etc., that has led them to a different result. Remember to be aware of biases (see below) where one side is making selective use of additional facts (i.e. that affect their result in one direction only).

2. (b) Different basis

   Has the work been performed on a different date, or excluded certain elements, ¹² that results in a different result?

   Are you talking to the one team about their review, when the results being used have been prepared by another (using a different basis)? ¹³

   Have you been asked to do a different exercise? Or is there a cultural/professional background difference? One known version of this is that US actuaries might produce a range of best estimates, whereas UK actuaries might produce a single point estimate with a confidence interval surrounding it.

   Another possible source of differences could be that Actuary A likes to produce best estimates that have no margin for survival bias in the underlying data (ENIDs), while Actuary B prefers to include this allowance. Or maybe Actuary A produces a mean estimate, including margins for downside risk, while Actuary B does not, leaving this as an add-on to be applied explicitly.

3. (c) Bias ( Don’t be fooled )

   Sometimes differences of opinion arise from inherent biases. This could be:

   • • deliberate – a “seller” (maybe in an M&A situation or an intermediary) wanting to take the most favourable interpretation;

     • commercially driven – an individual whose bonus is affected by the outcome; or

     • unconscious – a member of staff or a consultant wanting to believe the best for their employer or client even where they have no direct financial benefit. Sometimes the bias might even arise as a result of a desire to avoid conflict or alternatively to stimulate debate.

4. (d) An easy to resolve mistake

   This and the next point make use of the principle Models can be helpful, but also dangerous .

   Valuation models can be complex; could there be an error in a model, a portfolio omitted or a wording misunderstood?

5. (e) Poor quality underlying analysis.

   This is not unknown, particularly where the exercise is complex or time pressures are tight. It should be distinguished from mistakes, as this situation describes a failure to have fully understood the issue being addressed or failed to apply sufficient intellectual rigour.

   This can sometimes be tricky to unpick and may depend upon the personalities involved. Some practitioners can adopt a defensive position, trying to obfuscate or explain away why poor judgements taken are reasonable. Often a “running down the clock” strategy can be adopted making it virtually impossible to reach a satisfactory outcome.

   Sometimes, using multiple approaches can provide a helpful way to counter such arguments. This leads on to another variant of the “poor analysis” scenario. This is where the actuary has a preferred method, and over time develops a blind-spot to considering other approaches.

6. (f) Genuine uncertainty and different opinions in light of the facts received

   Once all of the reasons have been worked through and discounted, it is then possible to make progress. The point is that when there are differences of opinion arising from genuine uncertainty, then decisions can be taken and approaches adopted to suitably manage the uncertainty.

   These can lead to robust discussions, but framing the discussion around the sources of uncertainty can provide a much clearer way of working towards a decision. Here, it is important to apply the principle of Bring people with you , so that they understand the uncertainty and judgements being taken and what an estimate means. Such an approach enables decisions to be taken that Think about adaptability and resilience .

C.4 Holding two contradictory ideas in your head at the same time…

Once we have identified that the situation has arisen from a genuine difference of opinion in the face of uncertainty, then we need to determine how to give advice and take decisions. But taking decisions in the face of uncertainty is not easy, particularly conveying effectively the two messages of:

• This is my estimate.

• My estimate will definitely be wrong… and this is just how wrong my estimate might be.

A way to think about this apparent paradox is as follows: ¹⁴

“An advisor should imagine holding two pieces of paper, one in each pocket.

In the right pocket is written the idea that the advisor has a superhuman ability to provide an estimate: fixed, definite, certain. This enables accounts to be prepared, contract prices to be set and businesses to be built.

In the left pocket is written the idea that the advisor knows nothing. The uncertainty surrounding each situation dwarves any knowledge possessed, highlighting the inadequacy of the trade and curse of the forecaster. Any estimate will be wrong, and no reserve or capital held can provide certainty against all contingencies.

Sometimes the advisor will need to look more into the right pocket, and sometimes the left. The skill is to keep in mind both ideas and to know when and how much time to devote to each”

Essentially, we are faced with a form of cognitive dissonance, holding two contradictory ideas in our minds at the same time. While Face up to uncertainty does not take it away, it provides a legitimate rationale for seeking to strip away those elements that are not uncertainty. Once we have done this, it permits us to accept that there may well remain an area which is open for judgement and opinion.

To make progress, we then need to have a clear understanding of our purpose, coupled with humility to accept the limitations of what we can achieve. Adopting either extreme is unhelpful: bone-headed certainty is as dangerous as a weak-hearted attitude to uncertainty. To be successful and effective we need to attend to both.

There is no perfect answer to this, but using the above points will help. In summary:

Face up to uncertainty

Be honest about the limits of what you know, but do not let this prevent you placing a line in the sand with your opinion.

Deconstruct the problem

Break down problems into facts and assumptions.

Don’t be fooled

Be aware that it is easy to be steered to a wrong conclusion, by others … and yourself.

Models can be helpful, but also dangerous

Have a simple model to hand, not just a complex one.

Think about adaptability and resilience

Understand the role of margins and use them appropriately; and consider the consequences of future scenarios, even highly remote ones.

Bring people with you

While you must take accountability for your role in an exercise, balance this with the need to share the problem within your wider team. Tell them early, and often.