Kahneman (Reference Kahneman2011) defined the heuristics used in decision making as substituting a difficult to answer question with an easy question to answer. For example, the availability heuristic substitutes the hard question of how large a category is or how frequent an event is, with the simpler question of how easily instances come to mind. In John et al.'s terms, the answer to such a simpler question is a proxy for the harder question. Thinking of heuristics as proxies provides a different perspective on decision-making heuristics and on proxy failure. John et al. invite the reader to identify other phenomena that have similar characteristics to proxy failure; so what can examining heuristics in terms of proxy failure tell us about both?
John et al. identify the elements of proxies and we can map these to heuristics. For the availability heuristic the regulator would appear to be a person's decision-making system, the goal is to determine frequency, the agent is a person's memory, and the proxy is ease of bringing instances to mind. The type of fast and frugal heuristics found in Gigerenzer and Todd (Reference Gigerenzer and Todd1999) also map to these elements. For example, they write about a successful heuristic for determining treatment of heart attack patients that asks at most three yes/no questions related to tachycardia, age, and blood pressure. If a hospital requires doctors to use this treatment heuristic, then in terms of proxies, the hospital is the regulator, the goal is saving lives, the agent is a doctor, and the proxy is the rule. (Although John et al. say that proxies are typically expressed as scalars there does not seem to be any conceptual reason why proxies could not be simple rules.)
The three limitations on regulator and agent that John et al. point to also apply to heuristics. First, there is restricted legibility of goals, because the success of heuristics is often difficult to directly observe. Second, they often make imperfect predictions because of their own limitations and because the world does not supply all the information needed. Instead they satisfice (Simon, Reference Simon1955). Third, there is a necessity to choose, which is why we have heuristics.
Decision-making heuristics by definition are not guaranteed to succeed. So do they fail for the same reasons as proxies? John et al. suggest that proxy failure occurs under two conditions:
(1) There is regulatory feedback based on the proxy that has consequences for agents and
(2) the system is sufficiently complex such that there are multiple paths to the proxy that are partially independent of the goal.
The second condition appears to be necessary for both proxy and heuristic failure. For example, the body's response to heart attack can be complex so the treatment heuristic referred to above can fail because there is more than one reason why blood pressure is high. However, the first condition suggests that proxy failure is dynamic in a way that heuristic failure is not normally. For example, Goodhart's law seems to apply when the proxy starts as a good indicator of the behavior it is designed to measure, but becomes less effective as the agents learn how to game the system. Heuristics such as availability may fail, but they do not appear to degrade over time. Heuristics can fail because of flaws in the proxy they use or when the environment is manipulated to trip them up, such as when excessive media coverage of an event leads the availability heuristic to draw incorrect conclusions about the frequency of an event. Famously Tversky and Kahneman (Reference Tversky and Kahneman1974) established the existence of heuristics by demonstrating conditions under which they fail, but presumably the heuristics that we persist with are ones that often succeed. If a heuristic degraded because of its use then it is likely it would fade from use in a way that common heuristics have not.
What often appears to lead to the regulatory feedback producing proxy failure is that there is a divergence of the goals of the regulator and the agent. In the example of the rat plague that opens John et al.'s paper, the rat tails may have worked reasonably well as a proxy for reducing the rat population if the residents of Hanoi had cared as much about reducing that population as the colonial officials did. Instead, the officials’ goal of reducing the number of rats ran up against the residents’ stronger goal of preserving a steady source of income. Successful heuristics may be successful partly because they avoid such divergence of the regulator and agent goals. For example, the memory system does not have goals that undermine the decision-making system's goals so availability can succeed; and the hospitals and doctors presumably have somewhat similar goals in wanting to treat people who will benefit the most, so the treatment heuristic succeeds.
Looked at this way, proxies are an attempt to bring the agent's goals into alignment with the regulator's goals. If the regulator's and the agent's goals are difficult to align then the proxy is in effect an attempt to impose the regulator's goals on the agent, and thus it is likely to fail if the agent has means to resist this. This suggests that an important factor determining how successful a proxy will be is how well aligned the goals of the regulator and agent are.
Examining how the phenomenon of proxy failure relates to decision-making heuristics can broaden the scope of John et al.'s analysis and says something about both heuristic and proxy failure. For heuristics, this comparison makes clear something that is not otherwise readily apparent, that the success of heuristics depends in part on the goals of the regulator and the agent being aligned, or at least not in conflict. For proxy failure this comparison suggests that failure may sometimes be less because of characteristics of proxies than to them being used to impose the regulator's goals onto the agent. So the solution to proxy failure may not be a better proxy but instead be better alignment of goals.
You have
Access
Kahneman (Reference Kahneman2011) defined the heuristics used in decision making as substituting a difficult to answer question with an easy question to answer. For example, the availability heuristic substitutes the hard question of how large a category is or how frequent an event is, with the simpler question of how easily instances come to mind. In John et al.'s terms, the answer to such a simpler question is a proxy for the harder question. Thinking of heuristics as proxies provides a different perspective on decision-making heuristics and on proxy failure. John et al. invite the reader to identify other phenomena that have similar characteristics to proxy failure; so what can examining heuristics in terms of proxy failure tell us about both?
John et al. identify the elements of proxies and we can map these to heuristics. For the availability heuristic the regulator would appear to be a person's decision-making system, the goal is to determine frequency, the agent is a person's memory, and the proxy is ease of bringing instances to mind. The type of fast and frugal heuristics found in Gigerenzer and Todd (Reference Gigerenzer and Todd1999) also map to these elements. For example, they write about a successful heuristic for determining treatment of heart attack patients that asks at most three yes/no questions related to tachycardia, age, and blood pressure. If a hospital requires doctors to use this treatment heuristic, then in terms of proxies, the hospital is the regulator, the goal is saving lives, the agent is a doctor, and the proxy is the rule. (Although John et al. say that proxies are typically expressed as scalars there does not seem to be any conceptual reason why proxies could not be simple rules.)
The three limitations on regulator and agent that John et al. point to also apply to heuristics. First, there is restricted legibility of goals, because the success of heuristics is often difficult to directly observe. Second, they often make imperfect predictions because of their own limitations and because the world does not supply all the information needed. Instead they satisfice (Simon, Reference Simon1955). Third, there is a necessity to choose, which is why we have heuristics.
Decision-making heuristics by definition are not guaranteed to succeed. So do they fail for the same reasons as proxies? John et al. suggest that proxy failure occurs under two conditions:
(1) There is regulatory feedback based on the proxy that has consequences for agents and
(2) the system is sufficiently complex such that there are multiple paths to the proxy that are partially independent of the goal.
The second condition appears to be necessary for both proxy and heuristic failure. For example, the body's response to heart attack can be complex so the treatment heuristic referred to above can fail because there is more than one reason why blood pressure is high. However, the first condition suggests that proxy failure is dynamic in a way that heuristic failure is not normally. For example, Goodhart's law seems to apply when the proxy starts as a good indicator of the behavior it is designed to measure, but becomes less effective as the agents learn how to game the system. Heuristics such as availability may fail, but they do not appear to degrade over time. Heuristics can fail because of flaws in the proxy they use or when the environment is manipulated to trip them up, such as when excessive media coverage of an event leads the availability heuristic to draw incorrect conclusions about the frequency of an event. Famously Tversky and Kahneman (Reference Tversky and Kahneman1974) established the existence of heuristics by demonstrating conditions under which they fail, but presumably the heuristics that we persist with are ones that often succeed. If a heuristic degraded because of its use then it is likely it would fade from use in a way that common heuristics have not.
What often appears to lead to the regulatory feedback producing proxy failure is that there is a divergence of the goals of the regulator and the agent. In the example of the rat plague that opens John et al.'s paper, the rat tails may have worked reasonably well as a proxy for reducing the rat population if the residents of Hanoi had cared as much about reducing that population as the colonial officials did. Instead, the officials’ goal of reducing the number of rats ran up against the residents’ stronger goal of preserving a steady source of income. Successful heuristics may be successful partly because they avoid such divergence of the regulator and agent goals. For example, the memory system does not have goals that undermine the decision-making system's goals so availability can succeed; and the hospitals and doctors presumably have somewhat similar goals in wanting to treat people who will benefit the most, so the treatment heuristic succeeds.
Looked at this way, proxies are an attempt to bring the agent's goals into alignment with the regulator's goals. If the regulator's and the agent's goals are difficult to align then the proxy is in effect an attempt to impose the regulator's goals on the agent, and thus it is likely to fail if the agent has means to resist this. This suggests that an important factor determining how successful a proxy will be is how well aligned the goals of the regulator and agent are.
Examining how the phenomenon of proxy failure relates to decision-making heuristics can broaden the scope of John et al.'s analysis and says something about both heuristic and proxy failure. For heuristics, this comparison makes clear something that is not otherwise readily apparent, that the success of heuristics depends in part on the goals of the regulator and the agent being aligned, or at least not in conflict. For proxy failure this comparison suggests that failure may sometimes be less because of characteristics of proxies than to them being used to impose the regulator's goals onto the agent. So the solution to proxy failure may not be a better proxy but instead be better alignment of goals.
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Competing interest
None.