Proportion dominance in valuing lives: The role of deliberative thinking

Abstract

Proportion responding (PR) is the preference for proportionally higher gains, such that the same absolute quantity is valued more as the reference group decreases. This research investigated this kind of proportion PR in decisions about saving lives (e.g., saving 10/10 lives is preferred to saving 10/100 lives). The results of two studies suggest that PR does not stem from an overall tendency to choose higher proportions, but rather from faulty deliberative reasoning. In particular, people who display PR are less likely to engage in deliberative reflection as measured by response time, the Process Dissociation Procedure, the Cognitive Reflection Test, a numeracy test, and a task assessing denominator neglect. This association between faulty deliberation and PR was observed only when choosing the highest proportion was non-normative because it came at the expense of absolute gains (e.g., saving 10/10 lives is preferred to saving 11/100 lives). These results help to make sense of discrepant findings in previous research, pertaining to how PR relates to biased reasoning and decision making.

1 Introduction

People evaluate a program that can save 2 out of 4 people more favorably than a program that can save 2 out of 112 (Jenni & Loewenstein, 1997) because the proportion of the reference group that is saved is higher in the first case than in the second. This preference for the higher proportion can be so strong as to lead people to forego an option with the higher absolute number of gains. For instance, people have been shown to prefer saving 225 of 300 people rather than saving 230 of 920 (Bartels, 2006). This sensitivity to relative savings even when absolute savings are reduced has been termed proportion dominance (Bartels, 2006; Finucane, Peters & Slovic, 2003), the reference group effect (Jenni & Loewenstein, 1997), drop-in-the-bucket thinking (Bartels & Burnett, 2011), or psychophysical numbing (Friedrich et al., 1999; Slovic, 2007), and it is widely replicated (Baron, 1997; Bartels & Burnett, 2011; Erlandsson, Björklund & Bäckström, 2014, 2015; Fetherstonhaugh, Slovic, Johnson & Friedrich, 1997; Friedrich et al., 1999; Kleber, Dickert, Peters & Florack, 2013; Kogut & Beyth-Marom, 2008).

This kind of proportion responding (PR) has been discussed as a case that violates normative tenets of judgment and decision making, as the value of a life should not change as a function of the size of the reference group (Baron, 1997; Bartels, 2006; Fetherstonhaugh et al., 1997; Friedrich et al., 1999; Jenni & Loewenstein, 1997; Peters, Slovic, Västfjäll & Mertz, 2008; Slovic, 2007), and especially not to the point of justifying foregoing the chance to save more lives. Bartels (2006) found PR even when decision makers were in a joint evaluation mode (Hsee, 1996), and it was therefore clear that PR leads to saving fewer lives. Furthermore, Bartels found that, even though people recognize that PR is wrong in these cases when they are retrospectively asked to judge what weight should absolute vs. relative numbers have for their judgments, these retrospective judgments do not reflect people’s actual PR-biased judgments (as also found by Kogut & Beyth-Marom, 2008). In this paper, I use PR to refer to attention to proportion, and I distinguish between non-normative and normative types of PR, depending on whether PR leads to a worse outcome or not, respectively.

The present studies test whether PR stems from faulty thinking, that is, a failure to engage in deliberative thinking, or whether it originates in a general tendency to engage in proportional reasoning and prefer proportional gains over absolute ones.

1.1 Is PR related to faulty deliberation?

The findings from extant research are mixed in this regard. On the one hand, Bartels (2006) showed that PR correlates with a low disposition to think in a reflective manner (though those studies did not assess subjects’ actual ability to think reflectively, but rather relied only on a self-report measure of thinking style: the Rational-Experiential Inventory of Epstein, Pacini, Denes-Raj & Heier, 1996). On the other hand, Kleber et al. (2013) showed that PR is associated with having higher numerical reasoning skills. Adding to the mixed pile of findings, Peters et al. (2008) found that numeracy was not related to PR, although the precision of numerical representation influenced choices, albeit in a negative way: more precision was associated with choosing the proportionally superior but normatively incorrect option. Finally, Stanovich and West (2008) also found that PR is unrelated to cognitive ability as measured by subjects’ SAT scores.

One possible reason for these seemingly discrepant findings is that previous research has sometimes investigated PR when it clearly violates normativity, whereas in other studies this is not the case. Indeed, in many studies, PR does not necessarily constitute a violation of normativity, because both options are equal in absolute numbers and only vary in proportions, and therefore PR does not come at the expense of sacrificing a greater absolute number of lives (Jenni & Lowenstein, 1997; see also Erlandsson et al., 2014, 2015; Fetherstonhaugh et al., 1997, Studies 1 & 2; Friedrich et al., 1999; Kleber et al., 2013, Study 2). However, in other studies, PR is clearly non-normative: the option with the highest value of relative savings has the lowest value in absolute savings (Bartels, 2006; Fetherstonhaugh et al., 1997, Study 3; Kleber et al., 2013, Studies 1 & 3; Peters et al., 2008).

The present research tested whether the preference for higher proportions relates to intuitive thinking, as a function of whether it violates normative standards or not. Rather than expecting preference for higher proportions to be associated with intuitive reasoning overall, we expect that to be the case only when it is non-normative. This is because, in that case, the proportionally superior option should still be compelling, but people should realize that it is not correct (Bartels, 2006; Kogut & Beyth-Marom, 2008).

The present studies provide an exhaustive set of tests of how PR relates to intuitive thinking vs. careful deliberative reasoning. First, they test whether PR correlates with poor performance in other reasoning tasks: a choice task assessing denominator neglect (Bonner & Newell, 2010; Denes-Raj & Epstein, 1994), the Cognitive Reflection Test (CRT; Frederick, 2005), and numeracy (Lipkus, Samsa & Rimer, 2001). Furthermore, these studies examine response times and test whether non-normative PR is based on hastier responses, another indicator that careful reflection was not used in the decision-making process. In addition, the present research applies the Process Dissociation Procedure (Jacoby, 1991; Jacoby, Toth & Yonelinas, 1993) to assess to what extent controlled/deliberative thought processes (or a lack thereof) contribute to PR.

1.2 Is PR related to a tendency to think proportionally?

An alternative to the faulty deliberation hypothesis (though not mutually exclusive) is that PR stems from an overall tendency to think proportionally and always choose the higher proportion, regardless of whether that is the normatively correct option or not.

Studies on the development of mathematical reasoning reveal situations where people over-apply proportional reasoning: Van Dooren, De Bock, Hessels, Janssens, and Verschaffel (2005) found that 1) at early ages, students are not capable of reasoning proportionally; 2) later on, students become more capable of proportional reasoning, but they also start to over-apply it to situations where it is not called for, as in the following problem: “Ellen and Kim are running around a track. They run equally fast but Ellen started later. When Ellen has run 4 laps, Kim has run 12 laps. When Ellen has run 24 laps, how many has Kim run?” So, with increasing numerical reasoning capacity, students engage in more proportional reasoning, but they do so both when it is correct and when it is not. And 3) only at a later developmental stage do most people acquire the ability to judge whether they should use proportional reasoning or not.

That research further suggests that the proportional response might be so extensively practiced in school that it becomes the prepotent heuristic or intuitive response (Gillard, Van Dooren, Schaeken & Verschaffel, 2009), which must be overridden by effortful and time-consuming reflective thinking (see also Kogut & Beyth-Marom, 2008). Consistent with this hypothesis, Gillard et al. showed that constraining response time or imposing cognitive load on reasoners had little influence on their ability to answer correctly to problems where proportional thinking is appropriate, but it made them more likely to over-apply proportional thinking to problems where that kind of thinking is not appropriate (problems like the aforementioned one, about the two runners).

Therefore, it might be that PR results from a tendency to over-apply proportional reasoning. If that is the case, then we should find that the same people who are prone to PR also show a preference for proportionally superior options in other choice domains. In this regard, the denominator neglect task that is used in Study 1 provides an important test.

Denominator neglect can be instantiated in the following example: When people are asked to choose an urn from which to draw a ball that might grant them a prize, and they can choose from either a small urn with 1 prized ball out of 10 balls in total or a large urn with 9 prized balls out of 100, many people go for the option with the highest number of prized balls even though the odds of winning are lower (Bonner & Newell, 2010; Denes-Raj & Epstein, 1994). Avoiding this bias takes time (Bonner & Newell, 2010; Ferreira et al., 2016) and working memory resources (Ferreira, Garcia-Marques, Sherman & Sherman, 2006), which are characteristics of reflection/deliberation in dual-process theories.

In Study 1, subjects completed different tasks that assessed PR and denominator neglect. This offers a strong test of whether biased reasoning contributes to PR or not, given that PR and denominator neglect are opposite biases: PR consists of valuing the higher proportion and neglecting absolute frequencies, whereas denominator neglect consists of choosing the highest absolute frequency even when that does not correspond to the highest proportion. Therefore, if PR results from an overall tendency to engage in proportional reasoning, then it should even help to prevent denominator neglect. But if PR comes from a tendency to engage in a biased tendency to unreflectively follow the more compelling or intuitive answer, particularly when PR is not normative, then it should be associated with a tendency for denominator neglect. Indeed, Ayal, Hochman, and Zakay (2011) showed that people with a low disposition to engage in rational thinking (as measured by the Rational-Experiential Inventory) tend to commit such opposite biases.

2 Study 1

This first study tests whether PR is associated with quick response times, denominator neglect (an indicator of poor reflection but greater attention to proportions; Pacini & Epstein, 1999; Toplak, West & Stanovich, 2011), and lower scores on the controlled parameter of the Process Dissociation Procedure, depending on whether PR is normative or not.

The denominator neglect task was adapted so as to use the Process Dissociation Procedure, which has proven apt to dissect judgment and decision making biases and examine the extent to which automatic vs. controlled thought processes underlie them (Ferreira et al., 2006, 2016; Mata, Ferreira & Reis, 2013; Mata, Fiedler, Ferreira & Almeida, 2013). The logic of this procedure requires having harmony trials, where automatic and controlled processes suggest the same response, and conflict trials, where the different kinds of processes lead to different responses. The conflict trials are those that are typically used in this task, whereby the option with the higher numerator is different from the option with the higher proportion (e.g., 1/10 vs. 9/100). In harmony trials, on the other hand, the option with the higher numerator is the same as the option with the higher proportion (e.g., 1/10 vs. 11/100). Comparing performance in harmony and conflict allows for independent estimates of automatic and controlled processing in people’s reasoning (see Method section below). Therefore, the inclusion of the denominator neglect task enabled us to calculate the strength of subjects’ controlled and automatic thought processes in an independent task and then assess how well each of these predicts PR. It is expected that the estimate of automatic processing does not vary as a function of whether people show PR in their choices or not, suggesting that the proportion dominant choice is equally compelling for all responders regardless of whether they end up overriding that response tendency or not. Rather, PR should relate to lower controlled processing, especially when PR is non-normative and people should therefore override the tendency to choose the higher proportion.

The Process Dissociation Procedure enables a fine-grained distinction of the kinds of processes that contribute to PR. In previous research using self-report measures (Bartels, 2006), PR was examined in relation to a relative measure of the degree to which the rational subscale score outweighed the intuitive score. Therefore, it is not clear from that research whether PR is due to more rational/deliberative thinking or less intuitive thinking. Kogut and Beyth-Marom (2008) make the specific prediction that PR is a bias that results from “the intuitive, automatic system (…) which failed to be corrected by the analytical, rational system” (p. 604). Given that the Process Dissociation Procedure aims to assess the independent contribution of these kinds of thought processes, the procedure can provide a test of the association between PR and reduced deliberation.

Finally, this study assesses response time, as yet another measure of reflection. As with the other indicators of reflection, it is expected that response time relates to PR (i.e., PR should be associated with quicker responses), but only when PR is non-normative.

2.1 Method

Subjects.

One hundred Psychology undergraduates participated for partial course credit.

Procedure and design.

Subjects were given three scenarios to read (presented in a different random order for each subject) involving decisions to save the lives of people or animals (see Appendix A). As an example, in one of the scenarios, subjects were asked to imagine that two houses were on fire, and that they had to decide whether to save either all 10 people that were in a small house, or another 10 people from a total of 100 that were in a big house. In the non-normative condition, the second option involved saving 11 out of 100, such that it represented the lower proportion but the higher absolute number.

There were two conditions, counterbalanced between-subjects, which varied on whether PR was non-normative or not. Subjects were randomly assigned to either the non-normative PR condition (the scenarios were about saving 10/10 or 11/100) or the condition where PR did not violate normativity (the scenarios were about saving 10/10 or 10/100).

For the denominator neglect task, subjects were asked to imagine that they were playing a game where they could draw one ball from one of two trays containing red balls and white balls. To win this game, subjects were told that they would have to draw a red ball. They could choose to draw a random ball from a small tray (e.g., containing 1 red ball out of 10 balls in total) or from a large tray (e.g., containing 9 red balls out of 100 balls in total; see instructions in Appendix A).

Unlike the instructions for the proportion dominance task, which required reading detailed trial-specific scenarios, the instructions for the denominator neglect task were simple and constant across trials, which enabled subjects to quickly perform this task over many trials. We wished to have a high quantity and diversity of trials so that we could have reliable Process Dissociation estimates. Specifically, having many trials, and trials of different sorts, was meant to make it more tempting for subjects to go with their gut feeling in some trials and not always decide in a calculating fashion. Indeed, it is sometimes hard to obtain biased responses in this task with a small number of trials (e.g., Pacini & Epstein, 1999), and an invariant correct performance makes it difficult to estimate the automatic/heuristic parameter of the Process Dissociation Procedure (Ferreira et al., 2006).

There were several kinds of trials in the denominator neglect task (adapted from Bonner & Newell, 2010). The small tray had between 1 and 3 red balls out of 10 balls in total (i.e., 1/10, 2/10 or 3/10), and the large tray had a number of red balls out of a total of 100 such that the proportion of winning was 4% lower, 1% lower, equal, 1% higher, or 4% higher as compared to the small tray (e.g., 1/10 vs. 6/100, 1/10 vs. 9/100, 1/10 vs. 10/100, 1/10 vs. 11/100, 1/10 vs. 14/100). Crossing these factors produced 15 different trials: 6 harmony trials where the option with the larger numerator also corresponded to the larger proportion; 3 equal trials where the proportion was the same for both options; and 6 conflict trials where the option with the larger numerator was not the one with the larger proportion. Each trial was presented 17 times throughout the task. The resulting 255 trials were divided into 4 blocks, in the middle of which each of the 3 problems assessing proportion dominance was presented (i.e., 1 PR problem in each interval between blocks of the denominator neglect trials).

The A and C parameters of the Process Dissociation Procedure (referring to “automatic” and “controlled” processing, respectively) were calculated according to the following analytical procedure (Ferreira et al., 2006): In harmony trials, because automatic/intuitive and controlled/deliberative thought processes concur on the choice, the probability of choosing the option with both the larger numerator and the larger proportion is C + A (1 − C). In conflict trials, where intuitive and deliberative thought suggest different responses, the option with the larger numerator will be selected only if deliberative thought fails (as the proportion is smaller in this case), as a consequence of intuitive thought, and therefore the probability of choosing the larger numerator is A (1 − C). Based on the previous equations, it is possible to calculate C as the difference of probabilities of choosing the larger numerator in harmony versus conflict trials (i.e., p _{largenumerator/harmony} − p _{largenumerator/conflict}), and A as p _{largenumerator/conflict}/(1 − C).

2.2 Results and discussion

The proportion of problems in which subjects chose the PR option was higher when PR did not violate normativity than when it was non-normative, M = .62, SD = .29 vs. M = .35, SD = .38, t(90.61) = 3.94, p < .001. One-sample t tests comparing the number of PR choices against 50/50 revealed that most subjects preferred the PR option when it did not violate normativity, but chose against it when it was non-normative, ps ≤ .009.

Table 1 shows the correlations of normative and non-normative PR with the proportion of denominator-neglect trials where the subjects chose the option with the higher numerator across the three kinds of trials, the process dissociation parameters, and the average response times on PR problems (response times were log-transformed; see Ratcliff, 1993).

Table 1: Correlations of number of PR choices with other measures by normativity (p levels in parentheses).

Subjects who made more non-normative PR choices were also more likely to make more non-normative choices in the denominator neglect problems (i.e., choosing the higher numerator but lower proportion in conflict trials, and choosing the lower numerator and lower proportion in harmony trials). They also scored lower on the C parameter of the Process Dissociation Procedure. (The equivalent analysis for the A parameter revealed no significant effect.) And they took less time to make their choices in the PR task. In contrast, when PR was not against normativity, choices did not correlate with any of the other measures.

The results of Study 1 suggest that PR seems to result not from a general preference for higher proportions, but rather from faulty deliberative reasoning. That is, non-normative responding in one task correlates with non-normative responding in another task, whereas there was no evidence for a correlation based on differences in the tendency to respond in terms of proportions regardless of the task. Study 2 provides further tests of the deliberation-failure hypothesis.

3 Study 2

Whereas in Study 1 the normativity of PR was manipulated between subjects, in Study 2 all subjects solved PR problems where the PR option meant saving fewer lives or more lives overall. Respectively, these two kinds of problems constitute conflict vs. harmony trials, and comparing choices across them makes it possible to compute Process Dissociation estimates. Moreover, this is a better way to manipulate normativity than the normative condition used in Study 1, where, even though PR did not entail sacrificing more lives (i.e., both options involved the same absolute number, only the proportions varied), it was not clear what the correct option was, as there was not an option with a higher absolute number. Subjects also completed the CRT and a numeracy scale.

Several hypotheses were tested: First, if people recognize the non-normativity of PR in conflict trials, where PR implies saving fewer lives, then PR should be lower in those trials than in harmony trials. This would replicate the between-conditions difference that was found in Study 1. Second, as the CRT and the numeracy scale measure the tendency to think deliberatively about the problem at hand, scores on these tests should relate negatively to PR in conflict trials (i.e., non-normative PR) and positively to the C scores of the Process Dissociation Procedure (i.e., the ability to choose PR depending on whether it is normative or not). Finally, in terms of response times, in harmony trials, deciding quickly or deliberatively should both lead to the same decision: choosing the PR-option. However, in conflict trials, one should take longer to reflect on whether or not the PR-option is correct and give the other response. Therefore, just as in Study 1, decision time should not predict choices in harmony trials, but it should do so in conflict trials, with normative responses (to not choose the PR option) taking more time than non-normative ones.

3.1 Method

Subjects.

One hundred subjects were recruited through Amazon’s Mechanical Turk (104 ended up participating, as 4 subjects did not sign up for compensation and were therefore not registered on Mechanical Turk as having taken part in the study). Subjects were located in the United States and were required to have an approval rate in previous assignments of at least 95%.

Procedure and design.

Subjects were given ten scenarios to read, involving decisions to save the lives of people or other species (some of these were adapted from Bartels, 2006; see Appendix B). For half of them – the conflict trials – the PR option was different from the option where the highest absolute number of lives could be saved. For the other half – the harmony trials – the PR option was also the option where the highest absolute number of lives could be saved. The different scenarios were presented in random order.

After the scenarios, subjects responded to an adapted version of the CRT and to the Lipkus et al. (2001) Numeracy Scale (Appendix B).

3.2 Results and discussion

In harmony trials, where PR was the normative option, the proportion of PR choices was higher than in conflict trials, where PR violated normativity, M = .92, SD = .17 vs. M = .52, SD = .38, paired t(104) = 10.05, p < . 001.

Table 2 shows the correlations of CRT, numeracy and average (log-transformed) response times with normative and non-normative PR.

Table 2: Correlations of number of PR choices with other measures by normativity (p levels in parentheses).

Note. The mean response time in the top rows is specific to type of trial (non-normative or normative PR). In the bottom rows it is averaged across trials.

Subjects who made more non-normative PR choices scored lower on both the CRT and the numeracy test, and they also took less time to make their choices. In contrast, in harmony trials, PR choices correlate only with numeracy, and not with CRT score, nor with response time. Sinayev and Peters’ (2015) indices of calculation (i.e., number of correct responses) and reflection (i.e., number of non-intuitive responses, both correct and incorrect) do not differ in terms of the correlations between CRT and the other measures.

Additionally, having higher C Process-Dissociation scores correlated with taking longer to decide on the PR scenarios overall, whereas the A scores did not (see Table 2).

4 General discussion

This research investigated people’s preference for relative gains (i.e., PR) in decisions about saving lives. Even though many people refrained from showing PR when it came at the expense of absolute gains – in this particular case, at the expense of lives – a large percentage of subjects showed PR even when that was the case (35% in Study 1, and 52% in Study 2). Corroborating the non-normative nature of PR in this case, subjects who chose higher proportions when that meant saving fewer lives overall were also more likely to display biased decision making in another domain (denominator neglect; Study 1); they manifested lower engagement in deliberative reflection, as measured by response times (Studies 1 and 2), the C score of the Process Dissociation Procedure (Study 1), the CRT and a numeracy scale (Study 2). Furthermore, the ability to choose for or against PR depending on whether it is normative or not (as measured by the C parameter of the Process Dissociation Procedure) correlated positively with measures of deliberative reasoning, such as taking longer to respond and scoring higher on the CRT and numeracy (Study 2).

These results, together with the fact that most people prefer the PR option when it does not violate normativity (e.g., preferring to save 10/10 instead of 10/100 in Study 1), support the notion that PR is a compelling prepotent response tendency, and that deliberative reasoning is required in order to judge its appropriateness and override it, if need be (Gillard et al., 2009; Kogut & Beyth-Marom, 2008; Van Dooren et al., 2005).

This deliberation-failure hypothesis seems to account better for PR than an overall preference for higher proportions. Indeed, if it were the case that PR comes from an overall tendency to think proportionally, then those subjects who displayed PR in Study 1 should be less likely to show denominator neglect, but the opposite was the case.

These results help to make sense of some seemingly discrepant findings in previous research, with regard to whether PR relates to biased reasoning and decision making. Those studies considered only scenarios where PR was normative or scenarios where it was not normative. Results were therefore mixed (cf. Bartels, 2006; Kleber et al., 2013; Peters et al., 2008; Stanovich & West, 2008). The present studies systematically investigated the role of deliberative thinking in PR depending on whether PR is normative or not. The results form a consistent picture. To summarize: PR is a compelling prepotent response option (Gillard et al., 2009; Kogut & Beyth-Marom, 2008). People can override this tendency and forego PR when it is not normative, but this requires the ability to engage in careful deliberation.