Motivated reasoning when assessing the effects of refugee intake

Abstract

Do differences in worldview ideology hinder people from objectively interpreting the effect of immigration? In an experiment with Swedish adults (n = 1015), we investigate whether people display motivated reasoning when interpreting numerical information about the effects of refugee intake on crime rate. Our results show clear evidence of motivated reasoning along the lines of worldview ideology (i.e., whether people identify themselves primarily as nationally oriented or globally oriented). In scenarios where refugee intake was associated with higher crime rate, nationally oriented people were 18 percentage points more likely to make the correct assessment compared to globally oriented people. Likewise, in scenarios where refugee intake was associated with lower crime rate, nationally oriented people were 20 percentage points less likely to make the correct assessment compared to globally oriented people. Individuals with higher numeric ability were less likely to engage in motivated reasoning, suggesting that motivated reasoning more commonly is driven by feelings and emotional cues rather than deliberate analytical processes.

Introduction

We all sometimes believe what we want to believe. Parents often think that their own children are amazing in all areas. The authors of this paper think the same about themselves, even though their “brilliant” paper keeps getting rejected. Most of us would consider this type of motivated reasoning a rather harmless – and sometimes even endearing – feature of human behavior. However, for matters at the public level, such as immigration and climate change, holding rigid and uniformed beliefs can lead to harmful group delusions. In an experiment, we explore whether people engage in motivated reasoning when they assess numerical information about the effects of refugee intake on crime rates. Furthermore, we investigate the role of numeric ability on motivated reasoning. We show that motivated reasoning asserts itself as polarized assessments in line with people's worldview ideology (i.e., whether people identify themselves primarily as Swedes [i.e., nationally oriented] or primarily as world citizens [i.e., globally oriented]). Our results also provide suggestive evidence that numeric ability can serve as a buffer against motivated reasoning.

When people engage in motivated reasoning, they evaluate information in a biased manner in order to support a particular belief or idea rather than objectively considering information (Lord et al., 1979; Kunda, 1990). In theoretical models about decision-making, it is commonly assumed that people update their beliefs according to some updating rule, such as Bayes’ rule, in the light of new information. Motivated reasoning thus implies that if (motivated) beliefs are strong, this updating process will not take place; instead, people make the information fit their beliefs. In the decision-making process, this translates into people making trade-offs between accuracy and desirability (Bénabou, 2015). This contrasts with both economic theory and decision theory, which assume agents objectively interpret information and make optimal judgments based on the facts at hand.

Previous studies have demonstrated motivated reasoning in relation to, for example, climate change (Drummond & Fischhoff, 2017) and gun control (Kahan et al., 2017). Studies investigating motivated reasoning in the context of refugee intake are scarce.¹ Moreover, existing studies on motivated reasoning have almost exclusively used American samples and focused on polarization in relation political or religious affinities. Thus, there is a need to assess the occurrence of motivated reasoning in other areas and for other underlying ideological constructs. In this study, we use a Swedish sample and focus on motivated reasoning that conforms with one's worldview ideology when assessing the information about the effects of refugee intake on crime rate. Moreover, we explore the role of numeric ability in motivated reasoning. We juxtapose the idea of motivated reasoning-as-analysis (suggesting that individuals with higher numeric ability are more likely to show motivated reasoning) with the idea of motivated reasoning-as-feelings (suggesting that people with higher numeric ability are less likely to engage in motivated reasoning).

The large influx of refugees, primarily from Syria, in recent years² has intensified the political debate and ideological polarization related to immigration and refugee intake worldwide. On the one hand, people have a supportive view of refugee intake and immigration and see it as a way of taking global responsibility and a necessary ingredient for a cosmopolitan society. On the other hand, people propose more restrictive strategies for refugee intake and immigration on the basis that cultural and religious differences spur conflict and harm the host country. Within political psychology, it is a well-established fact that partisan groups evaluate information about the initiatives of political parties or candidates they (dis)like asymmetrically (see e.g., Lebo & Cassino, 2007; Bolsen et al., 2014). Partisan motivated reasoning thus refers to when people evaluate information so that it aligns with their party commitment. In Sweden, where the current study is conducted, the polarization in opinions about refugee intake has not been reflected clearly along traditional political lines (Green-Pedersen & Otjes, 2017). Instead, polarization has arguably been more pronounced with respect to worldview ideology³ (i.e., whether people identify themselves primarily as patriots or world citizens). Several international political initiatives have further deepened this ideological divide. For example, the American President's electoral pledge to build a wall along the southern border, the travel ban into the USA, and Brexit are all recent controversies relating to immigration and refugee policies that have fractionalized entire countries.

Motivated reasoning-as-analysis or motivated reasoning-as-feelings?

When thinking about why people are sometimes willingly drawn to distorted beliefs, theories about belief-based utility provide a natural point of departure. According to such models, people derive utility from maintaining beliefs, much like they derive utility from consumption and other types of behaviors that they willingly engage in (Grant et al., 1998; Bénabou, 2015; Golman et al., 2016; Hagmann & Loewenstein, 2017). The idea of belief-based utility also implies that there may be a substantial cost involved in changing a belief. As Hagmann and Loewenstein (2017) point out, “people often enjoy immediate benefits from holding (or professing to hold) a particular belief, especially to the extent that others share it.” In other words, when individuals belong to a certain community, it could be beneficial to maintain false beliefs that are aligned with the community, particularly if this makes it easier to fit in or uphold status within the group (see e.g., Van Bavel & Pereira, 2018). Thus, an important reason for why people engage in motivated reasoning is simply that it in many situations maximizes their utility. A related reason for why people engage in motivated reasoning stems from the disutility that arises when they experience cognitive dissonance. When beliefs and personal values or feelings are in conflict, individuals typically experience a state of psychological discomfort (cognitive dissonance; Festinger, 1976). To avoid this discomfort, individuals may unconsciously interpret information in a way that reduces the discrepancy between beliefs and personal values. Instead of analytically processing new information, people might, for example, rely on a “how do I feel about it” heuristic (Slovic et al., 2007). That is, the feelings of the relative “goodness” or “badness” of the information determines whether you think it is true or false.

Turning to the question of how people engage in motivated reasoning, this involves both the processes of how people recruit information and how they evaluate it. Because people attach value to particular beliefs, they develop strategies in order to block or manage information that threatens the validity of them. We avoid undesirable information. For example, people might unfriend or unfollow people that voice conflicting opinions on social media or avoid taking medical tests because they dread the potential result. There is now a large literature on information avoidance, showing that people willingly often avoid useful and non-costly information (see e.g., Karlsson et al., 2009; Hertwig & Engel, 2016; Golman et al., 2017). Importantly, however, motivated reasoning takes place also when information is evaluated, implying that people interpret the same information differently depending on their prior beliefs.

The experimental task in this study is numerical in character. In an ideologically non-polarizing context, it would be natural to assume that higher numeric ability should make people better at accurately interpreting numerical information. In a polarizing context, however, conflicting hypotheses arises. Following dual-process models, human cognition can be characterized by a distinction between intuition (“System 1”) and reflection (“System 2”; Epstein, 1994; Stanovich & West, 1998; Chaiken & Trope, 1999; Kahneman, 2011). Intuitive processes are typically characterized as fast, automatic, effortless, and based on feelings. Reflective processes are characterized as slow, controlled, effortful, and based on analysis. Thus, we shall refer to these conflicting hypotheses as motivated reasoning-as-analysis and motivated reasoning-as-feelings. According to the former, motivated reasoning is primarily driven by deliberate analytical processes where people seek to maximize their own utility by falsifying information that threatens valued beliefs. Consequently, people with higher numeric ability should be better equipped to reason their way around information in conflict with their valued beliefs. The findings from Kahan et al. (2017) support this account of motivated reasoning. In the politically dividing issue of gun control, they showed that people with high numeric ability were more likely to engage in motivated reasoning compared to people with low numeric ability (Kahan et al. [2017] call this motivated numeracy).

According to the motivated reasoning-as-feelings hypothesis, the information processing is primarily driven by intuitive responses. When faced with new information, motivated reasoning happens automatically: people use emotional cues that allow them to avoid the mentally taxing task of forming a judgment based on deliberation. For example, people might rely on a “how do I feel about it” heuristic when processing new information (Slovic et al., 2007). Consequently, people with low levels of numeric ability should be less able to detect and override incorrect responses from their intuition and people with higher numeric ability should be better at overriding feelings in favor for an objectively correct answer. In support of this account of motivated reasoning, Pennycook and Rand (2018) found that people who performed better on the Cognitive Reflection Task were better at detecting fake news, regardless of whether the presented news was consistent or inconsistent with their political views.

Methods

Sample

An online experiment was conducted with a diverse sample of the adult Swedish population.⁴ The experiment was administrated by CMA Research in December of 2016. We requested 1000 respondents from CMA Research, and when this quota was filled, with satisfactory spread of the demographic variables, the web link was canceled. The participants in CMA Research's subject pool are recruited in several ways: via telephone, online, or face-to-face. The study was part of a broader data collection process consisting of 11 sections, several of which were included as part of other research projects and therefore are reported elsewhere (Erlandsson et al., 2018; Nilsson et al., 2018). An overview of the experiment can be found in the Supplementary Materials (available online). In total, 1015 subjects participated in the experiment (49.9% female; mean age was 48.9 years, SD = 15.5). Participants received a small monetary compensation, approximately 1.5 USD. The sample was representative of the general population in Sweden with regard to gender and age; however, the sample was more educated than the general population.⁵ The sample composition is shown in Table S1 in the Supplementary Materials.

Experimental scenarios

Four experimental scenarios were created. These were labeled “crime increase,” “crime decrease,” “rash increase,” and “rash decrease.” Each scenario is shown in Figure 1. The complete experimental instructions can be found in the Supplementary Materials. Common to all scenarios was that they involved a fictitious study and that the results from the fictitious study were presented numerically in a table.

Figure 1. Experimental scenarios

The crime scenarios represent the ideologically polarizing context. In these scenarios, participants’ task was to indicate what the numerical information suggested about the effect of refugee intake on crime rate in Norwegian communities. Participants were asked to interpret one table containing numerical information. As seen in Figure 1, each table included four values that showed the number of Norwegian communities that did and did not receive refugees and that also experienced an increasing crime rate or a decreasing crime rate. This presentation required participants to consider the relevant ratios rather than the absolute numbers presented in the table, thus requiring participants to use their numeric ability. Receiving refugees either increased or decreased the crime rate. The participants answered by indicating either “It is likely that crime rate has decreased over the last five years in Norwegian communities that have received refugees, compared to communities that did not,” or “It is likely that crime rate has increased over the last five years in Norwegian communities that have received refugees, compared to communities that did not.” In order to increase the chances of respondents using the provided information rather than relying on what they think about the situation in Sweden, the instructions indicated that the study had taken place in Norway. Participants responded to either the “crime increase” or the “crime decrease” scenario, but not both.

The rash scenarios represent the ideologically non-polarizing context. These scenarios were adapted from Kahan et al. (2017) and translated into Swedish. In the rash scenarios, participants’ task was to indicate what the numerical information suggested about the effect of a skin cream on a rash. The skin cream either reduced or increased the skin problems. Participants answered by indicating either “It is likely that patients who used the new cream reduced their skin problems, compared to those who did not use it,” or “It is likely that patients who used the new cream got increased skin problems, compared to those who did not use it.” Participants responded to either the “rash increase” or the “rash decrease” scenario, but not both.

At the start of the experiment, participants were randomly shown one of the four scenarios.⁶ Participants then went on to the other sections of the experiment. Toward the end of the experiment, participants were presented with another scenario from Figure 1 in a counterbalanced order. Hence, at the end of the experiment, all participants had responded to one of the rash scenarios and one of the crime scenarios. Thus, all participants responded to one of the non-polarizing skin cream scenarios and one potentially polarizing immigration scenario.

Numeric ability

To measure numeric ability, we used a combination of items from Schwartz et al. (1997), the Berlin Numeracy Test (BNT) developed by Cokely et al. (2012), and three items from Frederick (2005).⁷ Table 1 displays all of the items used to elicit numeracy together with the percentages of correct answers to each specific item. In total, we used six items, all of which have previously been used to measure numeracy (Weller et al., 2013). The α-score was 0.79, which suggests an acceptable internal validity. A factor analysis also confirmed that the items loaded on one underlying construct. The numeracy score varied between zero and six. On average, participants answered 2.3 questions correctly (SD = 1.96). Participants could respond “don't know”; these answers were treated as incorrect. The distribution of the numeracy score can be found in Table S1 in the Supplementary Materials.

Table 1. Questions used to elicit numeric ability and percentages of participants who answered correctly

Note: Cronbach α = 0.79.

Worldview ideology

Worldview was measured by asking participants whether they considered themselves as Swedes or world citizens. Participants were given the following definitions of the two concepts:

People sometimes talk about whether they view themselves as “world citizens” or as “Swedes”. People who view themselves as “world citizens” believe that all people should have equal rights and that we have a common responsibility to help everybody, no matter their nationality. People who view themselves as “Swedes” believe that Swedish citizens should have some privileges, and that we have a greater responsibility to help other Swedes than people from other countries.

If one uses the definition above, would you describe yourself as more of a “world citizen” or as more of a “Swede”?

Participants answered on a seven-point Likert scale, with 1 being “I'm much more of a world citizen” and 7 being “I'm much more of a Swede.” The answers roughly followed a normal distribution with a slight skew toward world citizenship. A total of 40.3% of participants leaned toward world citizenship, 21.5% were neutral and about 38.2% leaned toward being more of a Swede. Since we are interested in the divide between people with opposing worldviews, the variable was dichotomized. Therefore, in the main analysis, participants who answered 1–3 on the Likert scale are referred to as globally oriented and participants who answered 5–7 on the Likert scale are referred to as nationally or Swedish oriented. The remaining participants were excluded from the main analysis. In the Supplementary Materials, we include an analysis treating the worldview variable as continuous (see Table S2 and S3 for regression results); the results are similar to the ones presented in the main analysis.

Political orientation

Two questions were used to elicit political orientation. The results from both questions are shown in Table 2. First, participants were asked to indicate whether they consider themselves politically left or right oriented on a nine-point Likert scale, ranging from very far left to very far right. The distribution followed a normal distribution with 6.8% considering themselves as very far or far left, 29.6% weakly or somewhat left, 24.7% considered themselves as not leaning in any direction, 32.2% weakly or somewhat right, and 6.7% far or very far right. Second, participants were asked what political party they would vote for if an election was held today. They could select any of the eight political parties represented in parliament (at the time of the study) or write down the name of any other party. A total of 19.7% chose not to answer this question or did not know which party they would vote for.

Table 2. Worldview ideology and political orientation depending on political party affiliation

The correlation between political orientation, measured as left–right orientation, and worldview ideology was 0.40 in the sample. Table 2 shows the percentages of people reporting being more nationally oriented or more globally oriented depending on political party affiliation. For example, 50.8% of those who would vote for the Moderate Party would classify themselves as nationally oriented. However, the Moderate Party is a clearly self-defined right-wing party, with almost 85.5% classifying themselves as politically right oriented. Similar discrepancies between political orientation and worldview ideology are observed for the other political parties. Since these correlations are far from perfect, they support our decision to investigate several underlying affinities.

Since we are interested in the divide between people with opposing political orientations, the variable was dichotomized. In the main analysis, participants who answered 1–4 on the Likert scale are referred to as politically left oriented. Participants who answered 6–9 on the Likert scale are referred to as politically right oriented. The remaining participants were excluded from the main analysis.

Strategy of analysis

The strategy of analysis is straightforward. First, we investigate the presence of motivated reasoning when assessing the effects of immigration, and second, we investigate the role of numeracy in such evaluations. As a preliminary analysis, we graphically present our results. We then elaborate on the results by conducting regression analyses.⁸ The dependent variable in all regressions is the assessment made by participants of the numerical information in a particular experimental scenario. It is a binary variable, taking the value of 1 if the correct assessment was made and 0 otherwise. The explanatory variable of main interest is worldview ideology. To investigate the contradicting hypotheses with regard to numeracy, both numeracy score and an interaction term between worldview ideology and numeracy score are added to the model specification.

Hence, the first specification is simply:

$$correct\; assesment_i = \beta _0 + \beta _1nationally\; oriented_i + \beta _x^{\prime} X + u_i$$

The second specification is:

$$\eqalign{correct\; assesment_i & = \beta _0 + \beta _1nationally\; oriented_i \cr & \quad + \beta _2numeracy_i + \beta _3nationally\; oriented_i \times numeracy_i \cr & \quad + \beta _x^{\prime} X + u_i} $$

If the interaction term in the second specification is positive for the crime increase scenario (negative for crime decrease scenario), then the motivated reasoning-as-analysis is supported, and vice versa for motivated reasoning-as-feelings. Should the interaction term not be statistically significant, then people with higher and lower levels do not differ in their likelihood of engaging in motivated reasoning. The control variables are represented by the vector X, which includes a control for possible order effects of the experimental scenarios (since participants responded to two experimental scenarios, one polarizing immigration scenario and one non-polarizing skin cream scenario) and also controls for gender, age, and education. Education was measured by participants indicating one of five alternatives ranging from “not completed compulsory school” to “graduate from university or collage.” The complete distribution of all of the control variables can be found in Table S1 in the Supplementary Materials.

As a robustness check, we replicate our analyses using political orientation rather than worldview ideology. These analyses can be found in the Supplementary Materials. In general, they support the findings presented below, but are less pronounced.

Results

Motivated reasoning

The percentages of participants that were able to correctly assess the numerical information varied between 51.2% and 58.2% across the four scenarios. This result could be viewed as rather disconcerting since it implies that flipping a coin is almost as good as trusting the advice of a random stranger when it comes to interpreting this type of numeric information. However, the percentages of correct assessments are similar, and even higher, than what Kahan et al. (2017) observed in their study.

Figure 2 shows the percentages of correct assessments in each experimental scenario conditional on worldview ideology. In the crime scenarios, the ability to correctly assess the numerical information clearly differed depending on worldview ideology. Among those who classified themselves as nationally oriented, 69.3% made the correct judgment when crime rates increased following refugee intake, while only 50.5% of those who classified themselves as globally oriented correctly assessed the numerical information in the same scenario. This 18.8 percentage point difference in correct assessments between nationally and globally oriented participants is highly significant (χ ²(1, n = 411) = 15.09, p ≤ 0.001). When the numerical information in the scenario showed that crime rate decreased following refugee intake, the opposite pattern between worldview and correct assessment emerges. Here, participants who classified themselves as globally oriented were better at correctly assessing the numerical information compared to nationally oriented participants: 62.1% vs. 42.1%, respectively. This 20.0 percentage point difference in correct assessment between globally oriented and nationally oriented participants is also highly significant (χ ²(1, n = 386) = 15.43, p ≤ 0.001). Thus, we see clear evidence of motivated reasoning when nationally and globally oriented participants interpret the same information.

For the non-polarizing context, also shown in Figure 2, the ability to correctly assess the numerical information did not diverge depending on worldview ideology. Here, the percentages of correct assessments are similar for globally and nationally oriented participants for both the rash increase scenario and the rash decrease scenario (for rash increase: 55.2% vs. 47.8%, χ ²(1, n = 386) = 2.11, p = 0.146; for rash decrease: 53.7% vs 60.2%, χ ²(1, n = 411) = 1.79, p = 0.181).

Figure 2. Percentages of correct assessments depending on worldview ideology

Note: The error bars show the 95% confidence intervals

To test the robustness of the descriptive results presented in Figure 2 and to control for demographics and order effects, we performed regression analyses. The results from these are shown in Table 3. For the crime scenarios, Models 1 and 2, the coefficient for worldview ideology is statistically significant (p < 0.001) and of opposite sign depending on whether the information showed that immigration increased or decreased crime rates, indicating a clear case of motivated reasoning. When the numbers showed that refugee intake led to increased crime rates, nationally oriented participants were 18.0 percentage points more likely to assess the numbers correctly, compared to globally oriented participants. When the numbers showed that refugee intake lead to decreased crime rates, nationally oriented participants were 19.7 percentage points less likely to assess the information correctly, compared to participants considering themselves globally oriented. For the non-polarizing rash scenarios, Models 3 and 4, worldview ideology is not a statistically significant predictor of correct interpretations. Hence, the results presented in Figure 2 hold up in a regression setting when controlling for the individual characteristics of age, gender, and education.

Table 3. Linear probability regressions on motivated reasoning

Note: The dependent variable is the assessments provided to the experimental scenarios: 1 if the correct assessment was indicated and 0 otherwise.

^a Nationally oriented is a dichotomous variable: 1 indicating that participants consider themselves as being more nationally (Swedish) oriented and 0 indicating that participants consider themselves as being more globally oriented (participants indicating neither were excluded from the analysis).

Robust standard errors in parentheses; ***p < 0.01.

We conducted several robustness checks of the main specification shown in Table 3 – these can be found in the Supplementary Materials. Comparing people who indicated 1 and 2 on the worldview scale to those who answered 6 and 7 (i.e., strong tendencies to either side of the worldview scale) reinforces the results, and the estimate size changes to a 31.3 percentage point increase in the crime increase scenario and a 33.8 percentage point decrease in the crime decrease scenario (Table S4).⁹ Comparing only people who indicated 3 to those who indicated 5 on the worldview scale (i.e., weak tendencies to either side) did not yield significant results on the worldview estimate (Table S5), suggesting that the results are influenced primarily by people who identify more strongly with a certain worldview.

The role of numeracy

Increasing trends emerge for almost all scenarios when studying the percentages of correct assessments given to each experimental scenario as a function of numeric ability. Participants with higher levels of numeric ability made the correct judgment more often than participants with lower levels of numeric ability. This pattern arises both for the ideologically polarized crime scenarios and for the non-polarized rash scenarios.

Figure 3 shows the percentages of correct assessments as a function of numeracy score depending on participants’ worldview ideology. The numerical assessments made by nationally oriented participants are depicted as dotted lines and the assessments made by globally oriented participants are depicted as dashed lines. All plots in Figure 3 show an increasing trend (more or less), indicating that a higher level of numeracy is associated with a higher likelihood of providing the correct assessment, regardless of the underlying scenario. Nevertheless, we see clear evidence of motivated reasoning, asserting itself as a level effect. In Figure 3(a), nationally oriented people are more likely to assess information correctly when increased crime rate is linked to refugee intake, but the likelihood of correct assessment increases with numeric ability, lending no support for the motivated reasoning-as-analysis hypothesis. In Figure 3(b), where decreased crime rate is linked to refugee intake, the opposite pattern arises. Here, globally oriented participants are better at correctly assessing the information. Importantly, however, the likelihood of evaluating the information correctly increases with numeric ability, thus casting further doubt on the motivated reasoning-as-analysis hypothesis. Figures 3(c) and 3(d) display the results for the non-polarizing rash scenarios. Again, we observe a positive trend (with the exception of globally oriented participants in Figure 3(d)), indicating that higher numeracy is associated with a higher likelihood of providing the correct assessment. In the rash scenarios, no consistent difference can be observed in the assessments provided by nationally and globally oriented participants. We present these results in Figure 3 as a descriptive analysis and the reader should beware of the relatively small sample size of each numeracy group, especially for the higher numeracy levels.¹⁰

To test the robustness of the descriptive results presented in Figure 3, we perform regression analyses. Studying the effect of numeracy in Models 1–4 in Table 4, we observe that the coefficient is positive in all models and statistically significant in three of them. This means that people with higher numeric ability are more likely to assess the numerical information correctly. The interpretation in Model 1 would be that one more correct answer to the numeracy questions is associated with a 3.5 percentage point increase in the likelihood of making the correct assessment in the crime increase scenario, all else being equal. Hence, even though people are heavily influenced by their motivated reasoning, greater numeric ability makes people more likely to interpret the information correctly, all else being equal.

Figure 3. Percentages of correct assessments by numeracy score depending on worldview ideology

Note: Comparing the level of numeracy between nationally and globally oriented participants reveals no statistical difference. For nationally oriented participants, the mean numeracy score was 2.36 (n = 409, SD = 1.93), and for globally oriented participants, the numeracy score was 2.41 (n = 388, SD = 2.03); a t-test revealed no difference between the groups (t(795) = 0.374, p = 0.710)

Table 4. Linear probability regressions on the role of numeracy

Note: The dependent variable is the assessments of the experimental scenarios: 1 if the correct assessment was made and 0 otherwise.

^a Nationally oriented is a dichotomous variable: 1 indicating that participants consider themselves as being more nationally (Swedish) oriented and 0 indicating that participants consider themselves as being more globally oriented (participants indicating neither were excluded from the analysis).

^b The low significance of the numeracy score for the rash decrease scenario is due to the large differences in correct assessments for participants with a numeracy score of 0. Excluding this part of the sample yields results that are similar to those of the other regressions.

Robust standard errors in parentheses; *p < 0.1, **p < 0.05, ***p < 0.01.

To examine the hypotheses of motivated reasoning-as-analysis and motivated reasoning-as-feelings, we add an interaction term between numeracy and worldview ideology to the specifications in Models 5–8 in Table 4. In Model 5 (i.e., for the crime increase scenario), the coefficient of the interaction term is negative and marginally significant (p = 0.066). This implies that for participants classifying themselves as nationally oriented, one more correct answer to the numeracy questions is associated with being 4.3 percentage points less likely to provide the correct assessment, all else being equal. This contradicts the motivated reasoning-as-analysis hypothesis, since the crime increase scenario is assumed to be in line with a more nationally oriented worldview. In Model 6, the coefficient for the interaction term is negative, which would be in line with the motivated reasoning-as-analysis hypothesis; however, the coefficient is not statistically significant (p = 0.852). Thus, we find no support for the motivated reasoning-as-analysis hypothesis; rather, we find suggestive evidence supporting the idea of the motivated reasoning-as-feelings hypothesis. When treating the worldview ideology as a continuous variable, the results become more clearly supportive of the motivated reasoning-as-feelings hypothesis (p < 0.01), suggesting that people with higher numeric ability are less likely to engage in motivated reasoning (see Table S3 in Supplementary Materials).

Discussion

This study adds to the literature showing that motivated reasoning is prevalent for politically sensitive issues such as climate change, energy policy, and gun control (see e.g., Kahan et al., 2012, 2017; Bolsen et al., 2014; Dunlap et al., 2016; Drummond & Fischhoff, 2017; Washburn & Skitka, 2017). The fact that we find clear evidence of motivated reasoning in relation to refugee intake may be expected, but our study adds to the understanding of the width and breadth of motivated reasoning by extending the context, the sample, and the potential mitigating role of individual differences. Establishing for whom and for which issues motivated reasoning occurs is especially important given the severe potential consequences of false beliefs. We live in a new informational landscape, where recruitment and consumption of information is increasingly tailored in accordance with our preferences and social belonging. At an increasing rate, people are primarily confronted with information that supports their already existing beliefs and values. As Jonathan Haidt (2012, p. 85) put it, “[N]ow that we all have access to search engines on our cell phones, we can call up a team of supportive scientists for almost any conclusion twenty-four hours a day. Whatever you want to believe about the causes of global warming or whether a fetus can feel pain, just Google your belief … Science is a smorgasbord, and Google will guide you to the study that's right for you.”

Our study also contributes to the existing literature by showing that motivated reasoning is a widespread phenomenon that extends beyond North American samples and partisan responses. Importantly, we show that it is highly prevalent for the issue of refugee intake, a topic that is a top priority on many political agendas worldwide. Previous studies have shown that information avoidance is a common strategy for protecting beliefs about the social consequences of immigration (Freddi, 2017). Here, we show that people protect their beliefs about the effects of refugee intake also through the process of motivated reasoning. An important take-home message from this study is that neither side is immune to motivated reasoning when it comes to refugee intake. Regardless of worldview, people misinterpret facts in favor of their own prejudice. Thus, people who accuse others of motivated reasoning are probably as likely to engage in motivated reasoning themselves. Our study also shows that people with higher numeric ability are less likely to engage in motivated reasoning. This suggests that motivated reasoning more commonly is driven by feelings and emotional cues rather than deliberate analytical processes.

Conflicting theories suggest that there are multiple mechanisms for how people engage in motivated reasoning. In particular, it is unclear whether motivated reasoning is most accurately viewed as a deliberate (“System 2”) or intuitive (“System 1”) process. We conceptualize these as motivated reasoning-as-analysis and motivated reasoning-as-feelings. Our results show a positive association between percentages of correct assessments and numeracy. In other words, people with lower levels of numeric ability were less likely to make the correct assessment, no matter the context, supporting the motivated reasoning-as-feelings view. This corroborates the findings by Pennycook and Rand (2018), who showed a positive correlation between the propensity to think analytically and the rejection of politically concordant fake news articles. However, our findings contrast with the results of Kahan et al. (2017), who showed that motivated reasoning was stronger among individuals with higher numeric ability. Given the similarities between the experimental scenarios used in our study and those of Kahan et al. (2017), the discrepancy in results might appear a bit surprising. There are, however, several potential explanations. First, we explore different political issues, that of refugee intake and that of gun control. Second, we explore different underlying ideological constructs: patriotism and political affinity. Third, we have different samples: a general Swedish sample and a general American sample. Although we see no obvious reason for why these differences should lead to different results, they should be acknowledged.

A limitation of our study is that we cannot link the existence of motivated reasoning in refugee intake to any downstream behavior. This is an important issue for future research. To what extent does motivated reasoning translate into actual behavior? One of the few studies exploring this link is Babcock et al. (1995). They establish a causal link between biased reasoning and subsequent economic behavior in a court case setting, where motivated reasoning occurred when people were assigned the role of plaintiff or defendant before reading the court case materials. Self-deception about one's morality and altruism has also been established in numerous studies. For example, Bersoff (2001) showed that unethical behavior is promoted when people are able to develop and maintain a biased characterization of an unethical action as being morally acceptable. In the context of our study, an important question that should be raised is whether people who display biased reasoning would behave differently toward immigrants or refugees. Based on the results from our study, we cannot say anything for certain about this. However, a study by Nickerson and Louis (2008) on an Australian sample found that people who identify as nationally oriented (i.e., as “Australian”) were less welcoming toward asylum seekers, and that people who identified more as “humans” were more welcoming toward asylum seekers. This suggests that motivated reasoning is likely to influence actual behavior when it comes to refugee intake.

In sum, our results confirm that motivated reasoning is a widespread phenomenon. We show that motivated reasoning is prevalent when assessing the effects of refugee intake. From the perspective of democracy, this poses a significant challenge because it means that people often are unwilling to change their beliefs, even in the face of cold, hard facts. A fundamental aspect of deliberative democracy is that people are able to generate reliable and correct assessments of the information at hand when making judgments and decisions. Motivated reasoning thus counteracts the basis of a well-functioning democratic society. Furthermore, interest groups, political parties, and policy-makers may take advantage of people's tendency to engage in motivated reasoning and actively amplify such tendencies among groups, leading to increased societal fractionalization (see Hagmann & Loewenstein, 2017). Arguably, this is what is happening in many Western countries right now in the political discourse about immigration and refugee intake, which has become very polarized and dogmatic. From the perspective of democracy, it is therefore important to foster citizens who are able to objectively interpret information so that exchange of reasons can occur and informed decisions can be reached, whether it be with regard to immigration or other politically sensitive issues. After all, we cannot all believe what we want to believe when it comes to issues that have life-and-death implications.