Explaining normative–deliberative gaps is essential to dual-process theorizing

Abstract

We discuss significant challenges to assumptions of exclusivity and highlight methodological and conceptual pitfalls in inferring deliberative processes from reasoning responses. Causes of normative–deliberative gaps are considered (e.g., disputed or misunderstood normative standards, strategy preferences, task interpretations, cognitive ability, mindware and thinking dispositions) and a soft normativist approach is recommended for developing the dual-process 2.0 architecture.

Dual-process 2.0 accounts are increasingly compelling, and we welcome De Neys's proposed model, which we bolster here by noting further challenges to assumptions of “exclusivity” (the notion that intuition and deliberation generate unique responses). We additionally argue for considerable methodological care when exploring the nature of deliberative processing.

Among the most crucial considerations when devising thinking, reasoning, and decision-making tasks is determining what constitutes a “correct” answer and what it means when participants produce this answer. Indeed, De Neys cautions against an “ought-is fallacy” (Elqayam & Evans, 2011), which arises when responses aligning with “normative” theories (e.g., predicate logic or Bayes' theorem) are viewed as being diagnostic of deliberation. We contend that normative standards, although useful for performance benchmarking, can present blind spots for experimental design and theory building. As such, we concur with Elqayam and Evans (2011) that constructing theories of reasoning around normative standards is problematic for understanding psychological processes.

To evaluate deliberative processing successfully, it seems prudent to adopt a “soft normativist” (Stupple & Ball, 2014) or “descriptivist” (Elqayam & Evans, 2011) approach. Accordingly, research programs should acknowledge the distorting lens of normative standards (while also avoiding the trap of relativism), recognizing that although normative standards may be correlated with deliberation, they are not causally linked to it (Stupple & Ball, 2014). From a soft-normativism perspective, “normative–deliberative gaps” are expected for many reasons (e.g., disputed or misunderstood norms, strategy preferences, alternative task interpretations, cognitive ability and mindware constraints, and impoverished thinking dispositions), necessitating careful consideration.

Normative standards should also be contested and evaluated whenever multiple, candidate standards exist (Stenning & Varga, 2018). For some tasks, the normative response is uncontroversial, but for others, participants must make sense of task requirements and may not construe the task as intended. For example, Oaksford and Chater (2009) proposed an alternative normative standard for the Wason selection task based upon “information gain,” which is consistent with the most common responses (contrasting with Wason's [1966] logicist proposals). Oaksford and Chater (2009) extend this perspective to demonstrate that logical fallacies can be rationally persuasive. Indeed, caution is advised for researchers who associate endorsement of fallacies with a lack of deliberation. It is prudent not simplistically to equate standard normative responses with deliberative thinking without also considering individual goals.

In most thinking tasks, participants are not explicitly prescribed a goal or norm. Indeed, Cohen (1981) famously argued that reasoning research presents “trick” questions with minimalist instructions to naïve participants. The assumption that participants identify tasks as requiring deliberation may itself be naïve. Stupple and Ball (2014) proposed that when naïve participants attempt novel reasoning problems, they determine an appropriate normative standard and select a strategy through a process of “informal reflective equilibrium.” Through this, increasing familiarity with problem forms – even in the absence of feedback – can result in participants aligning with normative responses assumed to require deliberation (Ball, 2013; Dames, Klauer, & Ragni, 2022). This alignment need not be deliberative, however, but could instead entail detection of patterns in problems and an increasing intuition strength for normatively aligned heuristic responses.

These variations in participants' goals and strategies are captured by Markovits, Brisson, and de Chantal (2017) (cf. Verschueren, Schaeken, & d'Ydewalle, 2005), who demonstrated individual differences in strategy preferences (probabilistic vs. counterexample) that are orthogonal to preferences for intuitive versus deliberative thinking. These strategies have implications for the interplay between deliberation and normative standards. Participants adopting a counterexample strategy (based on mental models) versus a probabilistic strategy (based on information gain or probability heuristics; Beeson, Stupple, Schofield, & Staples, 2019; Oaksford & Chater, 2009; Verschueren et al., 2005) may differ in their task construal and understanding of “correct” answers. Although it is unclear whether strategies necessarily entail adoption of particular normative standards, responding to a problem in terms of information gain versus a necessary truth derived from a mental model would reasonably be assumed to require differing degrees of deliberation and differing use of intuitive cues.

When judging whether deliberation has occurred, we also suggest that responses can be less reliable than response times. For example, for the lily-pad cognitive reflective task (CRT) problem, incorrect non-intuitive answers averaged longer response times than incorrect intuitive or correct answers (Stupple, Pitchford, Ball, Hunt, & Steel, 2017), which is inconsistent with “cognitive miserliness” and the absence of deliberation. Such outcomes can arise from task misinterpretation, lack of mindware, or the strategy selected. When relying on responses to judge a process, we cannot know if a participant has reasoned deliberatively unless we presume the task was understood as intended, and we cannot know they understood the task as intended unless we presume they reasoned deliberatively (cf. Smedslund, 1990).

We also note that meta-reasoning studies offer vital insights into individual differences in uncertainty monitoring, facilitating a more nuanced understanding of deliberative processing on a task. For example, when participants determine how long to persevere, they may be optimizing or satisficing, and those of a miserly disposition may simply be looking to bail out through a “computational escape hatch” (Ackerman, Douven, Elqayam, & Teodorescu, 2020; Ball & Quayle, 2000). Low confidence responses after an “impasse” can also decouple the link between response time and deliberative thinking, as can uncertainty about the intended “correct” answer. As such an array of individual-differences measures are necessary to understand the nature of deliberative processes. Furthermore, unpicking such deliberative processes goes beyond the observation of fast and slow thinking. Intuitive processes are always necessary for a participant to respond and sometimes they are sufficient. Participants who understand a task as requiring the alleged system 1 response will not be prompted into deliberation by an awareness of an alleged system 2 response (as this is not necessarily the normative response or the participant's goal).

In sum, we advocate for an approach that follows dual-process model 2.0, but which triangulates task responses with response times, metacognitive measures and individual-difference variables, while aligning with a soft or agnostic view of normative standards. When deciding which responses are the product of deliberative thinking, researchers must be mindful of the myriad individual differences in task interpretations, strategies, and perceived “normative” responses.