Dual-process 2.0 accounts are increasingly compelling, and we welcome De Neys's proposed model, which we bolster here by noting further challenges to assumptions of “exclusivity” (the notion that intuition and deliberation generate unique responses). We additionally argue for considerable methodological care when exploring the nature of deliberative processing.
Among the most crucial considerations when devising thinking, reasoning, and decision-making tasks is determining what constitutes a “correct” answer and what it means when participants produce this answer. Indeed, De Neys cautions against an “ought-is fallacy” (Elqayam & Evans, Reference Elqayam and Evans2011), which arises when responses aligning with “normative” theories (e.g., predicate logic or Bayes' theorem) are viewed as being diagnostic of deliberation. We contend that normative standards, although useful for performance benchmarking, can present blind spots for experimental design and theory building. As such, we concur with Elqayam and Evans (Reference Elqayam and Evans2011) that constructing theories of reasoning around normative standards is problematic for understanding psychological processes.
To evaluate deliberative processing successfully, it seems prudent to adopt a “soft normativist” (Stupple & Ball, Reference Stupple and Ball2014) or “descriptivist” (Elqayam & Evans, Reference Elqayam and Evans2011) approach. Accordingly, research programs should acknowledge the distorting lens of normative standards (while also avoiding the trap of relativism), recognizing that although normative standards may be correlated with deliberation, they are not causally linked to it (Stupple & Ball, Reference Stupple and Ball2014). From a soft-normativism perspective, “normative–deliberative gaps” are expected for many reasons (e.g., disputed or misunderstood norms, strategy preferences, alternative task interpretations, cognitive ability and mindware constraints, and impoverished thinking dispositions), necessitating careful consideration.
Normative standards should also be contested and evaluated whenever multiple, candidate standards exist (Stenning & Varga, Reference Stenning, Varga, Ball and Thompson2018). For some tasks, the normative response is uncontroversial, but for others, participants must make sense of task requirements and may not construe the task as intended. For example, Oaksford and Chater (Reference Oaksford and Chater2009) proposed an alternative normative standard for the Wason selection task based upon “information gain,” which is consistent with the most common responses (contrasting with Wason's [1966] logicist proposals). Oaksford and Chater (Reference Oaksford and Chater2009) extend this perspective to demonstrate that logical fallacies can be rationally persuasive. Indeed, caution is advised for researchers who associate endorsement of fallacies with a lack of deliberation. It is prudent not simplistically to equate standard normative responses with deliberative thinking without also considering individual goals.
In most thinking tasks, participants are not explicitly prescribed a goal or norm. Indeed, Cohen (Reference Cohen1981) famously argued that reasoning research presents “trick” questions with minimalist instructions to naïve participants. The assumption that participants identify tasks as requiring deliberation may itself be naïve. Stupple and Ball (Reference Stupple and Ball2014) proposed that when naïve participants attempt novel reasoning problems, they determine an appropriate normative standard and select a strategy through a process of “informal reflective equilibrium.” Through this, increasing familiarity with problem forms – even in the absence of feedback – can result in participants aligning with normative responses assumed to require deliberation (Ball, Reference Ball2013; Dames, Klauer, & Ragni, Reference Dames, Klauer and Ragni2022). This alignment need not be deliberative, however, but could instead entail detection of patterns in problems and an increasing intuition strength for normatively aligned heuristic responses.
These variations in participants' goals and strategies are captured by Markovits, Brisson, and de Chantal (Reference Markovits, Brisson and de Chantal2017) (cf. Verschueren, Schaeken, & d'Ydewalle, Reference Verschueren, Schaeken and d'Ydewalle2005), who demonstrated individual differences in strategy preferences (probabilistic vs. counterexample) that are orthogonal to preferences for intuitive versus deliberative thinking. These strategies have implications for the interplay between deliberation and normative standards. Participants adopting a counterexample strategy (based on mental models) versus a probabilistic strategy (based on information gain or probability heuristics; Beeson, Stupple, Schofield, & Staples, Reference Beeson, Stupple, Schofield and Staples2019; Oaksford & Chater, Reference Oaksford and Chater2009; Verschueren et al., Reference Verschueren, Schaeken and d'Ydewalle2005) may differ in their task construal and understanding of “correct” answers. Although it is unclear whether strategies necessarily entail adoption of particular normative standards, responding to a problem in terms of information gain versus a necessary truth derived from a mental model would reasonably be assumed to require differing degrees of deliberation and differing use of intuitive cues.
When judging whether deliberation has occurred, we also suggest that responses can be less reliable than response times. For example, for the lily-pad cognitive reflective task (CRT) problem, incorrect non-intuitive answers averaged longer response times than incorrect intuitive or correct answers (Stupple, Pitchford, Ball, Hunt, & Steel, Reference Stupple, Pitchford, Ball, Hunt and Steel2017), which is inconsistent with “cognitive miserliness” and the absence of deliberation. Such outcomes can arise from task misinterpretation, lack of mindware, or the strategy selected. When relying on responses to judge a process, we cannot know if a participant has reasoned deliberatively unless we presume the task was understood as intended, and we cannot know they understood the task as intended unless we presume they reasoned deliberatively (cf. Smedslund, Reference Smedslund1990).
We also note that meta-reasoning studies offer vital insights into individual differences in uncertainty monitoring, facilitating a more nuanced understanding of deliberative processing on a task. For example, when participants determine how long to persevere, they may be optimizing or satisficing, and those of a miserly disposition may simply be looking to bail out through a “computational escape hatch” (Ackerman, Douven, Elqayam, & Teodorescu, Reference Ackerman, Douven, Elqayam, Teodorescu, Elqayam, Douven, Evans and Cruz2020; Ball & Quayle, Reference Ball and Quayle2000). Low confidence responses after an “impasse” can also decouple the link between response time and deliberative thinking, as can uncertainty about the intended “correct” answer. As such an array of individual-differences measures are necessary to understand the nature of deliberative processes. Furthermore, unpicking such deliberative processes goes beyond the observation of fast and slow thinking. Intuitive processes are always necessary for a participant to respond and sometimes they are sufficient. Participants who understand a task as requiring the alleged system 1 response will not be prompted into deliberation by an awareness of an alleged system 2 response (as this is not necessarily the normative response or the participant's goal).
In sum, we advocate for an approach that follows dual-process model 2.0, but which triangulates task responses with response times, metacognitive measures and individual-difference variables, while aligning with a soft or agnostic view of normative standards. When deciding which responses are the product of deliberative thinking, researchers must be mindful of the myriad individual differences in task interpretations, strategies, and perceived “normative” responses.
You have
Access
Dual-process 2.0 accounts are increasingly compelling, and we welcome De Neys's proposed model, which we bolster here by noting further challenges to assumptions of “exclusivity” (the notion that intuition and deliberation generate unique responses). We additionally argue for considerable methodological care when exploring the nature of deliberative processing.
Among the most crucial considerations when devising thinking, reasoning, and decision-making tasks is determining what constitutes a “correct” answer and what it means when participants produce this answer. Indeed, De Neys cautions against an “ought-is fallacy” (Elqayam & Evans, Reference Elqayam and Evans2011), which arises when responses aligning with “normative” theories (e.g., predicate logic or Bayes' theorem) are viewed as being diagnostic of deliberation. We contend that normative standards, although useful for performance benchmarking, can present blind spots for experimental design and theory building. As such, we concur with Elqayam and Evans (Reference Elqayam and Evans2011) that constructing theories of reasoning around normative standards is problematic for understanding psychological processes.
To evaluate deliberative processing successfully, it seems prudent to adopt a “soft normativist” (Stupple & Ball, Reference Stupple and Ball2014) or “descriptivist” (Elqayam & Evans, Reference Elqayam and Evans2011) approach. Accordingly, research programs should acknowledge the distorting lens of normative standards (while also avoiding the trap of relativism), recognizing that although normative standards may be correlated with deliberation, they are not causally linked to it (Stupple & Ball, Reference Stupple and Ball2014). From a soft-normativism perspective, “normative–deliberative gaps” are expected for many reasons (e.g., disputed or misunderstood norms, strategy preferences, alternative task interpretations, cognitive ability and mindware constraints, and impoverished thinking dispositions), necessitating careful consideration.
Normative standards should also be contested and evaluated whenever multiple, candidate standards exist (Stenning & Varga, Reference Stenning, Varga, Ball and Thompson2018). For some tasks, the normative response is uncontroversial, but for others, participants must make sense of task requirements and may not construe the task as intended. For example, Oaksford and Chater (Reference Oaksford and Chater2009) proposed an alternative normative standard for the Wason selection task based upon “information gain,” which is consistent with the most common responses (contrasting with Wason's [1966] logicist proposals). Oaksford and Chater (Reference Oaksford and Chater2009) extend this perspective to demonstrate that logical fallacies can be rationally persuasive. Indeed, caution is advised for researchers who associate endorsement of fallacies with a lack of deliberation. It is prudent not simplistically to equate standard normative responses with deliberative thinking without also considering individual goals.
In most thinking tasks, participants are not explicitly prescribed a goal or norm. Indeed, Cohen (Reference Cohen1981) famously argued that reasoning research presents “trick” questions with minimalist instructions to naïve participants. The assumption that participants identify tasks as requiring deliberation may itself be naïve. Stupple and Ball (Reference Stupple and Ball2014) proposed that when naïve participants attempt novel reasoning problems, they determine an appropriate normative standard and select a strategy through a process of “informal reflective equilibrium.” Through this, increasing familiarity with problem forms – even in the absence of feedback – can result in participants aligning with normative responses assumed to require deliberation (Ball, Reference Ball2013; Dames, Klauer, & Ragni, Reference Dames, Klauer and Ragni2022). This alignment need not be deliberative, however, but could instead entail detection of patterns in problems and an increasing intuition strength for normatively aligned heuristic responses.
These variations in participants' goals and strategies are captured by Markovits, Brisson, and de Chantal (Reference Markovits, Brisson and de Chantal2017) (cf. Verschueren, Schaeken, & d'Ydewalle, Reference Verschueren, Schaeken and d'Ydewalle2005), who demonstrated individual differences in strategy preferences (probabilistic vs. counterexample) that are orthogonal to preferences for intuitive versus deliberative thinking. These strategies have implications for the interplay between deliberation and normative standards. Participants adopting a counterexample strategy (based on mental models) versus a probabilistic strategy (based on information gain or probability heuristics; Beeson, Stupple, Schofield, & Staples, Reference Beeson, Stupple, Schofield and Staples2019; Oaksford & Chater, Reference Oaksford and Chater2009; Verschueren et al., Reference Verschueren, Schaeken and d'Ydewalle2005) may differ in their task construal and understanding of “correct” answers. Although it is unclear whether strategies necessarily entail adoption of particular normative standards, responding to a problem in terms of information gain versus a necessary truth derived from a mental model would reasonably be assumed to require differing degrees of deliberation and differing use of intuitive cues.
When judging whether deliberation has occurred, we also suggest that responses can be less reliable than response times. For example, for the lily-pad cognitive reflective task (CRT) problem, incorrect non-intuitive answers averaged longer response times than incorrect intuitive or correct answers (Stupple, Pitchford, Ball, Hunt, & Steel, Reference Stupple, Pitchford, Ball, Hunt and Steel2017), which is inconsistent with “cognitive miserliness” and the absence of deliberation. Such outcomes can arise from task misinterpretation, lack of mindware, or the strategy selected. When relying on responses to judge a process, we cannot know if a participant has reasoned deliberatively unless we presume the task was understood as intended, and we cannot know they understood the task as intended unless we presume they reasoned deliberatively (cf. Smedslund, Reference Smedslund1990).
We also note that meta-reasoning studies offer vital insights into individual differences in uncertainty monitoring, facilitating a more nuanced understanding of deliberative processing on a task. For example, when participants determine how long to persevere, they may be optimizing or satisficing, and those of a miserly disposition may simply be looking to bail out through a “computational escape hatch” (Ackerman, Douven, Elqayam, & Teodorescu, Reference Ackerman, Douven, Elqayam, Teodorescu, Elqayam, Douven, Evans and Cruz2020; Ball & Quayle, Reference Ball and Quayle2000). Low confidence responses after an “impasse” can also decouple the link between response time and deliberative thinking, as can uncertainty about the intended “correct” answer. As such an array of individual-differences measures are necessary to understand the nature of deliberative processes. Furthermore, unpicking such deliberative processes goes beyond the observation of fast and slow thinking. Intuitive processes are always necessary for a participant to respond and sometimes they are sufficient. Participants who understand a task as requiring the alleged system 1 response will not be prompted into deliberation by an awareness of an alleged system 2 response (as this is not necessarily the normative response or the participant's goal).
In sum, we advocate for an approach that follows dual-process model 2.0, but which triangulates task responses with response times, metacognitive measures and individual-difference variables, while aligning with a soft or agnostic view of normative standards. When deciding which responses are the product of deliberative thinking, researchers must be mindful of the myriad individual differences in task interpretations, strategies, and perceived “normative” responses.
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Competing interest
None.