Choice blindness in financial decision making

Abstract

Choice Blindness is an experimental paradigm that examines the interplay between individuals’ preferences, decisions, and expectations by manipulating the relationship between intention and choice. This paper expands upon the existing Choice Blindness framework by investigating the presence of the effect in an economically significant decision context, specifically that of pension choice. In addition, it investigates a number of secondary factors hypothesized to modulate Choice Blindness, including reaction time, risk preference, and decision complexity, as well as analysing the verbal reports of non-detecting participants. The experiment was administered to 100 participants of mixed age and educational attainment. The principal finding was that no more than 37.2% of manipulated trials were detected over all conditions, a result consistent with previous Choice Blindness research. Analysis of secondary factors found that reaction time, financial sophistication and decision complexity were significant predictors of Choice Blindness detection, while content analysis of non-detecting participant responses found that 20% implied significant preference changes and 62% adhered to initial preferences. Implications of the Choice Blindness effect in the context of behavioural economics are discussed, and an agenda for further investigation of the paradigm in this context is outlined.

1 Introduction

The Choice Blindness paradigm is a novel experimental method which uses misdirection or sleight-of-hand to manipulate the relationship between intention and choice, by presenting decision-makers with outcomes they did not actually choose and eliciting a response (Johansson, Hall and Sikström, 2008).

The principle findings in the Choice Blindness literature are that many participants do not notice when their stated preferences are manipulated, even when factors such as attention, task engagement and social effects are taken into account (Johansson, Hall and Chater 2011). First demonstrated in a study examining preferences for female faces (Johansson, Hall, Sikström and Olsson 2005), Choice Blindness experiments have since been extended into various other domains, including naturalistic consumer choices (Hall et al. 2010), attitude formation and moral sentiment (Hall, Johansson and Strandberg 2012), polling and political preferences (Hall et al. 2013), symptomatology and psychiatric self-diagnoses (Merckelbach, Jelicic and Pieters 2010), and to other sensory modalities (Steenfeldt-Kristensen and Thornton, 2013). Choice Blindness studies to date have consistently found that only 20-35% of participants detect the manipulation of their preferences (Johansson, Hall and Chater, 2011).

The significance of Choice Blindness can be extended beyond the simple question of whether participants detect a manipulation. In their original study, Johansson et al. (2005) demonstrated not only that many participants failed to notice manipulation of their preferences but also that they could be induced to defend these altered preferences when asked to motivate their choice. These “confabulated” explanations were not qualitatively different from explanations of unaltered preferences, and in some cases, participants would confabulate explanations contradicting their original choices—mentioning details entirely absent from the images initially chosen.

The purpose of the present study was to devise a Choice Blindness experiment involving a significant and realistically framed economic decision. In addition, we examine the factors that underlie and modulate it. This is an area which the existing literature has had some difficulty addressing (Johansson, Hall, and Sikström, 2008). Therefore the design was also intended to isolate a variety of factors intuited to underlie the Choice Blindness effect.

An economically significant class of decision was examined through a design featuring decisions about personal finance, specifically the choice of a pension. Pensions are a very familiar topic in behavioural economics (e.g., Madrian and Shea 2001, Thaler & Benartzi 2004, Tapia & Yermo 2007) in part because they represent a class of decision with important ramifications for both individual welfare and the future stability of an economy. Moreover it is an area in which traditional economic theorising has failed to adequately anticipate the financial decisions of agents, as evidenced by the abundance of chronically underfunded pension schemes (Thaler & Benartzi, 2004).

Pension decisions are also relatively accessible (if not familiar) because most individuals will make at least one such decision in their lifetime. Pensions also represented a topic of particular local relevance both before and during the time the study was conducted, as Ireland was then preparing to implement wide-ranging pension reforms and introduce an autoenrollment program (National Pensions Framework, 2010).

Furthermore, a pension paradigm requires participants to construct a portfolio of options, a task of greater complexity than the binary choices presented in most previous Choice Blindness studies. This has the potential to elicit richer and more nuanced responses from participants, and this added granularity is of crucial importance in evaluating confabulatory responses, and so distinguishing instances of significant preference change from responses motivated by inattention to specific choice characteristics or lack of engagement.

The design was intended to replicate the conditions, materials and appearance of actual personal finance decisions with maximum accuracy, so as to elicit as great a degree of participant engagement as possible within the confines of a hypothetical scenario. The presence of a Choice Blindness effect consistent with previous findings was thus the study’s primary hypothesis.

Additionally, the design built on previous Choice Blindness experiments by including a variety of auxiliary measures, enabling the investigation of a range of factors with the potential to influence the prevalence of Choice Blindness, represented by participant detection rates. Measures were included to evaluate participant reaction time, memory and task recall, risk preferences, and financial sophistication. The design also included cheater-detection priming for some participants. Reaction time was recorded as the time taken (in seconds) for a participant to complete each trial.

Reaction time was intended as a proxy measure for the degree of care and attention participants applied to the task. It was hypothesized that participants who spent longer on the task would be more likely to detect manipulations.

A brief memory task was administered to participants at the end of the experiment, just prior to debriefing. The task involved answering five simple questions about the task, of increasing sophistication and “difficulty”, and recorded as a score out of five. This instrument was intended primarily to address the theory that Choice Blindness represents a memory effect, in which participants appear to reverse their preferences because they do not recall their initial choices clearly (Johansson, Hall and Sikström 2008). A positive relationship between detection rate and recall accuracy would be expected in this case.

The experiment also included a brief risk profile (adapted from Holt & Laury, 2002) as well as a self-report measure of participants’ preferences for risk. We hypothesized that participants with a lower risk preference would be more cautious and reflective, and therefore more likely to detect a manipulation. Additionally, findings from an initial pilot study suggested that familiarity with the subject matter would provide a strong influence on detection rates, in accordance with previous findings on biases in financial decision-making (e.g., Feng and Seasholes, 2005).

The design therefore included a brief financial sophistication instrument (Lusardi & Mitchell, 2006), which has proven to be a robust proxy for general knowledge of financial issues such as personal finance. It was hypothesized that greater financial knowledge would increase detection rates. Half of the participants were assigned to a condition in which a subtle cheater detection prime was inserted into the experiment’s introductory literature, and it was hypothesized that this group would be more likely to detect manipulations.

2 Method

2.1 Participants

A total of 100 participants (53 female) were recruited for the study, ranging in age from 18 to 60 years (mean = 24.69, SD = 7.33). 41 were undergraduate students, with the remainder being graduate students and professionals ranging from basic degree to Ph.D. level of educational attainment. Participants were recruited through various forms of public advertising on university campuses. The study was advertised as an investigation into the factors affecting decisions about personal finance, and all participants were initially naive as to the experiment’s true purpose.

2.2 Materials

The primary materials consisted of two pages outlining the investment options (which numbered between ten and twelve) offered by one of six fictional pension companies. One page depicted the options graphically in a tiered format, from low risk to high risk. The other briefly explained each option in relatively simple language consistent with the vocabulary observed in actual financial prospectuses. The overall presentation was modelled closely on the explanatory materials provided by actual pension funds in the Republic of Ireland to new and prospective clients. An example of the experimental stimuli can be found in Appendix A.

Additional materials consisted of an information and briefing sheet, a short six-question financial literacy survey (adapted from Lusardi and Mitchell, 2006), a basic demographic questionnaire, and a one-question risk preference instrument (adapted from Holt and Laury, 2002). Participants in the cheater-detection condition were given a slightly different briefing sheet, in which the paragraph regarding auto-enrolment in Ireland in 2014 contained the following additional sentence as a cheater-detecting prime:

However, concerns have been raised in some quarters about the security and fairness of an auto-enrolment system for pensions, particularly the possibility that they may afford employers greater opportunity to cheat their employees by changing their preferences or opting them out of the system without their consent.

During the experiment proper, participants indicated their choices using a simple response sheet instructing them to choose between three and six options and allocate a percentage of their hypothetical investment to each one. All experimental materials are reproduced in Appendix B.

The experimenters themselves made use of a two-monitor computer system, with one display facing the participant. Participant responses were entered by an experimenter into a specialized interface which was designed to facilitate data entry, presentation and manipulation as quickly as possible, so as to minimize both memory degradation effects and participant suspicion.

2.3 Procedure

Following the general pattern of previous studies (e.g. Johansson et al. 2005, Hall et al. 2010) the experiment was administered by two experimenters, one who interacted directly with the participant and presented the physical materials and another responsible for notation and computerized data entry.

At the beginning of the experiment, participants were shown an information sheet explaining the ostensible purpose of the study (“to investigate some of the different factors which affect people’s choice of a pension plan”) and providing some simple background information on what pensions are, how they work, and how proposed auto-enrolment schemes in the future make them relevant to everyday life. Each participant was also allocated to a cheater-detection (CD) or non-CD condition, and given the explanation with or without the CD priming paragraph as appropriate. The brief demographic questionnaire, risk preference profile, and financial sophistication survey were then administered before the actual experiment began. The experimenters also recorded each participant’s responses on a hand-held recording device, for the purpose of qualitative analysis.

The experiment featured five trials, each featuring the materials for one of the six simulated companies. The first, third and fifth of the five trials were standard: The participants chose a portfolio, which was then presented to them on-screen for comment, just as they had chosen it. The second and fourth trials were manipulated—participants were shown a portfolio in which one item differed from the one they actually chosen at the beginning of the trial. The second trial was a manipulation of the similar condition, and involved the second experimenter replacing one of their selections, chosen at random, with another fund on the list of similar type and risk return profile, and differing primarily in investment type and name. The fourth trial represented the dissimilar condition: here the experimenter randomly replaced one of the participant’s choices with another fund from the list of significantly different character and risk profile—that is, an option significantly more (or less) risky than the original choice.

The order in which the trials themselves were presented was not varied, as the design assumes (following the findings of previous Choice Blindness experiments, e.g., Johansson et al,. 2005; Hall et al. 2010) that the detection threshold for the similar condition is lower than for the dissimilar condition. Therefore it is implicitly assumed that any participant who detects a manipulation in the similar condition (i.e., the one more difficult to detect) would also detect a dissimilar manipulation, and the experiment was concluded once any detection was observed. Proceeding with this assumption was also deemed more efficient as it maximized the number of trials per participant.

On each trial, participants were given the pension fund materials for one of the six fictional companies, chosen at random by the experimenter. Participants were instructed to allocate their hypothetical pension contribution between these fund options, selecting a portfolio of between three and six different funds, and attributing a percentage to each one. Participants were encouraged to take as much time as they needed to make a decision, and indicated their choices by filling in a response sheet. On completion of each portfolio, the participant returned this sheet to the experimenter, and the explanatory materials were also withdrawn.

The first experimenter then explained that the selected portfolio would be displayed on the monitor facing the participant, who would then be asked to explain the selection of each fund in the portfolio, while the second experimenter quickly entered her chosen portfolio into the computer. At this point in the second and fourth trials, the experimenter also made the appropriate similar or dissimilar manipulation. The result, with or without manipulation, was then displayed in a simple graphical format on the monitor facing the participant. The average time between participants handing over a response sheet and their portfolio being displayed back to them was approximately 5 seconds. In order to control for possible primacy or recency effects, all selected portfolios (manipulated or not) were presented to participants in a randomized order. Participants were then invited to comment on their portfolio, and were free to do so at their leisure.

The experimenter’s responses were limited to simple verbal prompting of the participant for more detail when required, and participants were not specifically asked if the pension portfolio shown was the same one they chose. During the explanation of fund choices, if a participant pointed out or otherwise expressed notice of a mismatch, the experiment ended and the participant was debriefed at the end of that trial. Following the established paradigm, such a result was recorded as concurrent detection. Debriefing involved explaining the actual purpose of the experiment and the deception/manipulation which it involved. Immediately before debriefing, all participants, detected or not, were asked a series of 5 memory questions designed to evaluate their recollection of and attention to the task.

If the participant finished explaining fund choices without any such evidence of detection, the trial was concluded and the next trial began, featuring a different set of stimuli (i.e. a different company’s brochure). Participants who proceeded through all trials without detecting a mismatch were told that the experiment was almost over and would conclude after a few brief questions. Consistent with previous studies, participants who had not detected any mismatch were then asked a series of increasingly specific questions designed to elicit how confident they were of their choices and whether they had noticed anything suspicious or unusual during the experiment. The questions were as follows:

1. 1 Did you notice anything unusual during the experiment?

2. 2 Did you suspect anything was amiss with your final portfolios?

3. 3 Did you suspect any trickery on behalf of the experimenters?

4. 4 Did you notice that we switched your portfolio options?

Any affirmative answer to the these question was recorded as retrospective detection. Finally, participants who showed no retrospective detection were explicitly asked whether they believed they would notice any hypothetical manipulation of their choices. Once these questions had been asked, all participants were fully debriefed.

3 Results

The experiment produced 100 sets of individual data across all experimental conditions, in addition to demographic and instrumental information. The data for two participants who withdrew before completing the experiment were discarded, leaving a final total of 98.

The principle descriptive data in this study were the number of detections (both concurrent and retrospective) registered in both the Similar and Dissimilar manipulated conditions. In the Similar condition, only 24.5% of responses feature detection of portfolio manipulation. In the Dissimilar condition, 33.8% of responses detect the manipulation. However, if we assume that the 24 participants who detected the Similar mismatch would have also detected the Dissimilar mismatch, this figure increases to 50%. We can thus view a 33.8% detection rate as a lower bound estimate in the dissimilar condition and 50% as an upper bound estimate. Combining the Similar and Dissimilar figures, we find an overall detection rate across all conditions ranging from 28.5% to 37.2%, depending on the bound.

The difference in detection rates between Similar and Dissimilar manipulations was statistically significant when compared using a simple t-test (p<0.01). This result is driven largely by our assumption that participants who detected Similar manipulations would have also detect Dissimilar manipulations. If this assumption is not made, these 24 participants could be removed from the sample and the groups compared; however it is highly unlikely that the propensity to detect for these individuals is equal to the sample mean detection rate. Nonetheless, caution is required when interpreting this result.

It should also be noted that retrospective detection (in which participants register a complaint that would qualify as detection after the fact or during debriefing) was very rarely observed in this study, occurring in only six instances overall. The overall detection rates thus closely resemble those for concurrent detection alone.

3.1 Regression analysis

The second empirical goal was to assess the factors that mediate detection rates. Inspection of the raw correlations provides a number of suggestive results (Table 1). Detection (defined here as either concurrent of retrospective) exhibited a positive association with reaction time, level of financial sophistication, and the memory task. Conversely, having a pension and the number of funds selected were negatively associated with detection rates. Several other correlations are of note. First, the degree of co-movement between our two risk preference measures was very high (55%). Second, the magnitude of the relationship between being enrolled in a pension and financial sophistication was relatively high (38%). While speculating on any causal association is beyond the scope of this paper, omitting either variable from the regression analysis is likely to bias the results.

Table 1: Raw correlation matrix

To isolate the effects of the main independent variables of interest on Choice Blindness detection, a random effects logistic regression model was estimated. The dependent variable was a binary variable indicating whether detection (either concurrent or retrospective) occurred. Both Similar and Dissimilar conditions were included in the analysis. The main explanatory variables of interest were memory task performance, the presence of the cheater detection prime, risk preference (survey), financial sophistication (a proxy for familiarity effects), the number of funds selected (a proxy for decision complexity) and reaction time (which is taken as a indicator of task engagement). Control variables were also included for age, gender, education and pension enrolment. The results are presented in Table 2.

Table 2: Random effects logit model. Dependent variable is Detection (1=yes; 0=no)

^† p-values correspond to one-tailed hypotheses (with the exception of the control variables—gender, age and pension—which are computed using a two-tailed hypothesis.)

^* p <.05

^‡ Marginal effects computed at the mean.

Variable key: Gender (1=female, 0=male); Pension (1=yes 0=No); Cheater detection (1=yes 0=No); Reaction time (in minutes); Financial sophistication (0 (lowest) – 5 (highest)); Memory task (0 (lowest) to 5 (highest)); Risk survey (1 (risk averse) – 5 (risk seeker)); Education (1 (lowest) to 10 (highest); Number of funds picked min 3 – max 6.

A number of intuitive results emerge from these data. Firstly, reaction time was positively related to detection rates (p<0.05), suggesting that the more engaged a participant was in the task, the more likely they were to detect a manipulation. Financial sophistication, which can be interpreted as a proxy for familiarity, exhibited a positive relationship with detection rates (p<0.05). Finally, the number of funds selected for each portfolio was negatively associated with detection (p<0.05), implying that manipulations were less likely to be perceived as portfolios became more complex. Memory task performance and the cheater detection prime did not affect detection statistically (p>0.05).

The marginal effects, presented in the third column of table 2, provide a more intuitive interpretation of these results. For each additional minute spent constructing portfolios, participants were 4 percentage points more likely to detect a manipulation. Each additional question answered correctly on the financial sophistication questionnaire corresponded to 8 percentage point increase in detection. For portfolio complexity, adding one extra fund reduced the probability of detection by 6 percentage points. Finally there was a large (and statistically significant) discrete effect for the pension dummy variable—those enrolled in a pension were 25 percentage points less likely to detect a manipulation. Possibly they were less interested in the task.

3.2 Qualitative analysis

The experiment featured a total of 398 experimental trials over its 98 participants, each with an associated audio recording of the participant’s response. Participants spent an average of 4.7 minutes constructing each portfolio, and described each choice with an average of 11 words. 174 trials involved a preference manipulation. Of these, 43 trials were detected by the participant concurrently, and 10 more in retrospect. A further 6 responses were dropped due to recording errors. The remaining 115 responses represent instances where the original choice was manipulated, but not detected by the participant. Responses in this “choice blind” group contained 13.2 words on average, but these participants spent slightly less time contemplating each choice, at 4.4 minutes on average. Given that the average participant selected approximately 4 funds per portfolio, this amounts to a very rich data source and reflects the degree of involvement demanded by the task.

Analysis of these responses identified 6 categories into which the responses fell: (1) Major confabulations; (2) minor confabulations; (3) descriptions of original choice; (4) bunched or undifferentiated responses; (5) spurious explanations; and (6) no explanation. To confirm reliability, the response data ratings were then submitted to three independent raters, who categorized the responses separately. Inter-rater reliability was assessed and was found to be consistently high, with 71% agreement, and a Fleiss kappa value of 0.52. Furthermore, when the categories are grouped into confabulatory responses (1 and 2), descriptions of original choice (3), and no clear explanation (4, 5 and 6), we observe 76.1% agreement and a Fleiss’ kappa of 0.58. Both of the reported kappa values were significant at the 1% level. The results of this analysis are presented in Table 3.

Table 3: transcripts of verbal reports

4 Discussion

The study’s principal finding was that across all conditions 28.5% to 37.2% of manipulated trials were detected by participants. Even with the most detection-favourable conditions and sampling assumptions, only 50% of manipulations were detected. This is in line with the main hypothesis, and also consistent with the findings of previous Choice Blindness experiments (Johansson et al., 2005, Hall et al., 2010), suggesting that the Choice Blindness effect does indeed extend into the domain of personal finance choices. Such findings offer further support for the proposition that Choice Blindness is a general cognitive effect, which can exist irrespective of the specific nature of the decision.

The study also considered a number of auxiliary measures intended to isolate some of the factors which might predict the presence of Choice Blindness—reaction time, recall accuracy, risk preference, financial sophistication and the presence or absence of cheater detection primes. Given that the design offered participants some freedom to choose portfolios of greater or lesser complexity, the number of funds chosen in each portfolio was also considered.

Regression analysis of these factors found both reaction time and the number of funds picked to be significant predictors of detection, and the magnitude of these effects is relatively large. The Ordinary Least Squares (OLS) specification perhaps gives the most intuitive interpretation of the results, implying that for each additional minute a person spends contemplating their initial decision, they are 4.0 percentage points more likely to detect a subsequent manipulation. Similarly for each additional fund selected, a person is 6.8 percentage points less likely to notice a manipulation of their portfolio. Recall accuracy, risk preference, and cheater detection were not found to be significant predictors.

The significance of reaction time (i.e., the time a participant took to select a portfolio) is an encouraging result. It makes sense that participants who take longer to choose are attending more carefully to the decision, and that reaction time can be interpreted as at least a partial proxy measure of that degree of care.

In their original study, Johansson et al. (2005) controlled reaction time across three conditions (participants were either given two seconds to make a choice, five seconds, or as long as they wished) and found detection rates to increase when unlimited time was allowed (Johansson et al., 2005). It is heartening to see this finding still holds in the present study’s more complex and naturalistic decision environment. More importantly, such a finding is consistent with general observations that cognitive attention and engagement can ameliorate the effects of bias (Kahneman, 2011), and Choice Blindness functions similarly to other cognitive biases of decision-making in this regard.

The significance of the number of funds selected by the participant is again intuitive. A manipulation of one of three options is more conspicuous that one of six. Earlier Choice Blindness studies have generally limited themselves to experiments involving straightforward, binary choices (Johansson, Hall & Chater, 2011). In designing this study, however, it soon became apparent that such a format was not consistent with the goal of a plausible, naturalistic experiment in the context of personal finance, and that a multi-attribute, free-choice environment—itself an extension of the paradigm—would be required.

The significance of a financial sophistication effect (i.e., a familiarity effect) is likewise a highly intuitive finding, suggesting that Choice Blindness can be ameliorated by familiarity with a particular domain.

The null results for the other auxiliary measures also merit brief consideration. The absence of a recall or memory effect (a correlation between better recall of task details and detection, for example) is of note in light one of the most straightforward objections to the Choice Blindness phenomenon, namely that it represents a simple failure of memory or confusion effect and that no real confabulation or preference change occurs. The link between choices and recollection is not always straightforward (Henkel & Mather, 2007), and the instrument used to measure task recall in this instance was admittedly imperfect, but some relationship would nevertheless be expected if this explanation were accurate (Johansson, Hall & Sikström 2008).

The absence of any risk-preference effect is similarly interesting, given that the experiment provided participants with choices along a risk-and-return spectrum. It may be that the relationship intuited between (financial) risk aversion and general cautiousness and care-taking was overestimated here. The inclusion of a cheater-detection condition was a somewhat speculative inclusion, and the null finding here may be due to insufficient priming, or the lack of any overtly suspicious behaviour by the experimenters (the relative rarity of retrospective detection, in which participants admit to being suspicious only at the end of the experiment, would support this interpretation).

Following previous studies (Johansson et al., 2005, 2006) qualitative analysis was conducted by transcribing the verbal explanations provided by participants, isolating those involving undetected manipulations (i.e., those that were choice blind), classifying them into categories, and ensuring the reliability of such classification through a panel of independent raters. Such an approach is particularly important in gauging the strength of the Choice Blindness effect—statistically, participants are said to be “choice blind” if they fail to either explicitly or implicitly detect a manipulation when asked to explain their preferences, regardless of what (if any) alternative explanation they provide. By analysing the responses of participants in greater detail, the finding of the statistical presence of Choice Blindness can be given much greater texture and substance, something that is crucial if we are to distinguish instances of significant preference reversal from inattention, lack of well-established preferences, or disengagement with the experimental task.

The 115 non-detecting responses were categorized as major confabulations, in which the participant provides significant details only relevant to the manipulated choice (for example, expressing a preference for investing in Asian markets when their original choice had been explicitly European or US-only), minor confabulations, in which the participant provides at least some detail contradicting their original choice (e.g., expressing a preference for risky options when their original choice was explicitly low-risk) or descriptions of original choice, in which the explanation is clearly applicable to their original decision. The other, miscellaneous categories were bunched explanations (in which the participant’s explanation is too general to yield any specific details about the particular option manipulated), spurious explanations (in which the participant does not give any coherent or relevant information) and no explanation (in which the participant is silent, trails off, or is otherwise unable to respond).

Aggregate ratings of these responses (of choice–blind participants on manipulated trials) found that 20% of responses were notable confabulations displaying some degree of preference change, while  62% were descriptions of the participant’s original choice. Generic, incoherent or empty explanations accounted for the remaining 18%. It should be noted here that raters were instructed to err on the side of caution in case of any ambiguity (e.g., detail which could conceivably apply to either the original or manipulated choice), so the estimates of confabulation and preference change are necessarily conservative.

Taken together with our quantitative findings, these results suggest a multitude of factors underpinning the Choice Blindness effect in this context. 20% of responses show evidence of genuinely unstable preferences, to the extent that they can be induced to take ownership of choices completely inconsistent with their original preference. Conversely, 18% of responses were clearly results of lack of engagement with the task, and whether this is an artefact of the experimental context, the specific subject matter (i.e. pensions) or other factors is unclear. The most contentious interpretation concerns those responses which consistently explain manipulated choices with details of the original choice, but which are not consistent with the makeup of the portfolio actually received. These participants appear to be attending to the task (as evidenced by the amount of time spent and number of words used to describe the decision), but also appear inattentive to the specific characteristics of individual choices. This may appear inconsequential in instances where the overall risk profile remains unchanged. However, in other circumstances participants appear indifferent in the face of profiles with dramatically altered risk levels, highly inconsistent with their original preferences—a finding of some practical concern. Overall, the qualitative evidence presented here suggests that the observed Choice Blindness is a pervasive cognitive bias driven by a combination of weak or unstable preferences and inattention to specific choice characteristics.

We now consider a number of issues these results raise for the existing literature. Firstly, from a practical standpoint, these findings emphasise the importance of ensuring that choices and options in important contexts are as clear and unambiguous as possible. The materials featured both distinct, immediately memorable and more generic or vague-sounding options, all closely based on extracts from actual pension plans, and it is evident from the results that larger numbers of the latter serve to reduce detection rates. In a more general sense, the finding that a majority of people can fail to detect portfolio manipulations underscores the importance of an optimal default option in any auto-enrolment pension system.

We have also previously discussed the possible theoretical implications of Choice Blindness for economic theories of preference, from a behavioural perspective in particular. Previous Choice Blindness researchers have similarly noted that the effect raises questions for economists “to ponder in their modelling” (Hall et al., 2010). However, great care must be taken when drawing even mild conclusions from such tentative results, and a full discussion of these issues is beyond the scope of this paper in any case. These findings do, however, suggest several important directions in which Choice Blindness could profitably be extended into the economic domain. Three areas are worth discussing in particular.

Firstly, it would prove valuable to examine this effect in a variety of different decision domains. Decisions about pensions are certainly among the most significant economic choices most individuals will ever make, but they are hardly the most familiar or salient to most people in any general sense. If agents have “stronger” and more defined preferences for some decisions relative to others, it would be important to investigate the extent to which Choice Blindness holds in different contexts—does someone who easily detects manipulations in one context perform similarly in others? Choice Blindness has potential to be a valuable instrument for drawing out “preference maps” and differentiating areas in which preferences are strongly or weakly held—and by extension the contexts in which they are most malleable.

Secondly, more research is needed to establish the longevity of Choice Blindness effects, particularly in situations in which significant confabulations or preference reversals arise. For example, do confabulated preferences persist when tested over the longer term?

Finally, and most importantly, we believe there is a particular need for Choice Blindness studies which effectively incentivize their participants. Incentivisation is a difficult issue to navigate, and often a challenge in behavioural economic research (see discussion in Read, 2005), but in our view it is absolutely critical that this challenge be taken up. No matter how realistic its premise or context, a hypothetical decision will always be a hypothetical decision, and choice data from participants who know that their decision has no real consequences can tell us only so much, particularly in a field that values monetized decisions so highly. Well-constructed field experiments such as (Hall et al., 2010) go some considerable way towards addressing this issue, but still fall short of the critical point of material consequences.

An experiment with “real” decisions of true significance might be impossible, but it should be possible to devise designs with even rudimentary monetary consequences for participants. Until this situation is examined, there will remain significant limits on what can be said about Choice Blindness and economic ideas of preference and choice.