Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-28T06:58:33.216Z Has data issue: false hasContentIssue false

Chapter 9 - Biased Preferences through Exploitation

from Part III - Consequences of Selective Sampling

Published online by Cambridge University Press:  01 June 2023

By
Chris Harris  and
Ruud Custers
Edited by
Klaus Fiedler ,
Peter Juslin  and
Jerker Denrell
Klaus Fiedler
Affiliation:
Universität Heidelberg
Peter Juslin
Affiliation:
Uppsala Universitet, Sweden
Jerker Denrell
Affiliation:
University of Warwick
Get access

Summary

In the present chapter we investigate how reward-rich environments can lead to the persistence of (initial) biases. More specifically, we argue that frequent rewards invite the exploitation of a supposedly best option which in turn will reinforce the biased preference. Because feedback is often contingent on the choices made, exploitation will result mostly in the aggregation of information about the exploited option. This, in turn, restricts the extent to which beliefs can be updated, with downstream consequences for further decisions. This dynamic might be responsible for why false beliefs about the outcomes of behavioral options can be maintained even when decision makers are motivated to choose the best choice alternative. We present data from simulations and empirical work to support this argument and conclude that the exploration–exploitation tradeoff serves as a particularly vivid example of the interplay between one’s cognition (goal-directed) behavior, and the sample that is aggregated.

Keywords

exploration–exploitation tradeoffreward pursuitexperience samplingpseudocontingenciespositive testingconfirmation bias
Type
Chapter
Information
Sampling in Judgment and Decision Making , pp. 207 - 221
Publisher: Cambridge University Press
Print publication year: 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, N. H. (1965). Primacy effects in personality impression formation using a generalized order effect paradigm. Journal of Personality and Social Psychology, 2(1), 1–9. doi: 10.1037/h0021966Google Scholar
Asch, S. E. (1946). Forming impressions of personality. Journal of Abnormal and Social Psychology, 41(3), 258–290. doi: 10.1037/h0055756Google Scholar
Bott, F. M., Kellen, D., & Klauer, K. C. (2021). Normative accounts of illusory correlations. Psychological Review, 128(5), 856–878. doi: 10.1037/rev0000273CrossRefGoogle ScholarPubMed
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 933–942. doi: 10.1098/rstb.2007.2098Google Scholar
Costello, F., & Watts, P. (2019). The rationality of illusory correlation. Psychological Review, 126(3), 437–450. doi: 10.1037/rev0000130CrossRefGoogle ScholarPubMed
Custers, R., & Aarts, H. (2010). The unconscious will: How the pursuit of goals operates outside of conscious awareness. Science, 329(5987), 47–50. doi:10.1126/science.1188595CrossRefGoogle ScholarPubMed
Dennis, M. J., & Ahn, W.-K. (2001). Primacy in causal strength judgments: The effect of initial evidence for generative versus inhibitory relationships. Memory & Cognition, 29(1), 152–164. doi: 10.3758/BF03195749CrossRefGoogle ScholarPubMed
Denrell, J. (2005). Why most people disapprove of me: Experience sampling in impression formation. Psychological Review, 112(4), 951–978. doi: 10.1037/0033-295X.112.4.951Google Scholar
Denrell, J., & Le Mens, G. (2012). Social judgments from adaptive samples. In Krueger, J. I. (Ed.), Social judgment and decision making (pp. 151–169). New York: Psychology Press.Google Scholar
Denrell, J., & March, J. G. (2001). Adaptation as information restriction: The hot stove effect. Organization Science, 12(5), 523–538. doi: 10.1287/orsc.12.5.523.10092Google Scholar
Feingold, A. (1992). Good-looking people are not what we think. Psychological Bulletin, 111(2), 304–341.Google Scholar
Fetchenhauer, D., & Dunning, D. (2010). Why so cynical? Asymmetric feedback underlies misguided skepticism regarding the trustworthiness of others. Psychological Science, 21(2), 189–193. doi: 10.1177/0956797609358586Google Scholar
Fiedler, K. (1996). Explaining and simulating judgment biases as an aggregation phenomenon in probabilistic, multiple-cue environments. Psychological Review, 103(1), 193–214.Google Scholar
Fiedler, K. (2010). Pseudocontingencies can override genuine contingencies between multiple cues. Psychonomic Bulletin & Review, 17(4), 504–509. doi: 10.3758/PBR.17.4.504Google Scholar
Fiedler, K., & Freytag, P. (2004). Pseudocontingencies. Journal of Personality and Social Psychology, 87(4), 453–467. doi: 10.1037/0022-3514.87.4.453Google Scholar
Fiedler, K., Freytag, P., & Meiser, T. (2009). Pseudocontingencies: An integrative account of an intriguing cognitive illusion. Psychological Review, 116(1), 187–206. doi: 10.1037/a0014480Google Scholar
Fiedler, K., Walther, E., & Nickel, S. (1999). The auto-verification of social hypotheses: Stereotyping and the power of sample size. Journal of Personality and Social Psychology, 77(1), 5–18. doi: 10.1037/0022-3514.77.1.5CrossRefGoogle Scholar
Fiedler, K., & Wänke, M. (2009). The cognitive-ecological approach to rationality in social psychology. Social Cognition, 27(5), 699–732. doi: 10.1521/soco.2009.27.5.699Google Scholar
Gaissmaier, W., & Schooler, L. J. (2008). The smart potential behind probability matching. Cognition, 109(3), 416–422.Google Scholar
Gervais, W. M., Shariff, A. F., & Norenzayan, A. (2011). Do you believe in atheists? Distrust is central to anti-atheist prejudice. Journal of Personality and Social Psychology, 101(6), 1189–1206. doi: 10.1037/a0025882Google Scholar
Harris, C., Aarts, H., Fiedler, K., & Custers, R. (2022a). Iterative cycles in psychology: How hedonic outcomes, idiosyncratic experiences, and decisions by experience shape our subjective world. Manuscript in preparation.Google Scholar
Harris, C., Aarts, H., Fiedler, K., & Custers, R. (2022b). Missing out by pursuing rewarding outcomes: Why initial biases can lead to persistent suboptimal choices. Manuscript submitted for publication.Google Scholar
Harris, C., Fiedler, K., Marien, H., & Custers, R. (2020). Biased preferences through exploitation: How initial biases are consolidated in reward-rich environments. Journal of Experimental Psychology: General, 149(10), 1855–1877.CrossRefGoogle ScholarPubMed
Hills, T. T., Todd, P. M., Lazer, D., Redish, A. D., & Couzin, I. D. (2015). Exploration versus exploitation in space, mind, and society. Trends in Cognitive Sciences, 19(1), 46–54. doi: https://doi.org/10.1016/j.tics.2014.10.004CrossRefGoogle ScholarPubMed
Jaeger, B., Harris, C., Custers, R., & Todorov, A. (2022). Biased partner selection propagates stereotypes and inequalities in social exchange. Manuscript submitted for publication.Google Scholar
Jones, E. E., Goethals, G. R., Kennington, G. E., & Severance, L. J. (1972). Primacy and assimilation in the attribution process: The stable entity proposition. Journal of Personality, 40(2), 250–274.Google Scholar
Kareev, Y., & Fiedler, K. (2006). Nonproportional sampling and the amplification of correlations. Psychological Science, 17(8), 715–720. doi: 10.1111/j.1467-9280.2006.01771.xGoogle Scholar
Kareev, Y., Lieberman, I., & Lev, M. (1997). Through a narrow window: Sample size and the perception of correlation. Journal of Experimental Psychology: General, 126(3), 278–287. doi: 10.1037/0096-3445.126.3.278Google Scholar
Keasar, T., Rashkovich, E., Cohen, D., & Shmida, A. (2002). Bees in two-armed bandit situations: Foraging choices and possible decision mechanisms. Behavioral Ecology, 13(6), 757–765. doi: 10.1093/beheco/13.6.757Google Scholar
Klayman, J., & Ha, Y.-W. (1987). Confirmation, disconfirmation, and information in hypothesis testing. Psychological Review, 94(2), 211–228. doi: 10.1037/0033-295X.94.2.211Google Scholar
Klein, W. M., & Kunda, Z. (1992). Motivated person perception: Constructing justifications for desired beliefs. Journal of Experimental Social Psychology, 28(2), 145–168. doi: 10.1016/0022-1031(92)90036-JGoogle Scholar
Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108(3), 480–498. doi: 10.1037/0033-2909.108.3.480Google Scholar
Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37(11), 2098–2109. doi: 10.1037/0022-3514.37.11.2098CrossRefGoogle Scholar
Mehlhorn, K., Newell, B. R., Todd, P. M. et al. (2015). Unpacking the exploration–exploitation tradeoff: A synthesis of human and animal literatures. Decision, 2(3), 191–215. doi: 10.1037/dec0000033CrossRefGoogle Scholar
Pettigrew, T. F., & Tropp, L. R. (2006). A meta-analytic test of intergroup contact theory. Journal of Personality & Social Psychology, 90(5), 751–783. doi: 10.1037/0022-3514.90.5.751Google Scholar
Pilditch, T. D., & Custers, R. (2018). Communicated beliefs about action-outcomes: The role of initial confirmation in the adoption and maintenance of unsupported beliefs. Acta Psychologica, 184, 46–63. doi: https://doi.org/10.1016/j.actpsy.2017.04.006Google Scholar
Rakow, T., & Miler, K. (2009). Doomed to repeat the successes of the past: History is best forgotten for repeated choices with nonstationary payoffs. Memory & Cognition, 37(7), 985–1000. doi: 10.3758/MC.37.7.985CrossRefGoogle ScholarPubMed
Relton, C., Cooper, K., Viksveen, P., Fibert, P., & Thomas, K. (2017). Prevalence of homeopathy use by the general population worldwide: A systematic review. Homeopathy, 106(2), 69–78. doi: https://doi.org/10.1016/j.homp.2017.03.002Google Scholar
Rescorla, R., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non reinforcement. In Black, A. H. & Prokasy, W. F. (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York: Appleton-Century-Crofts.Google Scholar
Shang, A., Huwiler-Müntener, K., Nartey, L., et al. (2005). Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. The Lancet, 366(9487), 726–732. doi: https://doi.org/10.1016/S0140-6736(05)67177-2Google Scholar
Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 69, 99–118. doi: 10.2307/1884852Google Scholar
Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63(2), 129–138. doi: 10.1037/h0042769Google Scholar
Vulkan, N. (2000). An economist’s perspective on probability matching. Journal of Economic Surveys, 14, 101–118. doi: 10.1111/1467-6419.00106CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×