Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-76c49bb84f-7l7zb Total loading time: 0 Render date: 2025-07-06T16:03:00.772Z Has data issue: false hasContentIssue false

Chapter 21 - Approximating Bayesian Inference through Internal Sampling

from Part VI - Computational Approaches

Published online by Cambridge University Press:  01 June 2023

By
Joakim Sundh ,
Adam Sanborn ,
Jian-Qiao Zhu ,
Jake Spicer ,
Pablo León-Villagrá  and
Nick Chater
Edited by
Klaus Fiedler ,
Peter Juslin  and
Jerker Denrell
Klaus Fiedler
Affiliation:
Universität Heidelberg
Peter Juslin
Affiliation:
Uppsala Universitet, Sweden
Jerker Denrell
Affiliation:
University of Warwick
Get access

Summary

People must often make inferences about, and decisions concerning, a highly complex and unpredictable world, on the basis of sparse evidence. An “ideal” normative approach to such challenges is often modeled in terms of Bayesian probabilistic inference. But for real-world problems of perception, motor control, categorization, language comprehension, or common-sense reasoning, exact probabilistic calculations are computationally intractable. Instead, we suggest that the brain solves these hard probability problems approximately, by considering one, or a few, samples from the relevant distributions. By virtue of being an approximation, the sampling approach inevitably leads to systematic biases. Thus, if we assume that the brain carries over the same sampling approach to easy probability problems, where the “ideal” solution can readily be calculated, then a brain designed for probabilistic inference should be expected to display characteristic errors. We argue that many of the “heuristics and biases” found in human judgment and decision-making research can be reinterpreted as side effects of the sampling approach to probabilistic reasoning.

Keywords

Bayesian inferenceinternal samplingprobabilitycognitive biasesapproximationrationality

Information

Type
Chapter
Information
Sampling in Judgment and Decision Making , pp. 490 - 512
Publisher: Cambridge University Press
Print publication year: 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Psychology Press.Google Scholar
Anderson, J. R. (1991). The adaptive nature of human categorization. Psychological Review, 98(3), 409.CrossRefGoogle Scholar
Baddeley, A. (1998). Random generation and the executive control of working memory. Quarterly Journal of Experimental Psychology: Section A, 51(4), 819–852.Google Scholar
Battaglia, P. W., Hamrick, J. B., & Tenenbaum, J. B. (2013). Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences, 110(45), 18327–18332.Google Scholar
Beaumont, M. A. (2019). Approximate Bayesian computation. Annual Review of Statistics and its Application, 6, 379–403.Google Scholar
Bousfield, W. A., & Sedgewick, C. H. W. (1944). An analysis of sequences of restricted associative responses. Journal of General Psychology, 30(2), 149–165.CrossRefGoogle Scholar
Buesing, L., Bill, J., Nessler, B., & Maass, W. (2011). Neural dynamics as sampling: A model for stochastic computation in recurrent networks of spiking neurons. PLoS Computational Biology, 7(11), e1002211.Google Scholar
Castillo, L., León-Villagrá, P., Chater, N., & Sanborn, A. (2021). Local sampling with momentum accounts for human random sequence generation. Proceedings of the Annual Meeting of the Cognitive Science Society, 43. https://escholarship.org/uc/item/3gz154kgGoogle Scholar
Zhu, J.-Q., Spicer, J., Sanborn, A. N., & Chater, N. (Preprint). Cognitive variability matches speculative price dynamics. doi: 10.31234/osf.io/gfjvsGoogle Scholar
Cont, R. (2001). Empirical properties of asset returns: Stylized facts and statistical issues. Quantitative Finance, 1(2), 223–236Google Scholar
Correll, J. (2008). 1/f noise and effort on implicit measures of bias. Journal of Personality and Social Psychology, 94(1), 48.Google Scholar
Costello, F., & Watts, P. (2014). Surprisingly rational: Probability theory plus noise explains biases in judgment. Psychological Review, 121(3), 463.CrossRefGoogle ScholarPubMed
Dasgupta, I., Schulz, E., & Gershman, S. J. (2017). Where do hypotheses come from? Cognitive Psychology, 96, 1–25.CrossRefGoogle ScholarPubMed
Edwards, W. (1968). Conservatism in human information processing: Formal representation of human judgment. Hoboken, NJ: John Wiley.Google Scholar
Erev, I., Wallsten, T. S., & Budescu, D. V. (1994). Simultaneous over-and underconfidence: The role of error in judgment processes. Psychological Review, 101(3), 519.Google Scholar
Fiser, J., Berkes, P., Orbán, G., & Lengyel, M. (2010). Statistically optimal perception and learning: From behavior to neural representations. Trends in Cognitive Sciences, 14(3), 119–130.Google Scholar
Gardner, M. (1978). White and brown music, fractal curves and one-over-f fluctuations. Scientific American, 238(4), 16–27.CrossRefGoogle Scholar
Gigerenzer, G., & Selten, R. (Eds.). (2002). Bounded rationality: The adaptive toolbox. Cambridge, MA: MIT.Google Scholar
Gilden, D. L. (1997). Fluctuations in the time required for elementary decisions. Psychological Science, 8(4), 296–301.CrossRefGoogle Scholar
Gilden, D. L., Thornton, T., & Mallon, M. W. (1995). 1/f noise in human cognition. Science, 267(5205), 1837–1839.CrossRefGoogle ScholarPubMed
Hahn, U., & Oaksford, M. (2007). The rationality of informal argumentation: A Bayesian approach to reasoning fallacies. Psychological Review, 114(3), 704.CrossRefGoogle ScholarPubMed
Hayes, B. K., Banner, S., Forrester, S., & Navarro, D. J. (2019). Selective sampling and inductive inference: Drawing inferences based on observed and missing evidence. Cognitive Psychology, 113, 101221.Google Scholar
Hilbert, M. (2012). Toward a synthesis of cognitive biases: How noisy information processing can bias human decision making. Psychological Bulletin, 138(2), 211.CrossRefGoogle Scholar
Hills, T. T., Jones, M. N., & Todd, P. M. (2012). Optimal foraging in semantic memory. Psychological Review, 119(2), 431.CrossRefGoogle ScholarPubMed
Howe, R., & Costello, F. (2020). Random variation and systematic biases in probability estimation. Cognitive Psychology, 123, 101306.Google Scholar
Hoyer, P. O., & Hyvärinen, A. (2003). Interpreting neural response variability as Monte Carlo sampling of the posterior. In Becker, S., Thrun, S. & Obermayer, K. (Eds.), Advances in neural information processing systems (pp. 293–300). Cambridge, MA: MIT.Google Scholar
Juslin, P., Winman, A., & Hansson, P. (2007). The naïve intuitive statistician: A naïve sampling model of intuitive confidence intervals. Psychological Review, 114(3), 678–703.CrossRefGoogle Scholar
Kahneman, D., & Tversky, A. (1972). Subjective probability: A judgment of representativeness. Cognitive Psychology, 3(3), 430–454.Google Scholar
Kersten, D., Mamassian, P., & Yuille, A. (2004). Object perception as Bayesian inference. Annual Review of Psychology, 55, 271–304.CrossRefGoogle ScholarPubMed
Koehler, D. J., & James, G. (2009). Probability matching in choice under uncertainty: Intuition versus deliberation. Cognition, 113(1), 123–127.CrossRefGoogle ScholarPubMed
Konovalova, E., & Le Mens, G. (2020). An information sampling explanation for the in-group heterogeneity effect. Psychological Review, 127(1), 47.Google Scholar
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015). Human-level concept learning through probabilistic program induction. Science, 350(6266), 1332–1338.CrossRefGoogle ScholarPubMed
Le Mens, G., & Denrell, J. (2011). Rational learning and information sampling: On the “naivety” assumption in sampling explanations of judgment biases. Psychological Review, 118(2), 379.CrossRefGoogle ScholarPubMed
Lieder, F., & Griffiths, T. L. (2020). Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources. Behavioral and Brain Sciences, 43.CrossRefGoogle Scholar
Lieder, F., Griffiths, T. L., Huys, Q. J., & Goodman, N. D. (2018). The anchoring bias reflects rational use of cognitive resources. Psychonomic Bulletin & Review, 25(1), 322–349.Google Scholar
Lloyd, K., Sanborn, A., Leslie, D., & Lewandowsky, S. (2019). Why higher working memory capacity may help you learn: Sampling, search, and degrees of approximation. Cognitive Science, 43(12), e12805.Google Scholar
Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. Cambridge, MA: MIT.Google Scholar
Moreno-Bote, R., Knill, D. C., & Pouget, A. (2011). Bayesian sampling in visual perception. Proceedings of the National Academy of Sciences, 108(30), 12491–12496.Google Scholar
Oaksford, M., & Chater, N. (1994). A rational analysis of the selection task as optimal data selection. Psychological Review, 101(4), 608.CrossRefGoogle Scholar
Oaksford, M., & Chater, N. (2009). Bayesian rationality: The probabilistic approach to human reasoning. Oxford: Oxford University Press.Google Scholar
Oaksford, M., & Chater, N. (2020). New paradigms in the psychology of reasoning. Annual Review of Psychology, 71, 305–330.Google Scholar
Peterson, C. R., & Beach, L. R. (1967). Man as an intuitive statistician. Psychological Bulletin, 68(1), 29.Google Scholar
Redelmeier, D. A., Koehler, D. J., Liberman, V., & Tversky, A. (1995). Probability judgment in medicine: Discounting unspecified possibilities. Medical Decision Making, 15(3), 227–230.Google Scholar
Reyna, V. F., & Lloyd, F. J. (2006). Physician decision making and cardiac risk: Effects of knowledge, risk perception, risk tolerance, and fuzzy processing. Journal of Experimental Psychology: Applied, 12(3), 179.Google Scholar
Rhodes, T., & Turvey, M. T. (2007). Human memory retrieval as Lévy foraging. Physica A: Statistical Mechanics and its Applications, 385(1), 255–260.Google Scholar
Sanborn, A. N., & Chater, N. (2016). Bayesian brains without probabilities. Trends in Cognitive Sciences, 20(12), 883–893.CrossRefGoogle ScholarPubMed
Sanborn, A. N., Griffiths, T. L., & Navarro, D. J. (2010). Rational approximations to rational models: Alternative algorithms for category learning. Psychological Review, 117(4), 1144.CrossRefGoogle ScholarPubMed
Sanborn, A. N., Mansinghka, V. K., & Griffiths, T. L. (2013). Reconciling intuitive physics and Newtonian mechanics for colliding objects. Psychological Review, 120(2), 411.Google Scholar
Sanborn, A. N., Zhu, J.-Q., Spicer, J., et al. (2021). Sampling as the human approximation to probabilistic inference. In Muggleton, S. & Chater, N. (Eds). Human-like machine intelligence. Oxford: Oxford University Press.Google Scholar
Shi, L., Griffiths, T. L., Feldman, N. H., & Sanborn, A. N. (2010). Exemplar models as a mechanism for performing Bayesian inference. Psychonomic Bulletin & Review, 17(4), 443–464.CrossRefGoogle ScholarPubMed
Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 69(1), 99–118.Google Scholar
Sloman, S., Rottenstreich, Y., Wisniewski, E., Hadjichristidis, C., & Fox, C. R. (2004). Typical versus atypical unpacking and superadditive probability judgment. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30(3), 573.Google ScholarPubMed
Stewart, N., Chater, N., & Brown, G. D. (2006). Decision by sampling. Cognitive Psychology, 53(1), 1–26.Google Scholar
Sundh, J., Zhu, J. Q., Chater, N., & Sanborn, A. (2021, July 30). The mean-variance signature of Bayesian probability judgment. https://doi.org/10.31234/osf.io/yuhazGoogle Scholar
Tentori, K., Crupi, V., & Russo, S. (2013). On the determinants of the conjunction fallacy: Probability versus inductive confirmation. Journal of Experimental Psychology: General, 142(1), 235.Google Scholar
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.Google Scholar
Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90(4), 293.Google Scholar
Van Rooij, I. (2008). The tractable cognition thesis. Cognitive Science, 32(6), 939–984.Google Scholar
Van Rooij, I., & Wareham, T. (2012). Intractability and approximation of optimization theories of cognition. Journal of Mathematical Psychology, 56(4), 232–247.CrossRefGoogle Scholar
Viswanathan, G. M., Buldyrev, S. V., Havlin, S., et al. (1999). Optimizing the success of random searches. Nature, 401(6756), 911–914.Google Scholar
Vul, E., Goodman, N., Griffiths, T. L., & Tenenbaum, J. B. (2014). One and done? Optimal decisions from very few samples. Cognitive Science, 38(4), 599–637.Google Scholar
Wagenaar, W. A. (1972). Generation of random sequences by human subjects: A critical survey of literature. Psychological Bulletin, 77(1), 65.CrossRefGoogle Scholar
Zhu, J.-Q., Leon-Villagra, P., Chater, N., & Sanborn, A. N. (2022). Understanding the structure of cognitive noise. PLOS Computational Biology, 18(8). doi: 10.1371/journal.pcbi.1010312Google Scholar
Zhu, J. Q., Sanborn, A. N., & Chater, N. (2018). Mental sampling in multimodal representations. In Bengio, S., Wallach, H., Larochelle, H. et al. (Eds.), Advances in neural information processing systems 31. Montreal: Curran Associates.Google Scholar
Zhu, J. Q., Sanborn, A., & Chater, N. (2019). Why decisions bias perception: An amortised sequential sampling account. Proceedings of the Annual Meeting of the Cognitive Science Society, 41, 3220–3226.Google Scholar
Zhu, J. Q., Sanborn, A. N., & Chater, N. (2020). The Bayesian sampler: Generic Bayesian inference causes incoherence in human probability judgments. Psychological Review, 127(5), 719–748.Google Scholar
Zhu, J.-Q., Spicer, J., Sanborn, A. N., & Chater, N. (Preprint). Cognitive variability matches speculative price dynamics. doi: 10.31234/osf.io/gfjvsGoogle Scholar
Zhu, J., Sundh, J., Spicer, J., Chater, N., & Sanborn, A. N. (2021, February 4). The autocorrelated Bayesian sampler: A rational process for probability judgments, estimates, confidence intervals, choices, confidence judgments, and response times. https://doi.org/10.31234/osf.io/3qxf7Google Scholar

Accessibility standard: Unknown

Accessibility compliance for the PDF of this book is currently unknown and may be updated in the future.

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×