Kolmogorov conditionalization is a strategy for updating credences based on propositions that have initial probability 0. I explore the connection between Kolmogorov conditionalization and Dutch books. Previous discussions of the connection rely crucially upon a factivity assumption: they assume that the agent updates credences based on true propositions. The factivity assumption discounts cases of misplaced certainty, i.e., cases where the agent invests credence 1 in a falsehood. Yet misplaced certainty arises routinely in scientific and philosophical applications of Bayesian decision theory. I prove a non-factive Dutch book theorem and converse Dutch book theorem for Kolmogorov conditionalization. The theorems do not rely upon the factivity assumption, so they establish that Kolmogorov conditionalization has unique pragmatic virtues that persist even in cases of misplaced certainty.

The calibration of probability or confidence judgments concerns the association between the judgments and some estimate of the correct probabilities of events. Researchers rely on estimates using relative frequencies computed by aggregating data over observations. We show that this approach creates conceptual problems, and may result in the confounding of explanatory variables or unstable estimates. To circumvent these problems we propose using probability estimates obtained from statistical models—specifically mixed models for binary data—in the analysis of calibration. We illustrate this methodology by re-analyzing data from a published study and comparing the results from this approach to those based on relative frequencies. The model-based estimates avoid problems with confounding variables and provided more precise estimates, resulting in better inferences.

When people judge risk or the probability of a risky prospect, single case narratives can bias judgments when a statistical base-rate is also provided. In this work we investigate various methodological and procedural factors that may influence this narrative bias. We found that narratives had the strongest effect on a non-numerical risk measure, which was also the best predictor of behavioral intentions. In contrast, two scales for subjective probability reflected primarily statistical variations. We observed a negativity bias on the risk measure, such that the narratives increased rather than decreased risk perceptions, whereas the effect on probability judgments was symmetric. Additionally, we found no evidence that the narrative bias is solely produced by adherence to conversational norms. Finally, changing the absolute number of narratives reporting the focal event, while keeping their relative frequency constant, had no effect. Thus, individuals extract a representation of likelihood from a sample of single-case narratives, which drives the bias. These results show that the narrative bias is in part dependent on the measure used to assess it and underline the conceptual distinction between subjective probability and perceived risk.

Overprecision is the most robust type of overconfidence. We present a new method that significantly reduces this bias and offers insight into its underlying cause. In three experiments, overprecision was significantly reduced by forcing participants to consider all possible outcomes of an event. Each participant was presented with the entire range of possible outcomes divided into intervals, and estimated each interval’s likelihood of including the true answer. The superiority of this Subjective Probability Interval Estimate (SPIES) method is robust to range widths and interval grain sizes. Its carryover effects are observed even in subsequent estimates made using the conventional, 90% confidence interval method: judges who first made SPIES judgments considered a broader range of values in subsequent conventional interval estimates as well.

We apply the principles of the “Wisdom of Crowds (WoC)” to improve the calibration of interval estimates. Previous research has documented the significant impact of the WoC on the accuracy of point estimates but only a few studies have examined its effectiveness in aggregating interval estimates. We demonstrate that collective probability intervals obtained by several heuristics can reduce the typical overconfidence of the individual estimates. We re-analyzed data from Glaser, Langer and Weber (2013) and from Soll and Klayman (2004) and applied four heuristics Averaging, Median, Enveloping, Probability averaging-suggested by Gaba, Tsetlin and Winkler (2014) and new heuristics, Averaging with trimming and Quartiles. We used the hit rate and the Mean Squared Error (MSE) to evaluate the quality of the methods. All methods reduced miscalibration to some degree, and Quartiles was the most beneficial securing accuracy and informativeness.

Redundant or excessive information can sometimes lead people to lean on it unnecessarily. Certain experimental designs can sometimes bias results in the researcher’s favor. And, sometimes, interesting effects are too small to be studied, practically, or are simply zero. We believe a confluence of these factors led to a recent paper (Isaac & Brough, 2014, JCR). This initial paper proposed a new means by which probability judgments can be led astray: the category size bias, by which an individual event coming from a large category is judged more likely to occur than an event coming from a small one. Our work shows that this effect may be due to instructional and mechanical confounds, rather than interesting psychology. We present eleven studies with over ten times the sample size of the original in support of our conclusion: We replicate three of the five original studies and reduce or eliminate the effect by resolving these methodological issues, even significantly reversing the bias in one case (Study 6). Studies 7–8c suggest the remaining two studies are false positives. We conclude with a discussion of the subtleties of instruction wording, the difficulties of correcting the record, and the importance of replication and open science.

The paper investigates the validity of individual perceptions of heart disease risks, and examines how information and risk perceptions affect marginal willingness to pay (MWTP) to reduce risk, using data from a stated preference survey. Results indicate that risk perceptions held before receiving risk information are plausibly related to objective risk factors and reflect individual-specific information not found in aggregate measures of objective risk. After receiving information, individuals’ updates of prior risk assessments are broadly consistent with Bayesian learning. Perceived heart disease risks thus satisfy construct validity and provide a valid basis for inferring MWTP to reduce risk. Consistent estimators of the relationship of MWTP to endogenously perceived risk are developed. Estimating MWTP based on objective rather than subjective risks causes misleading inferences about benefits of risk reduction. An empirical case study shows that estimated benefits may be as much as 60–98 % higher when estimated using individuals’ heterogeneous perceptions of risk than when using aggregate estimates of objective risk. The main contributions include assessing the validity of risk perceptions and their updating, consistently estimating the relationship between MWTP and endogenously perceived risk, and demonstrating the importance of employing risk perception information for accurate benefit measurement.

This article discusses how to update one’s credences based on evidence that has initial probability 0. I advance a diachronic norm, Kolmogorov Conditionalization, that governs credal reallocation in many such learning scenarios. The norm is based upon Kolmogorov’s theory of conditional probability. I prove a Dutch book theorem and converse Dutch book theorem for Kolmogorov Conditionalization. The two theorems establish Kolmogorov Conditionalization as the unique credal reallocation rule that avoids a sure loss in the relevant learning scenarios.

Forecast distributions based on historical yields and subjective expectations for 1987 expected crop yields were compared for 90 Western Kentucky grain farms. Different subjective probability elicitation techniques were also compared. In many individual cases, results indicate large differences between subjective and empirical moments. Overall, farmer expectations for 1987 corn yields were below those predicted from their past yields, while soybean expectations were above the historical forecast. Geographical location plays a larger role than crop in comparisons of relative variability of yield. Neither elicitation technique nor manager characteristics have significant effects on the comparisons of the forecasts.

To understand the endogenous components of the event-related brain potential (ERP), we must use data about the components' antecedent conditions to form hypotheses about the information-processing function of the underlying brain activity. These hypotheses, in turn, generate testable predictions about the consequences of the component. We review the application of this approach to the analysis of the P300 component. The amplitude of the P300 is controlled multiplicatively by the subjective probability and the task relevance of the eliciting events, whereas its latency depends on the duration of stimulus evaluation. These and other factors suggest that the P300 is a manifestation of activity occurring whenever one's model of the environment must be revised. Tests of three predictions based on this “context updating” model are reviewed. Verleger's critique is based on a misconstrual of the model as well as a partial and misleading reading of the relevant literature.

Objectives: A treatment pathway model was developed to examine the costs and benefits of the current bowel cancer service in England and to evaluate potential alternatives in service provision. To use the pathway model, various parameters and probability distributions had to be specified. They could not all be determined from empirical evidence and, instead, expert opinion was elicited in the form of statistical quantities that gave the required information. The purpose of this study is to describe the procedures used to quantify expert opinion and note examples of good practice contained in the case study.

Methods: The required information was identified and preparatory discussion with four experts refined the questions they would be asked. In individual elicitation sessions they quantified their opinions, mainly in the form of point and interval estimates for specified variables. New methods have been developed for quantifying expert opinion and these were implemented in specialized software that uses interactive graphics. This software was used to elicit opinion about quantities related to measurable covariates.

Results: Assessments for thirty-four quantities were elicited and available checks supported their validity. Eight points of good practice in eliciting and using expert judgment were evident. Parameters and probability distributions needed for the pathway model were determined from the elicited assessments. Simulation results from the pathway model were used to inform policy on bowel cancer service provision.

Conclusions: The study illustrates that quantifying and using expert judgment can be acceptable in real problems of practical importance. For full benefit to be gained from expert knowledge, elicitation must be conducted carefully and should be reported in detail.