7 - Bayesian Methods

Summary

Introduction

The approaches we have examined thus far belong to a school called the frequentist school. Frequentists use the likelihood function to calculate the probability of observing a particular set of data for a given value of the statistic, that is, given some set of observations, say x, the frequentist approach is to calculate the probability of occurrence of the observed data for some given statistic θ. Symbolically, this is written as P(x|θ), that is “the probability of x given θ.”

The Bayesian approach is different in that it reverses the probability statement and asks “what is the probability of θ given x,” which can be written as P(θ|x). To apply the Bayesian approach we require a prior probability distribution for the statistic θ. The current data are then used to modify this probability, thereby forming a posterior probability distribution. Where frequentists and Bayesians disagree is the case in which there is no prior probability, though as we shall see this disagreement in many cases is actually rather insignificant.

Both approaches can be useful in gaining a perspective on a set of data, in particular, the Bayesian approach can be highly informative in cases of decision making. For example, a physician may ask “what is the probability that this patient actually has a disease given that she has a set of symptoms that occur in 99% of people with the disease?