16 - Bayesian estimation method

Summary

This chapter provides an overview of the classical Bayesian method for point estimation. The main point of departure of this method from other methods is that it considers the unknown x as a random variable. All the prior knowledge about this unknown is summarized in the form of a known prior distribution p(x) of x. If z is the set of observations that contains information about the unknown x, this distribution is often given in the form of a conditional distribution p(z│x). The basic idea is to combine these two pieces of information to obtain an optimal estimate of x, called the Bayes estimate.

The Bayesian framework is developed in Section 16.1. Special classes of Bayesian estimators – Bayes least squares estimate leading to the conditional mean (which is also the minimum variance estimate), conditional mode, and conditional median estimates are derived in Section 16.2.

The Bayesian framework

Let x ∈ ℝⁿ be the unknown to be estimated and z ∈ ℝ^m be the observations that contain information about the unknown x to be estimated. The distinguishing feature of the Bayes framework is that it also treats the unknown x as a random variable. It is assumed that a prior distributionp(x) is known. This distribution summarizes our initial belief about the unknown. It is assumed that nature picks a value of x from the distribution p(x) but decides to tease us by not disclosing her choice, thereby defining a game.