2 - Maximum Likelihood

Summary

Introduction

Suppose that we have a model with a single parameter, θ, that predicts the outcome of an event that has some numerical value y. Further, suppose we have two choices for the parameter value, say θ₁ and θ₂, where θ₁ predicts that the numerical value of y will occur with a probability p₁ and θ₂ predicts that the numerical value of y w`ill occur with a probability p₂. Which of the two choices of θ is the better estimate of the true value of θ? It seems reasonable to suppose that the parameter value that gave the highest probability of actually observing what was observed would be the one that is also closer to the true value of θ. For example, if p₁ equals 0.9 and p₂ equals 0.1, then we would select θ₁ over θ₂, because the model with θ₂ predicts that one is unlikely to observe y, whereas the model with θ₁ predicts that one is quite likely to observe y. We can extend this idea to many values of θ by writing our predictive model as a function of the parameter values, ϕ(θ_i) = p_i, where i designates particular values of θ. More generally, we can dispense with the subscript and write ϕ(θ) = p, thereby allowing θ to take on any value. By the principle of maximum likelihood we select the value of θ that has the highest associated probability, p.