5 - M–and Z-Estimators

Summary

This chapter gives an introduction to the consistency and asymptotic normality of M -estimators and Z-estimators. Maximum likelihood estimators are treated as a special case.

Introduction

Suppose that we are interested in a parameter (or “functional“) attached to the distribution ofobservations, …,A popular method for finding an estimator is to maximize a criterion function of the type

Here are known functions. An estimator maximizing over is called an M -estimator. In this chapter we investigate the asymptotic behavior of sequences of M -estimators.

Often the maximizing value is sought by setting a derivative (or the set of partial derivatives in the multidimensional case) equal to zero. Therefore, the name M -estimator is also used for estimators satisfying systems of equations of the type

Here are known vector-valued maps. For instance, if is k-dimensional, thentypically has k coordinate functions and (5.2) is shorthand for the system of equations

Even though in many examples is the partial derivative of some function this is irrelevant for the following. Equations, such as (5.2), defining an estimator are called estimating equations and need not correspond to a maximization problem. In the latter case it is probably better to call the corresponding estimators Z-estimators (for zero), but the use of the name M -estimator is widespread.

Sometimes the maximum of the criterion function M_n is not taken or the estimating equation does not have an exact solution. Then it is natural to use as estimator a value that almost maximizes the criterion function or is a near zero. This yields approximate M-estimators or Z-estimators. Estimators that are sufficiently close to being a point of maximum or a zero often have the same asymptotic behavior.

An operator notation for taking expectations simplifies the formulas in this chapter. We write P for the marginal law of the observations which we assume to be identically distributed. Furthermore, we write for the expectation and abbreviate the average. Thus P_n is the empirical distribution: the (random) discrete distribution that puts mass at every of the observations.