6 - Hypothesis Testing

Summary

This chapter discusses the theory of hypothesis testing under the Neyman-Pearson Lemma. The concepts of null and alternate hypottheses and distributions, type 1 error, power and p-values are introduced, including several cautionary notes about the latter. The commonly used parametric hypothesis tests (z, t, chi square, F and Bartlett's M) are defined and illustrated with data, including power calculation. The distribution of the correlation coefficient is derived and implemented with Matlab. One-way analysis of variance as a method to compare the means of two or more groups is elucidated. The relationship between hypothesis tests and confidence intervals is established, Likelihood ratio tests for simple and composiite hypotheses are defined, and shown to be widely optimal. The Wald and score tests that are simple, but asymptotically equivalent to a likelihood ratio test, are introduced. The simultaneous testing of multiple hypotheses is then described, including the seminal Benjamini-Hochberg approach.