In this chapter we introduce the clustering problem and use it to motivate mixture models. We start by describing clustering in a frequentist paradigm and introduce the relevant likelihoods and latent variables. We then discuss properties of the likelihoods including invariance with respect to label swapping. Finally, we expand this discussion to describe clustering and mixture models more generally within a Bayesian paradigm. This allows us to introduce Dirichlet priors used in inferring the weight we ascribe to each cluster component from which the data are drawn. Finally, we describe the infinite mixture model and Dirichlet process priors within the Bayesian nonparametric paradigm, appropriate for the analysis of uncharacterized data that may contain an unspecified number of clusters.

In this chapter we formulate the general regression problem relevant to function estimation. We begin with simple frequentist methods and quickly move to regression within the Bayesian paradigm. We then present two complementary mathematical formulations: one that relies on Gaussian process priors, appropriate for the regression of continuous quantities, and one that relies on Beta–Bernoulli process priors, appropriate for the regression of discrete quantities. In the context of the Gaussian process, we discuss more advanced topics including various admissible kernel functions, inducing point methods, sampling methods for nonconjugate Gaussian process prior-likelihood pairs, and elliptical slice samplers. For Beta–Bernoulli processes, we address questions of posterior convergence in addition to applications. Taken together, both Gaussian processes and Beta–Bernoulli processes constitute our first foray into Bayesian nonparametrics. With end of chapter projects, we explore more advanced modeling questions relevant to optics and microscopy.