4 - Gaussian processes and other kernel methods

Summary

The theory of kernels offers a rich mathematical framework for the archetypical tasks of classification and regression. Its core insight consists of the representer theorem that asserts that an unknown target function underlying a dataset can be represented by a finite sum of evaluations of a singular function, the so-called kernel function. Together with the infamous kernel trick that provides a practical way of incorporating such a kernel function into a machine learning method, a plethora of algorithms can be made more versatile. This chapter first introduces the mathematical foundations required for understanding the distinguished role of the kernel function and its consequence in terms of the representer theorem. Afterwards, we show how selected popular algorithms, including Gaussian processes, can be promoted to their kernel variant. In addition, several ideas on how to construct suitable kernel functions are provided, before demonstrating the power of kernel methods in the context of quantum (chemistry) problems.