The objective of this chapter is to extend the ad hoc least squares method of somewhat arbitrarily selected base functions to a more generic method applicable to a broad range of functions – the Fourier series, which is an expansion of a relatively arbitrary function (with certain smoothness requirement and finite jumps at worst) with a series of sinusoidal functions. An important mathematical reason for using Fourier series is its “completeness” and almost guaranteed convergence. Here “completeness” means that the error goes to zero when the whole Fourier series with infinite base function is used. In other words, the Fourier series formed by the selected sinusoidal functions is sufficient to linearly combine into a function that converges to an arbitrary continuous function. This chapter on Fourier series will lay out a foundation that will lead to Fourier Transform and spectrum analysis. In this sense, this chapter is important as it provides background information and theoretical preparation.

This chapter discusses a generic least squares method and a special situation when the base functions are orthogonal to each other, which makes the solution explicit; in addition, we learn that the essence of the least squares method can be viewed as a way to project the target function in a higher dimension onto a lower dimension formed by the base functions. The least squares method ensures that the error vector is “perpendicular” to the projected (or approximate) vector in the base function dimension (a lower dimension) and thus has the shortest “length” or minimized error. Although this chapter does not have much computation involved, it is very important for a good understanding of the meaning of many techniques and methods in the subsequent chapters.