Chapter 5 starts out with a physics motivation, as well as a mathematical statement of the problems that will be tackled in later sections. Several methods are introduced to solve a single nonlinear equation in one variable: fixed-point iteration, the bisection method, Newton’s method, the secant method, and Ridders’ method. After providing some advice about advantages and disadvantages of each approach, the text then studies how to find zeros of polynomials, employing two different techniques. The sophistication is then increased, by tackling systems of nonlinear equations and examining the corresponding challenges; in addition to Newton’s method, the text derives the equations behind Broyden’s method. A related subject is then broached, minimization in one or several dimensions; this includes the gradient-descent method, as well as detailed analysis of critical points; the second edition includes extensive new material on derivative-free optimization (golden-section search and Powell’s method).The chapter is rounded out by a physics project, the extremization of the action in classical mechanics, and a problem set. The physics project shows Hamilton’s principle in... action, translated into a multidimensional minimization problem.

This chpater is dedicated to the solution of nonlinear systems of equations, that is finding roots of functions. We begin with a classification of roots into simple and non-simple, and a few words about their stability. Then we begin with some of the simplest methods for root finding: bisection, false position, fixed point iterations, and its variants. For all these schemes, we provide sufficient conditions for them to be well defined and convergent. A detailed analysis of Newton’s method, and its variants (collectively known as quasi-Newton methods), in one dimension is then presented. We show sufficient conditions for its local and global quadratic convergence,as well as how to proceed in the case of non simple roots. Then we present Newton’s method in several dimensions, and show its local quadratic convergence, including the celebrated Kantorovich’s theorem.