7 - First-Order Methods for Constrained Optimization

Summary

Here, we describe methods for minimizing a smooth function over a closed convex set, using gradient information. We first state results that characterize optimality of points in a way that can be checked, and describe the vital operation of projection onto the feasible set. We next describe the projected gradient algorithm, which is in a sense the extension of the steepest-descent method to the constrained case, analyze its convergence, and describe several extensions. We next analyze the conditional-gradient method (also known as “Frank-Wolfe”) for the case in which the feasible set is compact and demonstrate sublinear convergence of this approach when the objective function is convex.