Nonlinear Programming: 3rd Edition

This book provides a comprehensive and accessible presentation of algorithms for solving continuous optimization problems. It relies on rigorous mathematical analysis, but also aims at an intuitive exposition that makes use of visualization where possible. It places particular emphasis on modern developments, and their widespread applications in fields such as large-scale resource allocation problems, signal processing, and machine learning. The 3rd edition brings the book in closer harmony with the companion works Convex Optimization Theory (Athena Scientific, 2009), Convex Optimization Algorithms (Athena Scientific, 2015), Convex Analysis and Optimization (Athena Scientific, 2003), and Network Optimization (Athena Scientific, 1998). These works are complementary in that they deal primarily with convex, possibly nondifferentiable, optimization problems and rely on convex analysis. By contrast the nonlinear programming book focuses primarily on analytical and computational methods for possibly nonconvex differentiable problems. It relies primarily on calculus and variational analysis, yet it still contains a detailed presentation of duality theory and its uses for both convex and nonconvex problems. This on-line edition contains detailed solutions to all the theoretical book exercises. Among its special features, the book: Provides extensive coverage of iterative optimization methods within a unifying framework Covers in depth duality theory from both a variational and a geometric point of view Provides a detailed treatment of interior point methods for linear programming Includes much new material on a number of topics, such as proximal algorithms, alternating direction methods of multipliers, and conic programming Focuses on large-scale optimization topics of much current interest, such as first order methods, incremental methods, and distributed asynchronous computation, and their applications in machine learning, signal processing, neural network training, and b The third edition of the book is a thoroughly rewritten version of the 1999 2nd edition. New material was included, some of the old material was discarded, and a large portion of the remainder was reorganized or revised. This book provides a comprehensive and accessible presentation of algorithms for solving continuous optimization problems. It relies on rigorous mathematical analysis, but also aims at an intuitive exposition that makes use of visualization where possible. It places particular emphasis on modern developments, and their widespread applications in fields such as large-scale resource allocation problems, signal processing, and machine learning. The book was developed through instruction at MIT, focuses on nonlinear and other types of optimization : iterative algorithms for constrained and unconstrained optimization, Lagrange multipliers and duality, large scale problems, and the interface between continuous and discrete optimization. Among its special features, the book : 1) provides extensive coverage of iterative optimization methods within a unifying framework ; 2) provides a detailed treatment of interior point methods for linear programming ; 3) covers in depth duality theory from both a variational and a geometrical / convex analysis point of view ; 4) includes much new material on a number of topics, such as neural network training, large-scale optimization, signal processing, machine learning, and optimal control ; 5) includes a large number of examples and exercises detailed solutions of many of which are posted on the internet A rigorous and comprehensive treatment of network flow theory and monotropic optimization by one of the world's most renowned applied mathematicians. This classic textbook, first published by J. Wiley & Sons, Inc., in 1984, covers extensively the duality theory and the algorithms of linear and nonlinear network optimization optimization, and their significant extensions to monotropic programming (separable convex constrained optimization problems, including linear programs). Monotropic programming problems are characterized by a rich interplay between combinatorial structure and convexity properties. Rockafellar develops, for the first time, algorithms and a remarkably complete duality theory for these problems. This book develops in depths nonlinear programming, a central algorithmic method for optimization. The treatment focuses on constrained and unconstrained iterative algorithms, Lagrange multiplier theory, and large scale optimization methods. --back cover