Numerical optimization : theoretical and practical aspects : with 26 figures

Numerical Optimization has numerous applications in engineering sciences, operations research, economics, finance, etc. Starting with illustrations of this ubiquitous character, this book is essentially devoted to numerical algorithms for optimization, which are exposed in a tutorial way. It covers fundamental algorithms as well as more specialized and advanced topics for unconstrained and constrained problems. The theoretical bases of the subject, such as optimality conditions, Lagrange multipliers or duality, although recalled, are assumed known. Most of the algorithms described in the book are explained in a detailed manner, allowing straightforward implementation. This level of detail is intended to familiarize the reader with some of the crucial questions of numerical optimization: how algorithms operate, why they converge, difficulties that may be encountered and their possible remedies. Theoretical aspects of the approaches chosen are also addressed with care, often using minimal assumptions. Starting With Illustrative Real-world Examples, This Book Exposes In A Tutorial Way Algorithms For Numerical Optimization: Fundamental Ones (newtonian Methods, Line-searches, Trust-region, Sequential Quadratic Programming, Etc.), As Well As More Specialized And Advanced Ones (nonsmooth Optimization, Decomposition Techniques, And Interior-point Methods). Most Of These Algorithms Are Explained In A Detailed Manner, Allowing Straightforward Implementation. Theoretical Aspects Are Addressed With Care, Often Using Minimal Assumptions. The Present Version Contains Substantial Changes With Respect To The First Edition. Part I On Unconstrained Optimization Has Been Completed With A Section On Quadratic Programming. Part Ii On Nonsmooth Optimization Has Been Thoroughly Reorganized And Expanded. In Addition, Nontrivial Application Problems Have Been Inserted, In The Form Of Computational Exercises. These Should Help The Reader To Get A Better Understanding Of Optimization Methods Beyond Their Abstract Description, By Addressing Important Features To Be Taken Into Account When Passing To Implementation Of Any Numerical Algorithm. This Level Of Detail Is Intended To Familiarize The Reader With Some Of The Crucial Questions Of Numerical Optimization: How Algorithms Operate, Why They Converge, Difficulties That May Be Encountered And Their Possible Remedies. General Introduction -- Part I: Unconstraint Problems: Basic Methods; Line-searches; Newtonian Methods; Conjugate Gradient; Special Methods -- Part Ii: Nonsmooth Optimization: Some Theory Of Nonsmooth Optimization; Some Methods In Nonsmooth Optimization; Bundle Methods. The Quest Of Decent; Decomposition And Duality -- Part Iii: Newton's Methods In Constrained Optimization: Background; Local Methods For Problems With Equality Constraints; Local Methods For Problems With Equality And Inequality Constraints; Exact Penalization; Globalization By Line-search; Quasi-newton Versions -- Part Iv: Interior-point Algorithms For Linear And Quadratic Optimization: Linearly Constrained Optimization And Simplex Algorithm; Linear Monotone Complementary And Associated Vector Fields; Predictor-corrector Algorithms; Non-feasible Algorithms; Self-duality; One-step Methods; Complexity Of Linear Optimization Problems With Integer Data; Karmarkar's Algorithm -- References -- Index. J. Frédéric Bonnans ... [et Al.]. Includes Bibliographical References (p. [397]-413) And Index. Just as in its 1st edition, this book starts with illustrations of the ubiquitous character of optimization, and describes numerical algorithms in a tutorial way. It covers fundamental algorithms as well as more specialized and advanced topics for unconstrained and constrained problems. Most of the algorithms are explained in a detailed manner, allowing straightforward implementation. Theoretical aspects of the approaches chosen are also addressed with care, often using minimal assumptions. This new edition contains computational exercises in the form of case studies which help understanding optimization methods beyond their theoretical, description, when coming to actual implementation. Besides, the nonsmooth optimization part has been substantially reorganized and expanded. Features illustrations of the ubiquitous character of optimization and describes numerical algorithms in a tutorial way. This book covers fundamental algorithms as well as more specialized and advanced topics for unconstrained and constrained problems. It explains most of the algorithms to help in allowing straightforward implementation. We use the following notation: the working space is Rn, where the scalar product will be denoted indifferently by (x,y) or or xTy (actually, it will be the usual dot-product: (x,y) = ; | . | or || . || will denote the associated norm.