Continuous Nonlinear Optimization for Engineering Applications in GAMS Technology (Springer Optimization and Its Applications Book 121)

This book presents the theoretical details and computational performances of algorithms used for solving continuous nonlinear optimization applications imbedded in GAMS. Aimed toward scientists and graduate students who utilize optimization methods to model and solve problems in mathematical programming, operations research, business, engineering, and industry, this book enables readers with a background in nonlinear optimization and linear algebra to use GAMS technology to understand and utilize its important capabilities to optimize algorithms for modeling and solving complex, large-scale, continuous nonlinear optimization problems or applications. Beginning with an overview of constrained nonlinear optimization methods, this book moves on to illustrate key aspects of mathematical modeling through modeling technologies based on algebraically oriented modeling languages. Next, the main feature of GAMS, an algebraically oriented language that allows for high-level algebraic representation of mathematical optimization models, is introduced to model and solve continuous nonlinear optimization applications. More than 15 real nonlinear optimization applications in algebraic and GAMS representation are presented which are used to illustrate the performances of the algorithms described in this book. Theoretical and computational results, methods, and techniques effective for solving nonlinear optimization problems, are detailed through the algorithms MINOS, KNITRO, CONOPT, SNOPT and IPOPT which work in GAMS technology. Preface 5 Contents 10 List of Figures 14 List of Tables 17 List of Algorithms 21 Chapter 1: Introduction 23 1.1 Nonlinear Optimization Modeling 24 1.2 Constrained Nonlinear Optimization Methods: A Critical Review 25 1.2.1 Convergence Tests 26 1.2.2 Infeasible Points 27 1.2.3 Approximate Sub-problem: Local Models and Their Solving 27 1.2.4 Globalization Strategy: Convergence from Remote Starting Points 33 1.2.5 Refining the Local Model 36 1.3 Structure of the Book 38 Chapter 2: Mathematical Modeling Using Algebraic Oriented Languages for Nonlinear Optimization 40 2.1 Linguistic Models Versus Mathematical Models 40 2.2 Mathematical Modeling and Computational Sciences 42 2.3 Modeling Scheme 43 2.4 Algebraic Oriented Modeling Languages 45 Chapter 3: Introduction to GAMS Technology 49 3.1 Basics of Modeling 49 3.2 Structure of a GAMS Model 51 3.3 Sets 51 3.4 Data 52 3.5 Variables 53 3.6 Equations 54 3.7 Model Declarations 56 3.8 The SOLVE Statement and Model Types 57 3.9 DISPLAY and PUT Statements 60 3.10 GAMS Output 61 3.11 Basic Solver Usage 63 3.12 Running a Job 64 3.13 Program Development 64 Chapter 4: Applications of Continuous Nonlinear Optimization 66 4.1 Chemical Equilibrium (ELCH) 67 4.2 Optimization of an Alkilation Process (ALKI) 69 4.3 Optimal Design of a Reactor as a Geometric Programming Problem (PREC) 71 4.4 Cost Minimization of a Transformer Design (TRAFO) 73 4.5 Optimization of a Multi-spindle Automatic Lathe (LATHE) 75 4.6 Static Power Scheduling (PPSE) 77 4.7 Optimization of a Separation Process in a Membrane with three Stages (MSP3) 79 4.8 Optimization of a Separation Process in a Membrane with five Stages (MSP5) 82 4.9 Blending/Pooling with Five Feeds and Two Products (POOL) (Andrei, 1999, pp. 808; Andrei, 2003, pp. 382) 86 4.10 Distribution of Electrons on a Sphere (DES) 90 4.11 Hanging Chain (HANG) 92 4.12 Determine the Optimal Mixing Policy of Two Catalysts Along the Length of a Tubular Plug Flow Reactor Involving Several Re... 94 4.13 Optimal Control of a Continuous Stirred-Tank Chemical Reactor (CSTC) 96 4.14 Optimal Temperature Field in a Rectangular Area (DIFF) 100 4.15 Stationary Flow of an Incompressible Fluid in a Rectangular Area (FLOW/FLOWO) 110 4.16 Fed-Batch Fermenter for Penicillin Production (PENICI) 122 4.17 A Standard Linear Lumped Parameter System (CONT) 128 4.18 Van der Pol Oscillator (POL) 133 Chapter 5: Optimality Conditions for Continuous Nonlinear Optimization 137 5.1 General Concepts in Nonlinear Optimization 138 5.2 Optimality Conditions for Unconstrained Optimization 141 5.3 Optimality Conditions for Problems with Inequality Constraints 144 5.4 Optimality Conditions for Problems with Equality Constraints 148 5.5 Optimality Conditions for General Problems 157 Chapter 6: Simple Bound Constraints Optimization 164 6.1 Necessary Conditions for Optimality 165 6.2 Sufficient Conditions for Optimality 167 6.3 Methods for Solving Simple Bound Optimization Problems 168 6.4 Spectral Projected Gradient Method (SPG) 171 6.5 L-BFGS with Simple Bounds (L-BFGS-B) 177 6.6 Truncated Newton with Simple Bounds (TNBC) 186 6.7 Applications 188 Chapter 7: Penalty and Augmented Lagrangian Methods 202 7.1 The Quadratic Penalty Method 203 7.2 Nonsmooth Penalty Method 207 7.3 Augmented Lagrangian Method 210 7.4 Criticism of the Penalty and Augmented Lagrangian Methods 215 Chapter 8: A Penalty-Barrier Algorithm: SPENBAR 219 8.1 The Penalty-Barrier Method 222 8.2 Global Convergence 227 Chapter 9: Linearly Constrained Augmented Lagrangian: MINOS 238 9.1 MINOS for Linear Constraints 239 9.2 MINOS for Nonlinear Constraints 246 Chapter 10: Quadratic Programming 257 10.1 Equality-Constrained Quadratic Programming 257 10.2 Inequality-Constrained Quadratic Programming 265 Chapter 11: Sequential Quadratic Programming (SQP) 283 11.1 Reduced Hessian Quasi-Newton Approximations 288 11.2 Merit Functions 289 11.3 Second-Order Correction (Maratos Effect) 292 11.4 Line-Search SQP Algorithm 294 11.5 Trust-Region SQP Method 296 11.6 Sequential Linear-Quadratic Programming (SLQP) 299 Chapter 12: A SQP Method Using Only Equality-Constrained Sub-problems: DONLP 303 Chapter 13: A SQP Algorithm with Successive Error Restoration: NLPQLP 311 Chapter 14: Active-set Sequential Linear-Quadratic Programming: KNITRO/ACTIVE 318 14.1 KNITRO/ACTIVE Algorithm 319 14.2 Strategy for Penalty Parameter Update 322 14.3 Iteration of Projected Conjugate Gradient Algorithm 323 14.4 Hessian Options 325 Chapter 15: A SQP Algorithm for Large-Scale Constrained Optimization: SNOPT 329 15.1 Infeasible Constraints 330 15.2 The SQP Iteration for General Inequality Nonlinear Optimization 331 15.3 The Quadratic Programming Solver SQOPT 337 Chapter 16: Generalized Reduced Gradient with Sequential Linearization: CONOPT 343 Chapter 17: Interior Point Methods 355 17.1 Prototype of Interior Point Algorithm 358 17.2 Aspects of Algorithmic Development 361 17.3 Line-Search Interior Point Algorithm 367 17.4 A Variant of Line-Search Interior Point Algorithm 368 17.5 Trust-Region Interior Point Algorithm 386 Chapter 18: Filter Methods 393 18.1 Sequential Linear Programming Filter Algorithm 395 18.2 Sequential Quadratic Programming Filter Algorithm 399 Chapter 19: Interior Point Sequential Linear-Quadratic Programming: KNITRO/INTERIOR 409 19.1 KNITRO/INTERIOR-DIRECT Algorithm 411 19.2 KNITRO/INTERIOR-CG Algorithm 414 Chapter 20: Interior Point Filter Line Search: IPOPT 426 20.1 Basic Algorithm IPOPT 427 The Primal-Dual Barrier Approach 427 Solving the Barrier Problem 429 Line-Search Filter Method 431 Second-Order Corrections 433 The Algorithm 434 20.2 Implementation Details 437 General Lower and Upper Bounds 437 Initialization 437 Handling Unbounded Solution Sets 438 Inertia Correction 438 Automatic Scaling of the Problem 440 Feasibility Restoration Phase 441 Practical Hints in GAMS 444 Numerical Study 445 Chapter 21: Numerical Studies: Comparisons 447 Appendix A: Mathematical Review 458 A1. Elements of Linear Algebra 458 A2. Elements of Analysis 461 A3. Elements of Topology in the Euclidian Space n 465 A4. Elements of Convexity: Convex Sets and Convex Functions 466 Appendix B: Solving Linear Equation Systems 469 B1. Systems with Diagonal Matrices 469 B2. Systems with Upper Triangular Matrices (Back Substitution) 469 B3. Systems with Lower Triangular Matrices (Forward Substitution) 470 B4. Systems with Orthogonal Matrices 470 B5. Systems with Permutation Matrices 471 B6. Gaussian Elimination (LU Factorization) 471 Algorithm GE (LU Factorization with Pivoting) 476 Algorithm LU (LU Factorization with Pivoting, Overwriting L and U on A) 477 B7. Cholesky Factorization 477 Cholesky Factorization 477 B8. The Factor-Solve Method 478 B9. Solving Underdetermined Linear Systems 479 B10. The QR Factorization 480 B11. LU Factorization of Rectangular Matrices 481 References 483 Author Index 503 Subject Index 509 Front Matter ....Pages i-xxiv Introduction (Neculai Andrei)....Pages 1-17 Mathematical Modeling Using Algebraic Oriented Languages for Nonlinear Optimization (Neculai Andrei)....Pages 19-27 Introduction to GAMS Technology (Neculai Andrei)....Pages 29-45 Applications of Continuous Nonlinear Optimization (Neculai Andrei)....Pages 47-117 Optimality Conditions for Continuous Nonlinear Optimization (Neculai Andrei)....Pages 119-145 Simple Bound Constraints Optimization (Neculai Andrei)....Pages 147-184 Penalty and Augmented Lagrangian Methods (Neculai Andrei)....Pages 185-201 A Penalty-Barrier Algorithm: SPENBAR (Neculai Andrei)....Pages 203-221 Linearly Constrained Augmented Lagrangian: MINOS (Neculai Andrei)....Pages 223-241 Quadratic Programming (Neculai Andrei)....Pages 243-268 Sequential Quadratic Programming (SQP) (Neculai Andrei)....Pages 269-288 A SQP Method Using Only Equality-Constrained Sub-problems: DONLP (Neculai Andrei)....Pages 289-296 A SQP Algorithm with Successive Error Restoration: NLPQLP (Neculai Andrei)....Pages 297-303 Active-set Sequential Linear-Quadratic Programming: KNITRO/ACTIVE (Neculai Andrei)....Pages 305-315 A SQP Algorithm for Large-Scale Constrained Optimization: SNOPT (Neculai Andrei)....Pages 317-330 Generalized Reduced Gradient with Sequential Linearization: CONOPT (Neculai Andrei)....Pages 331-342 Interior Point Methods (Neculai Andrei)....Pages 343-380 Filter Methods (Neculai Andrei)....Pages 381-396 Interior Point Sequential Linear-Quadratic Programming: KNITRO/INTERIOR (Neculai Andrei)....Pages 397-413 Interior Point Filter Line Search: IPOPT (Neculai Andrei)....Pages 415-435 Numerical Studies: Comparisons (Neculai Andrei)....Pages 437-447 Back Matter ....Pages 449-506