Dynamic Programming

An introduction to the mathematical theory of multistage decision processes, this text takes a "functional equation" approach to the discovery of optimum policies. Written by a leading developer of such policies, it presents a series of methods, uniqueness and existence theorems, and examples for solving the relevant equations. The text examines existence and uniqueness theorems, the optimal inventory equation, bottleneck problems in multistage production processes, a new formalism in the calculus of variation, strategies behind multistage games, and Markovian decision processes. Each chapter concludes with a problem set that Eric V. Denardo of Yale University, in his informative new introduction, calls "a rich lode of applications and research topics." 1957 edition. 37 figures. Preface......Page bellman0005_0001.djvu Contents......Page bellman0017_0001.djvu 1. Introduction......Page bellman0026_0001.djvu 2. A multi-stage allocation process......Page bellman0027_0001.djvu 3. Discussion......Page bellman0028_0001.djvu 4. Functional equation approach......Page bellman0030_0001.djvu 5. Discussion......Page bellman0032_0001.djvu 7. A "smoothing" problem......Page bellman0033_0001.djvu 8. Infinite stage approximation......Page bellman0034_0001.djvu 9. Existence and uniqueness theorems......Page bellman0035_0001.djvu 11. Approximation in policy space......Page bellman0039_0001.djvu 12. Properties of the solution - I: Convexity......Page bellman0042_0001.djvu 13. Properties of the solution - II: Concavity......Page bellman0043_0001.djvu 14. Properties of the solution - III: Concavity......Page bellman0045_0001.djvu 15. An "ornery" example......Page bellman0048_0001.djvu 16. A particular example - I......Page bellman0049_0001.djvu 17. A particular example - II......Page bellman0051_0001.djvu 18. Approximation and stability......Page bellman0052_0001.djvu 19. Time-dependent processes......Page bellman0053_0001.djvu 20. Multi-activity processes......Page bellman0054_0001.djvu 21. Multi-dimensional structure theorems......Page bellman0056_0001.djvu 22. Locating the unique maximum of a concave function......Page bellman0057_0001.djvu 23. Continuity and memory......Page bellman0060_0001.djvu 24. Stohastic allocation processes......Page bellman0061_0001.djvu 25. Functional equations......Page bellman0062_0001.djvu Exercises and Research Problems for Chapter I......Page bellman0063_0001.djvu Bibliography and Comments for Chapter I......Page bellman0082_0001.djvu 2. Stohastic gold-mining......Page bellman0084_0001.djvu 3. Enumerative treatment......Page bellman0085_0001.djvu 5. Infinite stage approximation......Page bellman0086_0001.djvu 6. Existence and uniqueness......Page bellman0087_0001.djvu 7. Approximation in policy space and monotone convergence......Page bellman0088_0001.djvu 8. The solution......Page bellman0089_0001.djvu 10. Some generalizations......Page bellman0092_0001.djvu 11. The form of f(x,y)......Page bellman0094_0001.djvu 12. The problem for a finite number of stages......Page bellman0095_0001.djvu 13. A three-choice problem......Page bellman0097_0001.djvu 14. A stability theorem......Page bellman0099_0001.djvu Exercises and Research Problems for Chapter II......Page bellman0100_0001.djvu Bibliography and Comments for Chapter II......Page bellman0102_0001.djvu 2. Discussion of the two preceding processes......Page bellman0104_0001.djvu 4. Mathematical formulation - I. A discrete deterministic process......Page bellman0106_0001.djvu 5. Mathematical formulation - II. A discrete stohastic process ......Page bellman0108_0001.djvu 6. Mathematical formulation - III. A continuous deterministic process......Page bellman0109_0001.djvu 9. Casuality and optimality......Page bellman0110_0001.djvu 10. Approximation in policy space......Page bellman0111_0001.djvu Exercises and Research Problems for Chapter III......Page bellman0113_0001.djvu Bibliography and Comments for Chapter III......Page bellman0138_0001.djvu 1. Introduction......Page bellman0139_0001.djvu 2. A Fundamental inequality......Page bellman0140_0001.djvu 3. Equations of Type One......Page bellman0142_0001.djvu 4. Equations of Type Two......Page bellman0144_0001.djvu 5. Monotone convergence......Page bellman0145_0001.djvu 6. Stability theorems......Page bellman0146_0001.djvu 7. Some directions of generalization......Page bellman0147_0001.djvu 8. An equation of the third type......Page bellman0148_0001.djvu 9. An "optimal inventory" equation......Page bellman0152_0001.djvu Exercises and Research Problems for Chapter IV......Page bellman0155_0001.djvu Bibliography and Comments for Chapter IV......Page bellman0174_0001.djvu 1. Introduction......Page bellman0175_0001.djvu 2. Formulation of the general problem......Page bellman0176_0001.djvu A. Finite total time period......Page bellman0177_0001.djvu D. Unbounded time period - one period lag in supply......Page bellman0179_0001.djvu 3. A simple observation......Page bellman0180_0001.djvu 4. Constant stock level - preliminary discussion......Page bellman0181_0001.djvu 5. Proportional cost - one-dimensional case......Page bellman0182_0001.djvu 6. Proportional cost - multi-dimensional case......Page bellman0187_0001.djvu 7. Finite time period......Page bellman0189_0001.djvu 9. Non-proportional penalty cost - red tape......Page bellman0192_0001.djvu 11. The form of the general solution......Page bellman0194_0001.djvu 12. Fixed costs......Page bellman0195_0001.djvu 14. Unbounded process - one period time lag......Page bellman0196_0001.djvu 15. Convex cost function - unbounded process......Page bellman0199_0001.djvu Appendix Chapter V - The Renewal Equation......Page bellman0200_0001.djvu Exercises and Research Problems for Chapter V......Page bellman0201_0001.djvu Bibliography and Comments for Chapter V......Page bellman0205_0001.djvu 1. Introduction......Page bellman0206_0001.djvu 2. A General class of multi-stage production problems......Page bellman0207_0001.djvu 3. Discussion of the preceding model......Page bellman0210_0001.djvu 4. Functional equations......Page bellman0211_0001.djvu 5. A Continuous version......Page bellman0212_0001.djvu 6. Notation......Page bellman0214_0001.djvu 8. The basic functional equation......Page bellman0215_0001.djvu 10. Application of the partial differential equation......Page bellman0216_0001.djvu 11. A Particular example......Page bellman0217_0001.djvu 12. A Dual problem......Page bellman0220_0001.djvu 13. Verification of the solution given in 11......Page bellman0223_0001.djvu 14. Computational solution......Page bellman0225_0001.djvu 15. Nonlinear problems ......Page bellman0226_0001.djvu Exercises and Research Problems for Chapter VI......Page bellman0227_0001.djvu Bibliography and Comments for Chapter VI......Page bellman0228_0001.djvu 1. Introduction......Page bellman0230_0001.djvu 2. Preliminaries......Page bellman0232_0001.djvu 4. The solution......Page bellman0234_0001.djvu 5. The modified w solution......Page bellman0237_0001.djvu 6. The equilibrium solution......Page bellman0238_0001.djvu 7. A short-time w solution......Page bellman0240_0001.djvu 8. Description of solution and proof......Page bellman0241_0001.djvu Bibliography and Comments for Chapter VII......Page bellman0244_0001.djvu 1. Introduction......Page bellman0245_0001.djvu 2. Continuous versions - I: A differential approach......Page bellman0246_0001.djvu 4. Preliminary discussion......Page bellman0247_0001.djvu 5. Mixing at a point......Page bellman0249_0001.djvu 7. Derivation of the differential equations......Page bellman0250_0001.djvu 8. The variational procedure......Page bellman0251_0001.djvu 9. The behavior of Ki.......Page bellman0252_0001.djvu 10. The solution for T=infinity......Page bellman0253_0001.djvu 11. Solution for finite total time......Page bellman0254_0001.djvu 12. The three-choice problem......Page bellman0256_0001.djvu 13. Some lemmas and preliminary results......Page bellman0257_0001.djvu 14. Mixed policies......Page bellman0258_0001.djvu 15. The solution for infinite time, D>0......Page bellman0259_0001.djvu 16. D