2: Dynamic Programming and Optimal Control, Vol. II 2

A major revision of the second volume of a textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization. The second volume is oriented towards mathematical analysis and computation, and treats infinite horizon problems extensively. New features of the 3rd edition are: 1) A major enlargement in size and scope: the length has increased by more than 50%, and most of the old material has been restructured and/or revised. 2) Extensive coverage (more than 100 pages) of recent research on simulation-based approximate dynamic programming (neuro-dynamic programming), which allow the practical application of dynamic programming to large and complex problems. 3) An in-depth development of the average cost problem (more than 100 pages), including a full analysis of multichain problems, and an extensive analysis of infinite-spaces problems. 4) An introduction to infinite state space stochastic shortest path problems. 5) Expansion of the theory and use of contraction mappings in infinite state space problems and in neuro-dynamic programming. 6) A substantive appendix on the mathematical measure-theoretic issues that must be addressed for a rigorous theory of stochastic dynamic programming. Much supplementary material can be found in the book's web page The Leading And Most Up-to-date Textbook On The Far-ranging Algorithmic Methododogy Of Dynamic Programming, Which Can Be Used For Optimal Control, Markovian Decision Problems, Planning And Sequential Decision Making Under Uncertainty, And Discrete/combinatorial Optimization. The Treatment Focuses On Basic Unifying Themes, And Conceptual Foundations. It Illustrates The Versatility, Power, And Generality Of The Method With Many Examples And Applications From Engineering, Operations Research, And Other Fields. It Also Addresses Extensively The Practical Application Of The Methodology, Possibly Through The Use Of Approximations, And Provides An Extensive Treatment Of The Far-reaching Methodology Of Neuro-dynamic Programming/reinforcement Learning. The First Volume Is Oriented Towards Modeling, Conceptualization, And Finite-horizon Problems, But Also Includes A Substantive Introduction To Infinite Horizon Problems That Is Suitable For Classroom Use. The Second Volume Is Oriented Towards Mathematical Analysis And Computation, Treats Infinite Horizon Problems Extensively, And Provides An Up-to-date Account Of Approximate Large-scale Dynamic Programming And Reinforcement Learning. The Text Contains Many Illustrations, Worked-out Examples, And Exercises.--publisher's Website. V. 2. Approximate Dynamic Programming. Dimitri P. Bertsekas. Includes Bibliographical References And Indexes. The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization. The treatment focuses on basic unifying themes, and conceptual foundations. It illustrates the versatility, power, and generality of the method with many examples and applications from engineering, operations research, and other fields. It also addresses extensively the practical application of the methodology, possibly through the use of approximations, and provides an introduction to the far-reaching methodology of Neuro-Dynamic Programming/Reinforcement Learning. The first volume is oriented towards modeling, conceptualization, and finite-horizon problems, but also includes a substantive introduction to infinite horizon problems that is suitable for classroom use. The second volume is oriented towards mathematical analysis and computation, and treats infinite horizon problems extensively. The text contains many illustrations, worked-out examples, and exercises