Dynamic programming and Markov processes

Front cover Dynamic Programming and Markov Processes Copyright Preface Contents Introduction 1 Markov Processes The Toymaker Example-State Probabilities The z- Transformation z- Transform Analysis of Markov Processes Transient, Multichain; and Periodic Behavior 2 Markov Processes with Rewards Solution by Recurrence Relation The Toymaker Example z-Transform Analysis of the Markov Process with Rewards Asymptotic Behavior 3 The Solution of the Sequential Decision Process by Value Iteration Introduction of Alternatives The Toymaker's Problem Solved by Value Iteration Evaluation of the Value-Iteration Approach 4 The Policy-Iteration Method for the Solution of Sequential Decision Processes The Value-Determination Operation The Policy-Improvement Routine The Iteration Cycle The Toymaker's Problem A Proof of the Properties of the Policy-Iteration Method 5 Use of the Policy-Iteration Method in Problems of Taxicab Operation, Baseball, and Automobile Replacement An Example-Taxicab Operation A Baseball Problem The Policy-Improvement Routine 6 The Policy-Iteration Method for Multiple-Chain Processes The Value-Determination Operation The Policy-Improvement Routine A Multichain Example Properties of the Iteration Cycle 7 The Sequential Decision Process with Discounting The Sequential Decision Process with Discounting Solved by Value Iteration The Value-Determination Operation The Policy-Improvement Routine An Example Proof of the Properties of the Iteration Cycle The Sensitivity of the Optimal Policy to the Discount Factor The Automobile Problem with Discounting Summary 8 The Continuous- Time Decision Process The Continuous-Time Markov Process The Solution of Continuous-Time Markov Processes by Laplace Transformation The Continuous-Time Markov Process with Rewards The Continuous-Time Decision Problem The Value-Determination Operation The Policy-Improvement Routine Completely Ergodic Processes The Foreman's Dilemma Computational Considerations The Continuous-Time Decision Process with Discounting Policy Improvement An Example Comparison with Discrete-Time Case 9 Conclusion Appendix: The Relationship of Transient to Recurrent Behavior References General References Index Back cover