An Introduction to Stochastic Modeling, Third Edition

Serving as the foundation for a one-semester course in stochastic processes for students familiar with elementary probability theory and calculus, Introduction to Stochastic Modeling, Third Edition , bridges the gap between basic probability and an intermediate level course in stochastic processes. The objectives of the text are to introduce students to the standard concepts and methods of stochastic modeling, to illustrate the rich diversity of applications of stochastic processes in the applied sciences, and to provide exercises in the application of simple stochastic analysis to realistic problems. • Realistic applications from a variety of disciplines integrated throughout the text • Plentiful, updated and more rigorous problems, including computer "challenges" • Revised end-of-chapter exercises sets—in all, 250 exercises with answers • New chapter on Brownian motion and related processes • Additional sections on Matingales and Poisson process • Solutions manual available to adopting instructors Audience: Upper division undergraduate and graduate-level courses in stochastic processes and stochastic modeling, offered in statistics and mathematics departments at all major universities. Cover......Page 1 Title Page......Page 4 Copyright Page......Page 5 Contents......Page 6 Preface......Page 10 1. Stochastic Modeling......Page 16 2. Probability Review......Page 21 3. The Major Discrete Distributions......Page 39 4. Important Continuous Distributions......Page 48 5. Some Elementary Exercises......Page 58 6. Useful Functions, Integrals, and Sums......Page 68 1. The Discrete Case......Page 72 2. The Dice Game Craps......Page 79 3. Random Sums......Page 85 4. Conditioning on a Continuous Random Variable......Page 94 5. Martingales*......Page 102 1. Definitions......Page 110 2. Transition Probability Matrices of a Markov Chain......Page 115 3. Some Markov Chain Models......Page 120 4. First Step Analysis......Page 131 5. Some Special Markov Chains......Page 150 6. Functionals of Random Walks and Success Runs......Page 166 7. Another Look at First Step Analysis*......Page 184 8. Branching Processes*......Page 192 9. Branching Processes and Generating Functions*......Page 199 1. Regular Transition Probability Matrices......Page 214 2. Examples......Page 230 3. The Classification of States......Page 249 4. The Basic Limit Theorem of Markov Chains......Page 260 5. Reducible Markov Chains*......Page 273 1. The Poisson Distribution and the Poisson Process......Page 282 2. The Law of Rare Events......Page 294 3. Distributions Associated with the Poisson Process......Page 305 4. The Uniform Distribution and Poisson Processes......Page 312 5. Spatial Poisson Processes......Page 326 6. Compound and Marked Poisson Processes......Page 333 1. Pure Birth Processes......Page 348 2. Pure Death Processes......Page 360 3. Birth and Death Processes......Page 370 4. The Limiting Behavior of Birth and Death Processes......Page 381 5. Birth and Death Processes with Absorbing States......Page 394 6. Finite State Continuous Time Markov Chains......Page 409 7. A Poisson Process with a Markov Intensity*......Page 423 1. Definition of a Renewal Process and Related Concepts......Page 434 2. Some Examples of Renewal Processes......Page 441 3. The Poisson Process Viewed as a Renewal Process......Page 447 4. The Asymptotic Behavior of Renewal Processes......Page 452 5. Generalizations and Variations on Renewal Processes......Page 462 6. Discrete Renewal Theory*......Page 472 1. Brownian Motion and Gaussian Processes......Page 488 2. The Maximum Variable and the Reflection Principle......Page 506 3. Variations and Extensions......Page 513 4. Brownian Motion with Drift......Page 523 5. The Ornstein-Uhlenbeck Process*......Page 539 1. Queueing Processes......Page 556 2. Poisson Arrivals, Exponential Service Times......Page 562 3. General Service Time Distributions......Page 573 4. Variations and Extensions......Page 582 5. Open Acyclic Queueing Networks......Page 596 6. General Open Networks......Page 607 Further Reading......Page 616 Answers to Exercises......Page 618 Index......Page 640 An Introduction to Stochastic Modeling, Third Edition serves as the foundation for a one-semester course in stochastic processes for students familiar with elementary probability theory and calculus. It bridges the gap between basic probability know-how and an intermediate level course in stochastic processes. The objectives of the book are to introduce students to the standard concepts and methods of stochastic modeling, to illustrate the rich diversity of applications of stochastic processes in the applied sciences, and to provide exercises in the application of simple stochastic analysis to realistic problems.