A First Course in Stochastic Processes, Second Edition

The purpose, level, and style of this new edition conform to the tenets set forth in the original preface. The authors continue with their tack of developing simultaneously theory and applications, intertwined so that they refurbish and elucidate each other. The authors have made three main kinds of changes. First, they have enlarged on the topics treated in the first edition. Second, they have added many exercises and problems at the end of each chapter. Third, and most important, they have supplied, in new chapters, broad introductory discussions of several classes of stochastic processes not dealt with in the first edition, notably martingales, renewal and fluctuation phenomena associated with random sums, stationary stochastic processes, and diffusion theory. Front Cover......Page 1 A First Course In Stochastic Processes......Page 4 Copyright Page......Page 5 Table of Contents ......Page 6 Preface......Page 12 Preface to First Edition......Page 16 1. Review of Basic Terminology and Properties of Random Variables and Distribution Functions......Page 20 2. Two Simple Examples of Stochastic Processes......Page 39 3. Classification of General Stochastic Processes......Page 45 4. Defining a Stochastic Process......Page 51 Elementary Problems......Page 52 Problems......Page 55 References......Page 63 1. Definitions......Page 64 2. Examples of Markov Chains......Page 66 3. Transition Probability Matrices of a Markov Chain......Page 77 4. Classification of States of a Markov Chain......Page 78 5. Recurrence......Page 81 6. Examples of Recurrent Markov Chains......Page 86 7. More on Recurrence......Page 91 Elementary Problems......Page 92 Problems......Page 96 Notes......Page 98 References......Page 99 1. Discrete Renewal Equation......Page 100 2. Proof of Theorem 1.1......Page 106 3. Absorption Probabilities......Page 108 4. Criteria for Recurrence......Page 113 5. A Queueing Example......Page 115 6. Another Queueing Model......Page 121 7. Random Walk......Page 125 Elementary Problems......Page 127 Problems......Page 131 Reference......Page 135 1. General Pure Birth Processes and Poisson Processes......Page 136 2. More about Poisson Processes......Page 142 3. A Counter Model......Page 147 4. Birth and Death Processes......Page 150 5. Differential Equations of Birth and Death Processes......Page 154 6. Examples of Birth and Death Processes......Page 156 7. Birth and Death Processes with Absorbing States......Page 164 8. Finite State Continuous Time Markov Chains......Page 169 Elementary Problems......Page 171 Problems......Page 177 Notes......Page 184 References......Page 185 1. Definition of a Renewal Process and Related Concepts......Page 186 2. Some Examples of Renewal Processes......Page 189 3. More on Some Special Renewal Processes......Page 192 4. Renewal Equations and the Elementary Renewal Theorem......Page 200 5. The Renewal Theorem......Page 208 6. Applications of the Renewal Theorem......Page 211 7. Generalizations and Variations on Renewal Processes......Page 216 8. More Elaborate Applications of Renewal Theory......Page 231 9. Superposition of Renewal Processes......Page 240 Elementary Problems......Page 247 Problems......Page 249 Reference......Page 256 1. Preliminary Definitions and Examples......Page 257 2. Supermartingales and Submartingales......Page 267 3. The Optional Sampling Theorem......Page 272 4. Some Applications of the Optional Sampling Theorem......Page 282 5. Martingale Convergence Theorems......Page 297 6. Applications and Extensions of the Martingale Convergence Theorems......Page 306 7. Martingales with Respect to σ-Fields......Page 316 8. Other Martingales......Page 332 Elementary Problems......Page 344 Problems......Page 349 References......Page 358 1. Background Material......Page 359 2. Joint Probabilities for Brownian Motion......Page 362 3. Continuity of Paths and the Maximum Variables......Page 364 4. Variations and Extensions......Page 370 5. Computing Some Functionals of Brownian Motion by Martingale Methods......Page 376 6. Multidimensional Brownian Motion......Page 384 7. Brownian Paths......Page 390 Elementary Problems......Page 402 Problems......Page 405 References......Page 410 1. Discrete Time Branching Processes......Page 411 2. Generating Function Relations for Branching Processes......Page 413 3. Extinction Probabilities......Page 415 4. Examples......Page 419 5. Two-Type Branching Processes......Page 423 6. Multi-Type Branching Processes......Page 430 7. Continuous Time Branching Processes......Page 431 8. Extinction Probabilities for Continuous Time Branching Processes......Page 435 9. Limit Theorems for Continuous Time Branching Processes......Page 438 10. Two-Type Continuous Time Branching Process......Page 443 11. Branching Processes with General Variable Lifetime......Page 450 Elementary Problems......Page 455 Problems......Page 457 Reference......Page 461 1. Definitions and Examples......Page 462 2. Mean Square Distance......Page 470 3. Mean Square Error Prediction......Page 480 4. Prediction of Covariance Stationary Processes......Page 489 5. Ergodic Theory and Stationary Processes......Page 493 6. Applications of Ergodic Theory......Page 508 7. Spectral Analysis of Covariance Stationary Processes......Page 521 8. Gaussian Systems......Page 529 9. Stationary Point Processes......Page 535 10. The Level-Crossing Problem......Page 538 Elementary Problems......Page 543 Problems......Page 546 Notes......Page 553 References......Page 554 1. The Spectral Theorem......Page 555 2. The Frobenius Theory of Positive Matrices......Page 561 Index......Page 572