Markov Chains with Stationary Transition Probabilities (Grundlehren der mathematischen Wissenschaften, 104)

The theory of Markov chains, although a special case of Markov processes, is here developed for its own sake and presented on its own merits. In general, the hypothesis of a denumerable state space, which is the defining hypothesis of what we call a "chain" here, generates more clear-cut questions and demands more precise and definitive an­ swers. For example, the principal limit theorem (§§ 1. 6, II. 10), still the object of research for general Markov processes, is here in its neat final form; and the strong Markov property (§ 11. 9) is here always applicable. While probability theory has advanced far enough that a degree of sophistication is needed even in the limited context of this book, it is still possible here to keep the proportion of definitions to theorems relatively low. . From the standpoint of the general theory of stochastic processes, a continuous parameter Markov chain appears to be the first essentially discontinuous process that has been studied in some detail. It is common that the sample functions of such a chain have discontinuities worse than jumps, and these baser discontinuities play a central role in the theory, of which the mystery remains to be completely unraveled. In this connection the basic concepts of separability and measurability, which are usually applied only at an early stage of the discussion to establish a certain smoothness of the sample functions, are here applied constantly as indispensable tools. Front Matter....Pages II-X Fundamental defintions....Pages 1-5 Transition probabilities....Pages 5-11 Classification of states....Pages 11-15 Recurrence....Pages 15-20 Criteria and examples....Pages 20-26 The main limit theorem....Pages 26-33 Various complements....Pages 33-39 Repetitive pattern and renewal process....Pages 39-43 Taboo probabilities....Pages 43-52 The generating function....Pages 52-57 The moments of first entrance time distributions....Pages 57-65 A random walk example....Pages 65-71 System theorems....Pages 71-75 Functionals and associated random variables....Pages 75-85 Ergodic theorems....Pages 85-93 Further limit theorems....Pages 93-106 Almost closed and sojourn sets....Pages 106-113 Transition matrix: basic properties....Pages 114-123 Standard transition matrix....Pages 123-130 Differentiability....Pages 130-135 Definitions and measure-theoretic foundations....Pages 135-143 The sets of constancy....Pages 143-152 Continuity properties of sample functions....Pages 152-156 Further specifications of the process....Pages 156-160 Optional random variable....Pages 160-168 Strong Markov property....Pages 168-177 Classification of states....Pages 177-182 Taboo probability functions....Pages 182-191 Ratio limit theorems....Pages 191-196 Discrete approximations....Pages 196-203 Functionals....Pages 203-209 Post-exit process....Pages 209-218 Imbedded renewal process....Pages 218-224 The two systems of differential equations....Pages 224-229 The minimal solution....Pages 229-235 The first infinity....Pages 235-244 Examples....Pages 244-261 Back Matter....Pages 261-278 "This book presupposes no knowledge of Markov chains but it does assume the elements of general probability theory as given in a modern introductory course."--Preface