Mathematics of Probability (Graduate Studies in Mathematics) (Graduate Studies in Mathematics, 149)

This book covers the basics of modern probability theory. It begins with probability theory on finite and countable sample spaces and then passes from there to a concise course on measure theory, which is followed by some initial applications to probability theory, including independence and conditional expectations. The second half of the book deals with Gaussian random variables, with Markov chains, with a few continuous parameter processes, including Brownian motion, and, finally, with martingales, both discrete and continuous parameter ones. The book is a self-contained introduction to probability theory and the measure theory required to study it. Readership: Graduate students and researchers interested in probability. Preface ix Chapter 1. Some Background and Preliminaries 1 §1.1. The Language of Probability Theory 2 1.1.1. Sample Spaces and Events 3 1.1.2. Probability Measures 4 Exercises for §1.1 6 §1.2. Finite and Countable Sample Spaces 7 1.2.1. Probability Theory on a Countable Space 7 1.2.2. Uniform Probabilities and Coin Tossing 10 1.2.3. Tournaments 13 1.2.4. Symmetric Random Walk 15 1.2.5. De Moivre’s Central Limit Theorem 17 1.2.6. Independent Events 20 1.2.7. The Arc Sine Law 24 1.2.8. Conditional Probability 27 Exercises for §1.2 29 §1.3. Some Non-Uniform Probability Measures 32 1.3.1. Random Variables and Their Distributions 32 1.3.2. Biased Coins 33 1.3.3. Recurrence and Transience of Random Walks 36 Exercises for §1.3 39 1.4. Expectation Values 40 1.4.1. Some Elementary Examples 45 1.4.2. Independence and Moment Generating Functions 47 1.4.3. Basic Convergence Results 49 Exercises for §1.4 51 Comments on Chapter 1 52 Chapter 2. Probability Theory on Uncountable Sample Spaces 55 §2.1. A Little Measure Theory 56 2.1.1. Sigma Algebras, Measurable Functions, and Measures 56 2.1.2. ?- and ?-Systems 58 Exercises for §2.1 59 §2.2. A Construction of Pp on {0,1}Z+ 59 2.2.1. The Metric Space {0,1}Z+ 59 2.2.2. The Construction 61 Exercises for §2.2 65 §2.3. Other Probability Measures 65 2.3.1. The Uniform Probability Measure on [0,1] 66 2.3.2. Lebesgue Measure on R 68 2.3.3. Distribution Functions and Probability Measures 70 Exercises for §2.3 71 §2.4. Lebesgue Integration 71 2.4.1. Integration of Functions 72 2.4.2. Some Properties of the Lebesgue Integral 77 2.4.3. Basic Convergence Theorems 80 2.4.4. Inequalities 84 2.4.5. Fubini’s Theorem 88 Exercises for §2.4 91 §2.5. Lebesgue Measure on RN 95 2.5.1. Polar Coordinates 98 2.5.2. Gaussian Computations and Stirling’s Formula 99 Exercises for §2.5 102 Comments on Chapter 2 104 Chapter 3. Some Applications to Probability Theory 105 §3.1. Independence and Conditioning 105 3.1.1. Independent s-Algebras 105 3.1.2. Independent Random Variables 107 3.1.3. Conditioning 109 3.1.4. Some Properties of Conditional Expectations 113 Exercises for §3.1 114 §3.2. Distributions that Admit a Density 117 3.2.1. Densities 117 3.2.2. Densities and Conditioning 119 Exercises for §3.2 120 §3.3. Summing Independent Random Variables 121 3.3.1. Convolution of Distributions 121 3.3.2. Some Important Examples 122 3.3.3. Kolmogorov’s Inequality and the Strong Law 124 Exercises for §3.3 130 Comments on Chapter 3 134 Chapter 4. The Central Limit Theorem and Gaussian Distributions 135 §4.1. The Central Limit Theorem 135 4.1.1. Lindeberg’s Theorem 137 Exercises for §4.1 142 §4.2. Families of Normal Random Variables 143 4.2.1. Multidimensional Gaussian Distributions 143 4.2.2. Standard Normal Random Variables 144 4.2.3. More General Normal Random Variables 146 4.2.4. A Concentration Property of Gaussian Distributions 147 4.2.5. Linear Transformations of Normal Random Variables 150 4.2.6. Gaussian Families 152 Exercises for §4.2 155 Comments on Chapter 4 158 Chapter 5. Discrete Parameter Stochastic Processes 159 §5.1. Random Walks Revisited 159 5.1.1. Immediate Rewards 159 5.1.2. Computations via Conditioning 162 Exercises for §5.1 167 §5.2. Processes with the Markov Property 168 5.2.1. Sequences of Dependent Random Variables 168 5.2.2. Markov Chains 171 5.2.3. Long-Time Behavior 171 5.2.4. An Extension 174 Exercises for §5.2 178 §5.3. Markov Chains on a Countable State Space 179 5.3.1. The Markov Property 181 5.3.2. Return Times and the Renewal Equation 182 5.3.3. A Little Ergodic Theory 185 Exercises for §5.3 188 Comments on Chapter 5 190 Chapter 6. Some Continuous-Time Processes 193 §6.1. Transition Probability Functions and Markov Processes 193 6.1.1. Transition Probability Functions 194 Exercises for §6.1 196 §6.2. Markov Chains Run with a Poisson Clock 196 6.2.1. The Simple Poisson Process 197 6.2.2. A Generalization 199 6.2.3. Stationary Measures 200 Exercises for §6.2 203 §6.3. Brownian Motion 204 6.3.1. Some Preliminaries 205 6.3.2. L ́evy’s Construction 206 6.3.3. Some Elementary Properties of Brownian Motion 209 6.3.4. Path Properties 216 6.3.5. The Ornstein–Uhlenbeck Process 219 Exercises for §6.3 222 Comments on Chapter 6 224 Chapter 7. Martingales 225 §7.1. Discrete Parameter Martingales 225 7.1.1. Doob’s Inequality 226 Exercises for §7.1 232 §7.2. The Martingale Convergence Theorem 233 7.2.1. The Convergence Theorem 234 7.2.2. Application to the Radon–Nikodym Theorem 237 Exercises for §7.2 241 §7.3. Stopping Times 242 7.3.1. Stopping Time Theorems 242 7.3.2. Reversed Martingales 247 7.3.3. Exchangeable Sequences 249 Exercises for §7.3 252 §7.4. Continuous Parameter Martingales 254 7.4.1. Progressively Measurable Functions 254 7.4.2. Martingales and Submartingales 255 7.4.3. Stopping Times Again 257 7.4.4. Continuous Martingales and Brownian Motion 259 7.4.5. Brownian Motion and Di?erential Equations 266 Exercises for §7.4 271 Comments on Chapter 7 274 Notation 275 Bibliography 279 Index 281