Applied Stochastic Analysis (Courant Lecture Notes)

This textbook introduces the major ideas of stochastic analysis with a view to modeling or simulating systems involving randomness. Suitable for students and researchers in applied mathematics and related disciplines, this book prepares readers to solve concrete problems arising in physically motivated models. The author’s practical approach avoids measure theory while retaining rigor for cases where it helps build techniques or intuition. Topics covered include Markov chains (discrete and continuous), Gaussian processes, Itô calculus, and stochastic differential equations and their associated PDEs. We ask questions such as: How does probability evolve? How do statistics evolve? How can we solve for time-dependent quantities such as first-passage times? How can we set up a model that includes fundamental principles such as time-reversibility (detailed balance)? How can we simulate a stochastic process numerically? Applied Stochastic Analysis invites readers to develop tools and insights for tackling physical systems involving randomness. Exercises accompany the text throughout, with frequent opportunities to implement simulation algorithms. A strong undergraduate background in linear algebra, probability, ODEs, and PDEs is assumed, along with the mathematical sophistication characteristic of a graduate student. Cover Title page Copyright Contents Preface Chapter 1. Introduction 1.1. What is a stochastic process? 1.2. What classes of stochastic processes will we learn about? Chapter 2. Markov Chains (I) 2.1. What is a Markov chain? 2.2. Examples of Markov chains 2.3. Forward and backward Kolmogorov equations 2.4. Long-time behavior and stationary distribution 2.5. Mean first-passage time 2.6. Additional exercises Chapter 3. Markov Chains (II): Detailed Balance and Markov Chain Monte Carlo (MCMC) 3.1. Detailed balance 3.2. Spectral decomposition for a Markov chain that satisfies detailed balance 3.3. Markov chain Monte Carlo 3.4. Additional exercises Chapter 4. Continuous-Time Markov Chains 4.1. Definition and transition probabilities 4.2. Infinitesimal generator 4.3. Transition times and jumps 4.4. Forward and backward equations 4.5. Long-time behavior 4.6. Mean first-passage time 4.7. Additional exercises Chapter 5. Gaussian Processes and Stationary Processes 5.1. Setup 5.2. Gaussian processes 5.3. Stationary processes 5.4. Spectral representation of a covariance function of a weakly stationary process 5.5. Simulating stationary Gaussian processes 5.6. Ergodic properties of weakly stationary processes 5.7. Additional exercises Chapter 6. Brownian Motion 6.1. Definition and transition densities 6.2. Brownian motion as a limit of random walks 6.3. Properties of Brownian motion 6.4. White noise 6.5. Quadratic variation 6.6. Brownian motion as a Markov process 6.7. Additional Information on weak convergence 6.8. Additional exercises Chapter 7. Stochastic Integration 7.1. How to write down differential equations with noise 7.2. What is the Itô integral and why do we need it? 7.3. Rigorous construction of the Itô integral 7.4. Properties of the Itô integral 7.5. Itô formula 7.6. ito calculus in higher dimensions 7.7. Additional exercises Chapter 8. Stochastic Differential Equations 8.1. Existence and uniqueness 8.2. Examples of SDEs and their solutions 8.3. Stratonovich integral 8.4. Additional information: Gronwall’s inequality 8.5. Additional exercises Chapter 9. Numerically Solving SDEs 9.1. Stochastic Itô–Taylor expansion to derive basic schemes 9.2. Strong and weak convergence 9.3. Stochastic stability 9.4. Implicit methods 9.5. Additional exercises Chapter 10. Forward and Backward Equations for SDEs 10.1. Markov property and transition densities 10.2. Generator of a diffusion process 10.3. Backward Kolmogorov equation 10.4. Forward Kolmogorov equation (Fokker–Planck equation) 10.5. Boundary conditions for the forward and backward equations 10.6. Stationary distribution 10.7. Additional exercises Chapter 11. Some Applications of the Backward Equation 11.1. First-passage times 11.2. Boundary-value problems 11.3. Feynman–Kac formula 11.4. Additional exercises Chapter 12. Detailed Balance, Symmetry, and Eigenfunction Expansions 12.1. When does the steady-state flux vanish? 12.2. Symmetry of L 12.3. Detailed balance 12.4. Eigenfunction methods 12.5. Even versus odd variables 12.6. Additional exercises Chapter 13. Asymptotic Analysis of SDEs 13.1. White noise limit of a colored noise 13.2. Overdamped limit of the Langevin equation 13.3. A random walk converging to a diffusion process 13.4. Additional exercises Appendix A. Appendix A.1. A brief review of probability theory A.2. Integration with respect to a measure A.3. Multivariate Gaussian random variables A.4. Stopping times A.5. Collected results about PDEs Bibliography Index Back Cover