Advanced Statistical Computing (2022 Update)

The journey from statistical model to useful output has many steps, most of which are taught in other books and courses. The purpose of this book is to focus on one particular aspect of this journey: the development and implementation of statistical algorithms. It's often nice to think about statistical models and various inferential philosophies and techniques, but when the rubber meets the road, we need an algorithm and a computer program implementation to get the results we need from a combination of our data and our models. This book is about how we fit models to data and the algorithms that we use to do so. Examples are given using the R programming language. Welcome Stay in Touch! Setup Introduction Example: Linear Models Principle of Optimization Transfer Textbooks vs. Computers Solving Nonlinear Equations Bisection Algorithm Rates of Convergence Functional Iteration Newton's Method General Optimization Steepest Descent The Newton Direction Quasi-Newton Conjugate Gradient Coordinate Descent The EM Algorithm EM Algorithm for Exponential Families Canonical Examples A Minorizing Function Missing Information Principle Acceleration Methods Integration Laplace Approximation Independent Monte Carlo Random Number Generation Non-Uniform Random Numbers Rejection Sampling Importance Sampling Markov Chain Monte Carlo Background Metropolis-Hastings Gibbs Sampler Monitoring Convergence Simulated Annealing