Bag Of Bones

Taken literally, the title "All of Statistics" is an exaggeration. But in spirit, the title is apt, as the book does cover a much broader range of topics than a typical introductory book on mathematical statistics. This book is for people who want to learn probability and statistics quickly. It is suitable for graduate or advanced undergraduate students in computer science, mathematics, statistics, and related disciplines. The book includes modern topics like non-parametric curve estimation, bootstrapping, and classification, topics that are usually relegated to follow-up courses. The reader is presumed to know calculus and a little linear algebra. No previous knowledge of probability and statistics is required. Statistics, data mining, and machine learning are all concerned with collecting and analysing data. Cover Title Copyright Dedication Preface Contents I Probability 1 Probability 1.1 Introduction 1.2 Sample Spaces and Events 1.3 Probability 1.4 Probability on Finite Sample Spaces 1.5 Independent Events 1.6 Conditional Probability 1.7 Bayes’ Theorem 1.8 Bibliographic Remarks 1.9 Appendix 1.10 Exercises 2 Random Variables 2.1 Introduction 2.2 Distribution Functions and Probability Functions 2.3 Some Important Discrete Random Variables 2.4 Some Important Continuous Random Variables 2.5 Bivariate Distributions 2.6 Marginal Distributions 2.7 Independent Random Variables 2.8 Conditional Distributions 2.9 Multivariate Distributions and IID Samples 2.10 Two Important Multivariate Distributions 2.11 Transformations of Random Variables 2.12 Transformations of Several Random Variables 2.13 Appendix 2.14 Exercises 3 Expectation 3.1 Expectation of a Random Variable 3.2 Properties of Expectations 3.3 Variance and Covariance 3.4 Expectation and Variance of Important Random Variables 3.5 Conditional Expectation 3.6 Moment Generating Functions 3.7 Appendix 3.8 Exercises 4 Inequalities 4.1 Probability Inequalities 4.2 Inequalities For Expectations 4.3 Bibliographic Remarks 4.4 Appendix 4.5 Exercises 5 Convergence of Random Variables 5.1 Introduction 5.2 Types of Convergence 5.3 The Law of Large Numbers 5.4 The Central Limit Theorem 5.5 The Delta Method 5.6 Bibliographic Remarks 5.7 Appendix 5.8 Exercises II Statistical Inference 6 Models, Statistical Inference and Learning 6.1 Introduction 6.2 Parametric and Nonparametric Models 6.3 Fundamental Concepts in Inference 6.4 Bibliographic Remarks 6.5 Appendix 6.6 Exercises 7 Estimating the CDF and Statistical Functionals 7.1 The Empirical Distribution Function 7.2 Statistical Functionals 7.3 Bibliographic Remarks 7.4 Exercises 8 The Bootstrap 8.1 Simulation 8.2 Bootstrap Variance Estimation 8.3 Bootstrap Confidence Intervals 8.4 Bibliographic Remarks 8.5 Appendix 8.6 Exercises 9 Parametric Inference 9.1 Parameter of Interest 9.2 The Method of Moments 9.3 Maximum Likelihood 9.4 Properties of Maximum Likelihood Estimators 9.5 Consistency of Maximum Likelihood Estimators 9.6 Equivariance of the mle 9.7 Asymptotic Normality 9.8 Optimality 9.9 The Delta Method 9.10 Multiparameter Models 9.11 The Parametric Bootstrap 9.12 Checking Assumptions 9.13 Appendix 9.14 Exercises 10 Hypothesis Testing and p-values 10.1 The Wald Test 10.2 p-values 10.3 The χ2 Distribution 10.4 Pearson’s χ2 Test For Multinomial Data 10.5 The Permutation Test 10.6 The Likelihood Ratio Test 10.7 Multiple Testing 10.8 Goodness-of-fit Tests 10.9 Bibliographic Remarks 10.10 Appendix 10.11 Exercises 11 Bayesian Inference 11.1 The Bayesian Philosophy 11.2 The Bayesian Method 11.3 Functions of Parameters 11.4 Simulation 11.5 Large Sample Properties of Bayes’ Procedures 11.6 Flat Priors, Improper Priors, and “Noninformative” Priors 11.7 Multiparameter Problems 11.8 Bayesian Testing 11.9 Strengths and Weaknesses of Bayesian Inference 11.10 Bibliographic Remarks 11.11 Appendix 11.12 Exercises 12 Statistical Decision Theory 12.1 Preliminaries 12.2 Comparing Risk Functions 12.3 Bayes Estimators 12.4 Minimax Rules 12.5 Maximum Likelihood, Minimax, and Bayes 12.6 Admissibility 12.7 Stein’s Paradox 12.8 Bibliographic Remarks 12.9 Exercises III Statistical Models and Methods 13 Linear and Logistic Regression 13.1 Simple Linear Regression 13.2 Least Squares and Maximum Likelihood 13.3 Properties of the Least Squares Estimators 13.4 Prediction 13.5 Multiple Regression 13.6 Model Selection 13.7 Logistic Regression 13.8 Bibliographic Remarks 13.9 Appendix 13.10 Exercises 14 Multivariate Models 14.1 Random Vectors 14.2 Estimating the Correlation 14.3 Multivariate Normal 14.4 Multinomial 14.5 Bibliographic Remarks 14.6 Appendix 14.7 Exercises 15 Inference About Independence 15.1 Two Binary Variables 15.2 Two Discrete Variables 15.3 Two Continuous Variables 15.4 One Continuous Variable and One Discrete 15.5 Appendix 15.6 Exercises 16 Causal Inference 16.1 The Counterfactual Model 16.2 Beyond Binary Treatments 16.3 Observational Studies and Confounding 16.4 Simpson’s Paradox 16.5 Bibliographic Remarks 16.6 Exercises 17 Directed Graphs and Conditional Independence 17.1 Introduction 17.2 Conditional Independence 17.3 DAGs 17.4 Probability and DAGs 17.5 More Independence Relations 17.6 Estimation for DAGs 17.7 Bibliographic Remarks 17.8 Appendix 17.9 Exercises 18 Undirected Graphs 18.1 Undirected Graphs 18.2 Probability and Graphs 18.3 Cliques and Potentials 18.4 Fitting Graphs to Data 18.5 Bibliographic Remarks 18.6 Exercises 19 Log-Linear Models 19.1 The Log-Linear Model 19.2 Graphical Log-Linear Models 19.3 Hierarchical Log-Linear Models 19.4 Model Generators 19.5 Fitting Log-Linear Models to Data 19.6 Bibliographic Remarks 19.7 Exercises 20 Nonparametric Curve Estimation 20.1 The Bias-Variance Tradeoff 20.2 Histograms 20.3 Kernel Density Estimation 20.4 Nonparametric Regression 20.5 Appendix 20.6 Bibliographic Remarks 20.7 Exercises 21 Smoothing Using Orthogonal Functions 21.1 Orthogonal Functions and L2 Spaces 21.2 Density Estimation 21.3 Regression 21.4 Wavelets 21.5 Appendix 21.6 Bibliographic Remarks 21.7 Exercises 22 Classification 22.1 Introduction 22.2 Error Rates and the Bayes Classifier 22.3 Gaussian and Linear Classifiers 22.4 Linear Regression and Logistic Regression 22.5 Relationship Between Logistic Regression and LDA 22.6 Density Estimation and Naive Bayes 22.7 Trees 22.8 Assessing Error Rates and Choosing a Good Classifier 22.9 Support Vector Machines 22.10 Kernelization 22.11 Other Classifiers 22.12 Bibliographic Remarks 22.13 Exercises 23 Probability Redux: Stochastic Processes 23.1 Introduction 23.2 Markov Chains 23.3 Poisson Processes 23.4 Bibliographic Remarks 23.5 Exercises 24 Simulation Methods 24.1 Bayesian Inference Revisited 24.2 Basic Monte Carlo Integration 24.3 Importance Sampling 24.4 MCMC Part I: The Metropolis-Hastings Algorithm 24.5 MCMC Part II: Different Flavors 24.6 Bibliographic Remarks 24.7 Exercises Bibliography Index This Book Is For People Who Want To Learn Probability And Statistics Quickly. It Brings Together Many Of The Main Ideas In Modern Statistics In One Place. The Book Is Suitable For Students And Researchers In Statistics, Computer Science, Data Mining And Machine Learning. This Book Covers A Much Wider Range Of Topics Than A Typical Introductory Text On Mathematical Statistics. It Includes Modern Topics Like Nonparametric Curve Estimation, Bootstrapping And Classification, Topics That Are Usually Relegated To Follow-up Courses. The Reader Is Assumed To Know Calculus And A Little Linear Algebra. No Previous Knowledge Of Probability And Statistics Is Required. The Text Can Be Used At The Advanced Undergraduate And Graduate Level. Larry Wasserman Is Professor Of Statistics At Carnegie Mellon University. He Is Also A Member Of The Center For Automated Learning And Discovery In The School Of Computer Science. His Research Areas Include Nonparametric Inference, Asymptotic Theory, Causality, And Applications To Astrophysics, Bioinformatics, And Genetics. He Is The 1999 Winner Of The Committee Of Presidents Of Statistical Societies Presidents' Award And The 2002 Winner Of The Centre De Recherches Mathematiques De Montreal–statistical Society Of Canada Prize In Statistics. He Is Associate Editor Of The Journal Of The American Statistical Association And The Annals Of Statistics. He Is A Fellow Of The American Statistical Association And Of The Institute Of Mathematical Statistics. Probability -- Random Variables -- Expectation -- Inequalities -- Convergence Of Random Variables -- Models, Statistical Inference And Learning -- Estimating The Cdf And Statistical Functionals -- The Bootstrap -- Parametric Inference -- Hypothesis Testing And P-values -- Bayesian Inference -- Statistical Decision Theory -- Linear And Logistic Regression -- Multivariate Models -- Inference About Independence -- Causal Inference -- Directed Graphs And Conditional Independence -- Undirected Graphs -- Loglinear Models -- Nonparametric Curve Estimation -- Smoothing Using Orthogonal Functions -- Classification -- Probability Redux: Stochastic Processes -- Simulation Methods. By Larry Wasserman. "This book is for people who want to learn probability and statistics quickly. It brings together many of the main ideas in modern statistics in one place. The book is suitable for students and researchers in statistics, computer science, data mining, and machine learning. This book covers a much wider range of topics than a typical introductory text on mathematical statistics. It includes modern topics like nonparametric curve estimation, bootstrapping, and classification, topics that are usually relegated to follow-up courses. The reader is assumed to know calculus and a little linear algebra. No previous knowledge of probability and statistics is required. The text can be used at the advanced undergraduate and graduate levels"--Page 4 of cover This textbook can be used for a course for advanced undergraduates or M.A.-level courses in statistics & computer science departments. It will also serve as a reference for practitioners in machine learning