Statistical Learning with Math and Python : 100 Exercises for Building Logic

"The most crucial ability for machine learning and data science is mathematical logic for grasping their essence rather than knowledge and experience. This textbook approaches the essence of machine learning and data science by considering math problems and building Python programs. As the preliminary part, Chapter 1 provides a concise introduction to linear algebra, which will help novices read further to the following main chapters. Those succeeding chapters present essential topics in statistical learning: linear regression, classification, resampling, information criteria, regularization, nonlinear regression, decision trees, support vector machines, and unsupervised learning. Each chapter mathematically formulates and solves machine learning problems and builds the programs. The body of a chapter is accompanied by proofs and programs in an appendix, with exercises at the end of the chapter. Because the book is carefully organized to provide the solutions to the exercises in each chapter, readers can solve the total of 100 exercises by simply following the contents of each chapter. This textbook is suitable for an undergraduate or graduate course consisting of about 12 lectures. Written in an easy-to-follow and self-contained style, this book will also be perfect material for independent learning."--Back cover Preface What Makes SLMP Unique? How to Use This Book Acknowledgments Contents 1 Linear Algebra 1.1 Inverse Matrix 1.2 Determinant 1.3 Linear Independence 1.4 Vector Spaces and Their Dimensions 1.5 Eigenvalues and Eigenvectors 1.6 Orthonormal Bases and Orthogonal Matrix 1.7 Diagonalization of Symmetric Matrices Appendix: Proof of Propositions 2 Linear Regression 2.1 Least Squares Method 2.2 Multiple Regression 2.3 Distribution of 2.4 Distribution of the RSS Values 2.5 Hypothesis Testing for j=0 2.6 Coefficient of Determination and the Detection of Collinearity 2.7 Confidence and Prediction Intervals Appendix: Proofs of Propositions Exercises 1–18 3 Classification 3.1 Logistic Regression 3.2 Newton–Raphson Method 3.3 Linear and Quadratic Discrimination 3.4 k-Nearest Neighbor Method 3.5 ROC Curves Exercises 19–31 4 Resampling 4.1 Cross-Validation 4.2 CV Formula for Linear Regression 4.3 Bootstrapping Appendix: Proof of Propositions Exercises 32–39 5 Information Criteria 5.1 Information Criteria 5.2 Efficient Estimation and the Fisher Information Matrix 5.3 Kullback–Leibler Divergence 5.4 Derivation of Akaike's Information Criterion Appendix: Proof of Propositions Exercises 40–48 6 Regularization 6.1 Ridge 6.2 Subderivative 6.3 Lasso 6.4 Comparing Ridge and Lasso 6.5 Setting the λ Value Exercises 49–56 7 Nonlinear Regression 7.1 Polynomial Regression 7.2 Spline Regression 7.3 Natural Spline Regression 7.4 Smoothing Spline 7.5 Local Regression 7.6 Generalized Additive Models Appendix: Proofs of Propositions Exercises 57–68 8 Decision Trees 8.1 Decision Trees for Regression 8.2 Decision Tree for Classification 8.3 Bagging 8.4 Random Forest 8.5 Boosting Exercises 69–74 9 Support Vector Machine 9.1 Optimum Boarder 9.2 Theory of Optimization 9.3 The Solution of Support Vector Machines 9.4 Extension of Support Vector Machines Using a Kernel Appendix: Proofs of Propositions Exercises 75–87 10 Unsupervised Learning 10.1 K-means Clustering 10.2 Hierarchical Clustering 10.3 Principle Component Analysis Appendix: Program Exercises 88–100 Index