CS229 Lecture Notes

https://cs229.stanford.edu/ CS229: Machine Learning Course Description This course provides a broad introduction to machine learning and statistical pattern recognition. Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs, practical advice); reinforcement learning and adaptive control. The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing. I Supervised learning Linear regression LMS algorithm The normal equations Matrix derivatives Least squares revisited Probabilistic interpretation Locally weighted linear regression (optional reading) Classification and logistic regression Logistic regression Digression: the perceptron learning algorithn Another algorithm for maximizing () Generalized linear models The exponential family Constructing GLMs Ordinary least squares Logistic regression Softmax regression Generative learning algorithms Gaussian discriminant analysis The multivariate normal distribution The Gaussian discriminant analysis model Discussion: GDA and logistic regression Naive bayes Laplace smoothing Event models for text classification Kernel methods Feature maps LMS (least mean squares) with features LMS with the kernel trick Properties of kernels Support vector machines Margins: intuition Notation (option reading) Functional and geometric margins (option reading) The optimal margin classifier (option reading) Lagrange duality (optional reading) Optimal margin classifiers: the dual form (option reading) Regularization and the non-separable case (optional reading) The SMO algorithm (optional reading) Coordinate ascent SMO II Deep learning Deep learning Supervised learning with non-linear models Neural networks Backpropagation Preliminary: chain rule One-neuron neural networks Two-layer neural networks: a low-level unpacked computation Two-layer neural network with vector notation Multi-layer neural networks Vectorization over training examples III Generalization and regularization Generalization Bias-variance tradeoff A mathematical decomposition (for regression) The double descent phenomenon Sample complexity bounds (optional readings) Preliminaries The case of finite H The case of infinite H Regularization and model selection Regularization Implicit regularization effect Model selection via cross validation Bayesian statistics and regularization IV Unsupervised learning Clustering and the k-means algorithm EM algorithms EM for mixture of Gaussians Jensen's inequality General EM algorithms Other interpretation of ELBO Mixture of Gaussians revisited Variational inference and variational auto-encoder (optional reading) Principal components analysis Independent components analysis ICA ambiguities Densities and linear transformations ICA algorithm Self-supervised learning and foundation models Pretraining and adaptation Pretraining methods in computer vision Pretrained large language models Zero-shot learning and in-context learning V Reinforcement Learning and Control Reinforcement learning Markov decision processes Value iteration and policy iteration Learning a model for an MDP Continuous state MDPs Discretization Value function approximation Connections between Policy and Value Iteration (Optional) LQR, DDP and LQG Finite-horizon MDPs Linear Quadratic Regulation (LQR) From non-linear dynamics to LQR Linearization of dynamics Differential Dynamic Programming (DDP) Linear Quadratic Gaussian (LQG) Policy Gradient (REINFORCE)