Machine Learning: A Bayesian and Optimization Perspective (Net Developers)

This tutorial text gives a unifying perspective on machine learning by covering both probabilistic and deterministic approaches -which are based on optimization techniques – together with the Bayesian inference approach, whose essence lies in the use of a hierarchy of probabilistic models. The book presents the major machine learning methods as they have been developed in different disciplines, such as statistics, statistical and adaptive signal processing and computer science. Focusing on the physical reasoning behind the mathematics, all the various methods and techniques are explained in depth, supported by examples and problems, giving an invaluable resource to the student and researcher for understanding and applying machine learning concepts. The book builds carefully from the basic classical methods to the most recent trends, with chapters written to be as self-contained as possible, making the text suitable for different courses: pattern recognition, statistical/adaptive signal processing, statistical/Bayesian learning, as well as short courses on sparse modeling, deep learning, and probabilistic graphical models. All major classical techniques: Mean/Least-Squares regression and filtering, Kalman filtering, stochastic approximation and online learning, Bayesian classification, decision trees, logistic regression and boosting methods. The latest trends: Sparsity, convex analysis and optimization, online distributed algorithms, learning in RKH spaces, Bayesian inference, graphical and hidden Markov models, particle filtering, deep learning, dictionary learning and latent variables modeling. Case studies - protein folding prediction, optical character recognition, text authorship identification, fMRI data analysis, change point detection, hyperspectral image unmixing, target localization, channel equalization and echo cancellation, show how the theory can be applied. MATLAB code for all the main algorithms are available on an accompanying website, enabling the reader to experiment with the code.

This tutorial text gives a unifying perspective on machine learning by covering both probabilistic and deterministic approaches -which are based on optimization techniques – together with the Bayesian inference approach, whose essence lies in the use of a hierarchy of probabilistic models. The book presents the major machine learning methods as they have been developed in different disciplines, such as statistics, statistical and adaptive signal processing and computer science. Focusing on the physical reasoning behind the mathematics, all the various methods and techniques are explained in depth, supported by examples and problems, giving an invaluable resource to the student and researcher for understanding and applying machine learning concepts.

The book builds carefully from the basic classical methods to the most recent trends, with chapters written to be as self-contained as possible, making the text suitable for different courses: pattern recognition, statistical/adaptive signal processing, statistical/Bayesian learning, as well as short courses on sparse modeling, deep learning, and probabilistic graphical models.

• All major classical techniques: Mean/Least-Squares regression and filtering, Kalman filtering, stochastic approximation and online learning, Bayesian classification, decision trees, logistic regression and boosting methods.
• The latest trends: Sparsity, convex analysis and optimization, online distributed algorithms, learning in RKH spaces, Bayesian inference, graphical and hidden Markov models, particle filtering, deep learning, dictionary learning and latent variables modeling.
• Case studies - protein folding prediction, optical character recognition, text authorship identification, fMRI data analysis, change point detection, hyperspectral image unmixing, target localization, channel equalization and echo cancellation, show how the theory can be applied.
• MATLAB code for all the main algorithms are available on an accompanying website, enabling the reader to experiment with the code.

Content: Front Matter, Pages i-ii Copyright, Page iv Preface, Page xvii Acknowledgments, Page xix Notation, Page xxi Dedication, Page xxiii Chapter 1 - Introduction, Pages 1-8 Chapter 2 - Probability and Stochastic Processes, Pages 9-51 Chapter 3 - Learning in Parametric Modeling: Basic Concepts and Directions, Pages 53-103 Chapter 4 - Mean-Square Error Linear Estimation, Pages 105-160 Chapter 5 - Stochastic Gradient Descent: The LMS Algorithm and its Family, Pages 161-231 Chapter 6 - The Least-Squares Family, Pages 233-274 Chapter 7 - Classification: A Tour of the Classics, Pages 275-325 Chapter 8 - Parameter Learning: A Convex Analytic Path, Pages 327-402 Chapter 9 - Sparsity-Aware Learning: Concepts and Theoretical Foundations, Pages 403-448 Chapter 10 - Sparsity-Aware Learning: Algorithms and Applications, Pages 449-507 Chapter 11 - Learning in Reproducing Kernel Hilbert Spaces, Pages 509-583 Chapter 12 - Bayesian Learning: Inference and the EM Algorithm, Pages 585-638 Chapter 13 - Bayesian Learning: Approximate Inference and Nonparametric Models, Pages 639-706 Chapter 14 - Monte Carlo Methods, Pages 707-744 Chapter 15 - Probabilistic Graphical Models: Part I, Pages 745-793 Chapter 16 - Probabilistic Graphical Models: Part II, Pages 795-843 Chapter 17 - Particle Filtering, Pages 845-873 Chapter 18 - Neural Networks and Deep Learning, Pages 875-936 Chapter 19 - Dimensionality Reduction and Latent Variables Modeling, Pages 937-1011 Appendix A - Linear Algebra, Pages 1013-1017 Appendix B - Probability Theory and Statistics, Pages 1019-1022 Appendix C - Hints on Constrained Optimization, Pages 1023-1029 Index, Pages 1031-1050 This book presents the major machine learning methods as they have been developed in different disciplines, such as statistics, statistical and adaptive signal processing and computer science. Focusing on the physical reasoning behind the mathematics, all the various methods and techniques are explained in depth. Topics include: all major classical techniques: mean/least-squares regression and filtering, Kalman filtering, stochastic approximation and online learning, Bayesian classification, decision trees, logistic regression and boosting methods; latest trends; case studies - protein folding prediction, optical character recognition, text authorship identification, fMRI data analysis, change point detection, hyperspectral image unmixing, target localization, channel equalization and echo cancellation, and how the theory can be applied. -- Edited summary from book