Adaptive Machine Learning Algorithms with Python : Solve Data Analytics and Machine Learning Problems on Edge Devices

Learn to use adaptive algorithms to solve real-world streaming data problems. This book covers a multitude of data processing challenges, ranging from the simple to the complex. At each step, you will gain insight into real-world use cases, find solutions, explore code used to solve these problems, and create new algorithms for your own use. Authors Chanchal Chatterjee and Vwani P. Roychowdhury begin by introducing a common framework for creating adaptive algorithms, and demonstrating how to use it to address various streaming data issues. Examples range from using matrix functions to solve machine learning and data analysis problems to more critical edge computation problems. They handle time-varying, non-stationary data with minimal compute, memory, latency, and bandwidth. Upon finishing this book, you will have a solid understanding of how to solve adaptive machine learning and data analytics problems and be able to derive new algorithms for your own use cases. You will also come away with solutions to high volume time-varying data with high dimensionality in a low compute, low latency environment. What You Will Learn Apply adaptive algorithms to practical applications and examples Understand the relevant data representation features and computational models for time-varying multi-dimensional data Derive adaptive algorithms for mean, median, covariance, eigenvectors (PCA) and generalized eigenvectors with experiments on real data Speed up your algorithms and put them to use on real-world stationary and non-stationary data Master the applications of adaptive algorithms on critical edge device computation applications Who This Book Is For Machine learning engineers, data scientist and architects, software engineers and architects handling edge device computation and data management. Table of Contents About the Author About the Technical Reviewer Acknowledgments Preface Chapter 1: Introduction 1.1 Commonly Used Features Obtained by Linear Transform Data Whitening Principal Components Linear Discriminant Features Singular Value Features Summary 1.2 Multi-Disciplinary Origin of Linear Features Hebbian Learning or Neural Biology Auto-Associative Networks Hetero-Associative Networks Statistical Pattern Recognition Information Theory Optimization Theory 1.3 Why Adaptive Algorithms? Iterative or Batch Processing of Static Data My Approach: Adaptive Processing of Streaming Data Requirements of Adaptive Algorithms Real-World Use of Adaptive Matrix Computation Algorithms and GitHub 1.4 Common Methodology for Derivations of Algorithms Matrix Algebra Problems Solved Here 1.5 Outline of The Book Chapter 2: General Theories and Notations 2.1 Introduction 2.2 Stationary and Non-Stationary Sequences 2.3 Use Cases for Adaptive Mean, Median, and Covariances Handwritten Character Recognition Anomaly Detection of Streaming Data 2.4 Adaptive Mean and Covariance of Nonstationary Sequences 2.5 Adaptive Covariance and Inverses 2.6 Adaptive Normalized Mean Algorithm Variations of the Adaptive Normalized Mean Algorithm 2.7 Adaptive Median Algorithm 2.8 Experimental Results Chapter 3: Square Root and Inverse Square Root 3.1 Introduction and Use Cases Various Solutions for A1⁄2 and A–1⁄2 Outline of This Chapter 3.2 Adaptive Square Root Algorithm: Method 1 Objective Function Adaptive Algorithm 3.3 Adaptive Square Root Algorithm: Method 2 Objective Function Adaptive Algorithm 3.4 Adaptive Square Root Algorithm: Method 3 Adaptive Algorithm 3.5 Adaptive Inverse Square Root Algorithm: Method 1 Objective Function Adaptive Algorithm 3.6 Adaptive Inverse Square Root Algorithm: Method 2 Objective Function Adaptive Algorithm 3.7 Adaptive Inverse Square Root Algorithm: Method 3 Adaptive Algorithm 3.8 Experimental Results Experiments for Adaptive Square Root Algorithms Experiments for Adaptive Inverse Square Root Algorithms 3.9 Concluding Remarks Chapter 4: First Principal Eigenvector 4.1 Introduction and Use Cases Outline of This Chapter 4.2 Algorithms and Objective Functions Adaptive Algorithms Objective Functions 4.3 OJA Algorithm Objective Function Adaptive Algorithm Rate of Convergence 4.4 RQ, OJAN, and LUO Algorithms Objective Function Adaptive Algorithms Rate of Convergence 4.5 IT Algorithm Objective Function Adaptive Algorithm Rate of Convergence Upper Bound of ηk 4.6 XU Algorithm Objective Function Adaptive Algorithm Rate of Convergence Upper Bound of ηk 4.7 Penalty Function Algorithm Objective Function Adaptive Algorithm Rate of Convergence Upper Bound of ηk 4.8 Augmented Lagrangian 1 Algorithm Objective Function and Adaptive Algorithm Rate of Convergence Upper Bound of ηk 4.9 Augmented Lagrangian 2 Algorithm Objective Function Adaptive Algorithm Rate of Convergence Upper Bound of ηk 4.10 Summary of Algorithms 4.11 Experimental Results Experiments with Various Starting Vectors w0 Experiments with Various Data Sets: Set 1 Experiments with Various Data Sets: Set 2 Experiments with Real-World Non-Stationary Data 4.12 Concluding Remarks Chapter 5: Principal and Minor Eigenvectors 5.1 Introduction and Use Cases Unified Framework Outline of This Chapter 5.2 Algorithms and Objective Functions Summary of Objective Functions for Adaptive Algorithms 5.3 OJA Algorithms OJA Homogeneous Algorithm OJA Deflation Algorithm OJA Weighted Algorithm OJA Algorithm Python Code 5.4 XU Algorithms XU Homogeneous Algorithm XU Deflation Algorithm XU Weighted Algorithm XU Algorithm Python Code 5.5 PF Algorithms PF Homogeneous Algorithm PF Deflation Algorithm PF Weighted Algorithm PF Algorithm Python Code 5.6 AL1 Algorithms AL1 Homogeneous Algorithm AL1 Deflation Algorithm AL1 Weighted Algorithm AL1 Algorithm Python Code 5.7 AL2 Algorithms AL2 Homogeneous Algorithm AL2 Deflation Algorithm AL2 Weighted Algorithm AL2 Algorithm Python Code 5.8 IT Algorithms IT Homogeneous Function IT Deflation Algorithm IT Weighted Algorithm IT Algorithm Python Code 5.9 RQ Algorithms RQ Homogeneous Algorithm RQ Deflation Algorithm RQ Weighted Algorithm RQ Algorithm Python Code 5.10 Summary of Adaptive Eigenvector Algorithms 5.11 Experimental Results 5.12 Concluding Remarks Chapter 6: Accelerated Computation of Eigenvectors 6.1 Introduction Objective Functions for Gradient-Based Adaptive PCA Outline of This Chapter 6.2 Gradient Descent Algorithm 6.3 Steepest Descent Algorithm Computation of for Steepest Descent Steepest Descent Algorithm Code 6.4 Conjugate Direction Algorithm Conjugate Direction Algorithm Code 6.5 Newton-Raphson Algorithm Newton-Raphson Algorithm Code 6.6 Experimental Results Experiments with Stationary Data Experiments with Non-Stationary Data Comparison with State-of-the-Art Algorithms 6.7 Concluding Remarks Chapter 7: Generalized Eigenvectors 7.1 Introduction and Use Cases Application of GEVD in Pattern Recognition Application of GEVD in Signal Processing Methods for Generalized Eigen-Decomposition Outline of This Chapter 7.2 Algorithms and Objective Functions Summary of Objective Functions for Adaptive GEVD Algorithms Summary of Generalized Eigenvector Algorithms 7.3 OJA GEVD Algorithms OJA Homogeneous Algorithm OJA Deflation Algorithm OJA Weighted Algorithm OJA Algorithm Python Code 7.4 XU GEVD Algorithms XU Homogeneous Algorithm XU Deflation Algorithm XI Weighted Algorithm XU Algorithm Python Code 7.5 PF GEVD Algorithms PF Homogeneous Algorithm PF Deflation Algorithm PF Weighted Algorithm PF Algorithm Python Code 7.6 AL1 GEVD Algorithms AL1 Homogeneous Algorithm AL1 Deflation Algorithm AL1 Weighted Algorithm AL1 Algorithm Python Code 7.7 AL2 GEVD Algorithms AL2 Homogeneous Algorithm AL2 Deflation Algorithm AL2 Weighted Algorithm AL2 Algorithm Python Code 7.8 IT GEVD Algorithms IT Homogeneous Algorithm IT Deflation Algorithm IT Weighted Algorithm IT Algorithm Python Code 7.9 RQ GEVD Algorithms RQ Homogeneous Algorithm RQ Deflation Algorithm RQ Weighted Algorithm RQ Algorithm Python Code 7.10 Experimental Results 7.11 Concluding Remarks Chapter 8: Real-World Applications of Adaptive Linear Algorithms 8.1 Detecting Feature Drift INSECTS-incremental_balanced_norm Dataset: Eigenvector Test Adaptive EVD of Semi-Stationary Components Adaptive EVD of Non-Stationary Components INSECTS-incremental-abrupt_balanced _norm Dataset Electricity Dataset 8.2 Adapting to Incoming Data Drift 8.3 Compressing High Volume and High Dimensional Data Data Representation (PCA) Features 8.4 Detecting Feature Anomalies Yahoo Real Dataset NOAA Dataset References Index