Beginning Anomaly Detection Using Python-Based Deep Learning, 2nd Edition: Implement Anomaly Detection Applications with Keras and PyTorch

workshop on Dependable and Secure Machine Learning held in Portland, Oregon, USA in June 2019. Currently, he works on various anomaly detection tasks spanning behavioral tracking and geospatial trajectory modeling. He is quite passionate about deep learning, and specializes in various fields ranging from video processing to generative modeling, object tracking, time-series modeling, and more. Sridhar Alla is the co-founder and CTO of Bluewhale, which helps organizations big and small in building AI-driven big data solutions and analytics, as well as SAS2PY, a powerful tool to automate migration of SAS workloads to Python-based environments using Pandas or PySpark. He is a published author of books and an avid presenter at numerous Strata, Hadoop World, Spark Summit, and other conferences. He also has several patents filed with the US PTO on large-scale computing and distributed systems. He has extensive hands-on experience in several technologies, including Spark, Flink, Hadoop, AWS, Azure, TensorFlow, Cassandra, and others. He spoke on anomaly detection using deep learning at Strata SFO in March 2019 and at Strata London in October 2019. He was born in Hyderabad, India, and now lives in New Jersey with his wife, Rosie, his daughters, Evelyn and Madelyn, and his son, Jayson. When he is not busy writing code, he loves to spend time with his family and also training, coaching, and organizing meetups. xi ## About the Technical Reviewers Puneet Sinha has accumulated more than 12 years of work experience in developing and deploying end-to-end models in credit risk, multiple marketing optimization, A/B testing, demand forecasting and brand evaluation, profit and price analyses, anomaly and fraud detection, propensity modeling, recommender systems, upsell/cross-sell models, modeling response to incentives, price optimization, natural language processing, and OCR using ML/deep learning algorithms. Shubho Mohanty is a product thinker and creator, bringing two decades of experience in the "concept-to-market" life cycle of some of the unique, innovative, and highly successful industry-first products and platforms in the data and security spaces. Shubho holds 12+ US patents in data, analytics, and cloud security. He has also been awarded IDG CIO100, 2020 for strategizing and developing a technology innovation ecosystem. He currently serves as the Chief Product Officer at Calibo, where he leads the product vision, strategy, innovation, and development of Calibo's enterprise PaaS. Prior to Calibo, Shubho was the Global VP of Product & Engineering at CDK Global (formerly, ADP Inc). He has also served in various product leadership roles in organizations like Symantec and Microsoft. He also co-founded Ganos, a B2B data start-up. He received his B.Tech. in Electrical Engineering from National Institute of Technology (NIT), India. He is a mentor to many high-repute start-up programs where he Table of Contents About the Authors About the Technical Reviewers Acknowledgments Introduction Chapter 1: Introduction to Anomaly Detection What Is an Anomaly? Anomalous Swans Anomalies as Data Points Anomalies in a Time Series Personal Spending Pattern Taxi Cabs Categories of Anomalies Data Point–Based Anomalies Context-Based Anomalies Pattern-Based Anomalies Anomaly Detection Outlier Detection Noise Removal Novelty Detection Event Detection Change Point Detection Anomaly Score Calculation The Three Styles of Anomaly Detection Where Is Anomaly Detection Used? Data Breaches Identity Theft Manufacturing Networking Medicine Video Surveillance Environment Summary Chapter 2: Introduction to Data Science Data Science Dataset Pandas, Scikit-Learn, and Matplotlib Data I/O Data Loading Data Saving DataFrame Creation Data Manipulation Select Filtering Sorting Applying Functions Grouping Combining DataFrames Creating, Renaming, and Dropping Columns Data Analysis Value Counts Pandas .describe() Method Pandas Correlation Matrix Visualization Line Chart Chart Customization Scatter Plot Histogram Bar Graph Data Processing Nulls Categorical Encoding Scaling and Normalizing Feature Engineering and Selection Summary Chapter 3: Introduction to Machine Learning Machine Learning Introduction to Machine Learning Data Splitting Modeling and Evaluation Classification Metrics Regression Metrics Overfitting and Bias-Variance Tradeoff Hyperparameter Tuning Validation Summary Chapter 4: Traditional Machine Learning Algorithms Traditional Machine Learning Algorithms Isolation Forest Example of an Isolation Forest Anomaly Detection with an Isolation Forest Data Preparation Training Hyperparameter Tuning Evaluation and Summary One-Class Support Vector Machine How Does OC-SVM Work? Anomaly Detection with OC-SVM Data Preparation Training Hyperparameter Tuning Evaluation and Summary Summary Chapter 5: Introduction to Deep Learning Introduction to Deep Learning What Is Deep Learning? The Neuron Activation Functions Neural Networks Loss Functions Regression Classification Gradient Descent and Backpropagation Loss Curve Regularization Optimizers Multilayer Perceptron Supervised Anomaly Detection Simple Neural Network: Keras Simple Neural Network: PyTorch Summary Chapter 6: Autoencoders What Are Autoencoders? Simple Autoencoders Sparse Autoencoders Deep Autoencoders Convolutional Autoencoders Denoising Autoencoders Variational Autoencoders Summary Chapter 7: Generative Adversarial Networks What Is a Generative Adversarial Network? Generative Adversarial Network Architecture Wasserstein GAN WGAN-GP Anomaly Detection with a GAN Summary Chapter 8: Long Short-Term Memory Models Sequences and Time Series Analysis What Is an RNN? What Is an LSTM? LSTM for Anomaly Detection Examples of Time Series art_daily_no_noise.csv art_daily_nojump.csv art_daily_jumpsdown.csv art_daily_perfect_square_wave.csv art_load_balancer_spikes.csv ambient_temperature_system_failure.csv ec2_cpu_utilization.csv rds_cpu_utilization.csv Summary Chapter 9: Temporal Convolutional Networks What Is a Temporal Convolutional Network? Dilated Temporal Convolutional Network Anomaly Detection with the Dilated TCN Encoder-Decoder Temporal Convolutional Network Anomaly Detection with the ED-TCN Summary Chapter 10: Transformers What Is a Transformer? Transformer Architecture Transformer Encoder Transformer Decoder Transformer Inference Anomaly Detection with the Transformer Summary Chapter 11: Practical Use Cases and Future Trends of Anomaly Detection Anomaly Detection Real-World Use Cases of Anomaly Detection Telecom Banking Environmental Health Care Transportation Social Media Finance and Insurance Cybersecurity Video Surveillance Manufacturing Smart Home Retail Implementation of Deep Learning–Based Anomaly Detection Future Trends Summary Index This beginner-oriented book will help you understand and perform anomaly detection by learning cutting-edge machine learning and deep learning techniques. This updated second edition focuses on supervised, semi-supervised, and unsupervised approaches to anomaly detection. Over the course of the book, you will learn how to use Keras and PyTorch in practical applications. It also introduces new chapters on GANs and transformers to reflect the latest trends in deep learning. Beginning Anomaly Detection Using Python-Based Deep Learning begins with an introduction to anomaly detection, its importance, and its applications. It then covers core data science and machine learning modeling concepts before delving into traditional machine learning algorithms such as OC-SVM and Isolation Forest for anomaly detection using scikit-learn. Following this, the authors explain the essentials of machine learning and deep learning, and how to implement multilayer perceptrons for supervised anomaly detection in both Keras and PyTorch. From here, the focus shifts to the applications of deep learning models for anomaly detection, including various types of autoencoders, recurrent neural networks (via LSTM), temporal convolutional networks, and transformers, with the latter three architectures applied to time-series anomaly detection. This edition has a new chapter on GANs (Generative Adversarial Networks), as well as new material covering transformer architecture in the context of time-series anomaly detection. After completing this book, you will have a thorough understanding of anomaly detection as well as an assortment of methods to approach it in various contexts, including time-series data. Additionally, you will have gained an introduction to scikit-learn, GANs, transformers, Keras, and PyTorch, empowering you to create your own machine learning- or deep learning-based anomaly detectors. What You Will Learn Understand what anomaly detection is, why it it is important, and how it is applied Grasp the core concepts of machine learning. Master traditional machine learning approaches to anomaly detection using scikit-kearn. Understand deep learning in Python using Keras and PyTorch Process data through pandas and evaluate your model's performance using metrics like F1-score, precision, and recall Apply deep learning to supervised, semi-supervised, and unsupervised anomaly detection tasks for tabular datasets and time series applications Who This Book Is For Data scientists and machine learning engineers of all levels of experience interested in learning the basics of deep learning applications in anomaly detection.