MODERN DEEP LEARNING DESIGN AND APPLICATION DEVELOPMENT : versatile tools to solve deep... learning problems

Learn how to harness modern deep-learning methods in many contexts. Packed with intuitive theory, practical implementation methods, and deep-learning case studies, this book reveals how to acquire the tools you need to design and implement like a deep-learning architect. It covers tools deep learning engineers can use in a wide range of fields, from biology to computer vision to business. With nine in-depth case studies, this book will ground you in creative, real-world deep learning thinking. You’ll begin with a structured guide to using Keras, with helpful tips and best practices for making the most of the framework. Next, you’ll learn how to train models effectively with transfer learning and self-supervised pre-training. You will then learn how to use a variety of model compressions for practical usage. Lastly, you will learn how to design successful neural network architectures and creatively reframe difficult problems into solvable ones. You’ll learn not only to understand and apply methods successfully but to think critically about it. Modern Deep Learning Design and Methods is ideal for readers looking to utilize modern, flexible, and creative deep-learning design and methods. Get ready to design and implement innovative deep-learning solutions to today’s difficult problems. What You’ll Learn Improve the performance of deep learning models by using pre-trained models, extracting rich features, and automating optimization. Compress deep learning models while maintaining performance. Reframe a wide variety of difficult problems and design effective deep learning solutions to solve them. Use the Keras framework, with some help from libraries like HyperOpt, TensorFlow, and PyTorch, to implement a wide variety of deep learning approaches. Who This Book Is For Data scientists with some familiarity with deep learning to deep learning engineers seeking structured inspiration and direction on their next project. Developers interested in harnessing modern deep learning methods to solve a variety of difficult problems. Table of Contents About the Author About the Technical Reviewer Acknowledgments Introduction Chapter 1: A Deep Dive into Keras Why Keras? Installing and Importing Keras The Simple Keras Workflow Step 1: Define Architecture Step 2: Compile Loss Functions Optimizers Metrics Step 3: Fit and Evaluate Visualizing Model Architectures Functional API Translating a Sequential to a Functional Model Building Nonlinear Topologies Dealing with Data TensorFlow Dataset from Loaded Data TensorFlow Dataset from Image Files Automatic Image Dataset from Directory ImageDataGenerator Key Points Chapter 2: Pretraining Strategies and Transfer Learning Developing Creative Training Structures The Power of Pretraining Transfer Learning Intuition Self-Supervised Learning Intuition Transfer Learning Practical Theory Transfer Learning Models and Model Structure The ImageNet Dataset ResNet InceptionV3 MobileNet EfficientNet Other Models Changing Pretrained Model Architectures Neural Network “Top” Inclusivity Layer Freezing Implementing Transfer Learning General Implementation Structure: A Template No Architecture or Weight Changes Transfer Learning Without Layer Freezing Transfer Learning with Layer Freezing Accessing PyTorch Models Implementing Simple Self-Supervised Learning Case Studies Transfer Learning Case Study: Adversarial Exploitation of Transfer Learning Self-Supervised Learning Case Study: Predicting Rotations Self-Supervised Learning Case Study: Learning Image Context and Designing Nontrivial Pretraining Tasks Key Points Chapter 3: The Versatility of Autoencoders Autoencoder Intuition and Theory The Design of Autoencoder Implementation Autoencoders for Tabular Data Autoencoders for Image Data Image Data Shape Structure and Transformations Convolutional Autoencoder Without Pooling Convolutional Autoencoder Vector Bottleneck Design Convolutional Autoencoder with Pooling and Padding Autoencoders for Other Data Forms Autoencoder Applications Using Autoencoders for Denoising Intuition and Theory Implementation Inducing Noise Using Denoising Autoencoders Using Autoencoders for Pretraining Intuition Implementation Using Autoencoders for Dimensionality Reduction Intuition Implementation Using Autoencoders for Feature Generation Intuition Implementation Using Variational Autoencoders for Data Generation Intuition Implementation Case Studies Autoencoders for Pretraining Case Study: TabNet Denoising Autoencoders Case Study: Chinese Spelling Checker Variational Autoencoders Case Study: Text Generation Key Points Chapter 4: Model Compression for Practical Deployment Introduction to Model Compression Pruning Pruning Theory and Intuition Pruning Implementation Setting Up Data and Benchmark Model Creating Cost Metrics Storage Size Latency Parameter Metrics Pruning an Entire Model Pruning Individual Layers Pruning in Theoretical Deep Learning: The Lottery Ticket Hypothesis Quantization Quantization Theory and Intuition Quantization Implementation Quantizing an Entire Model Quantizing Individual Layers Weight Clustering Weight Clustering Theory and Intuition Weight Clustering Implementation Weight Clustering on an Entire Model Weight Clustering on Individual Layers Collaborative Optimization Sparsity Preserving Quantization Cluster Preserving Quantization Sparsity Preserving Clustering Case Studies Extreme Collaborative Optimization Rethinking Quantization for Deeper Compression Responsible Compression: What Do Compressed Models Forget? Key Points Chapter 5: Automating Model Design with Meta-optimization Introduction to Meta-optimization General Hyperparameter Optimization Bayesian Optimization Intuition and Theory Hyperopt Syntax, Concepts, and Usage Hyperopt Syntax Overview: Finding the Minimum of a Simple Objective Function Using Hyperopt to Optimize Training Procedure Using Hyperopt to Optimize Model Architecture Hyperas Syntax, Concepts, and Usage Using Hyperas to Optimize Training Procedure Using Hyperas to Optimize Model Architecture Neural Architecture Search NAS Intuition and Theory Auto-Keras Auto-Keras System Simple NAS NAS with Custom Search Space NAS with Nonlinear Topology Case Studies NASNet Progressive Neural Architecture Search Efficient Neural Architecture Search Key Points Chapter 6: Successful Neural Network Architecture Design Nonlinear and Parallel Representation Residual Connections Branching and Cardinality Case Study: U-Net Block/Cell Design Sequential Cell Design Nonlinear Cell Design Case Study: InceptionV3 Neural Network Scaling Input Shape Adaptable Design Parametrization of Network Dimensions Case Study: EfficientNet Key Points Chapter 7: Reframing Difficult Deep Learning Problems Data Representation: DeepInsight Corrupted Data: Negative Learning with Noisy Labels Limited Data: Siamese Networks Key Points and Epilogue Index Learn how to harness modern deep-learning methods in many contexts. Packed with intuitive theory, practical implementation methods, and deep-learning case studies, this book reveals how to acquire the tools you need to design and implement like a deep-learning architect. It covers tools deep learning engineers can use in a wide range of fields, from biology to computer vision to business. With nine in-depth case studies, this book will ground you in creative, real-world deep learning thinking. You'll begin with a structured guide to using Keras, with helpful tips and best practices for making the most of the framework. Next, you'll learn how to train models effectively with transfer learning and self-supervised pre-training. You will then learn how to use a variety of model compressions for practical usage. Lastly, you will learn how to design successful neural network architectures and creatively reframe difficult problems into solvable ones. You'll learn not only to understand and apply methods successfully but to think critically about it. Modern Deep Learning Design and Methods is ideal for readers looking to utilize modern, flexible, and creative deep-learning design and methods. Get ready to design and implement innovative deep-learning solutions to today's difficult problems. You will: Improve the performance of deep learning models by using pre-trained models, extracting rich features, and automating optimization. Compress deep learning models while maintaining performance. Reframe a wide variety of difficult problems and design effective deep learning solutions to solve them. Use the Keras framework, with some help from libraries like HyperOpt, TensorFlow, and PyTorch, to implement a wide variety of deep learning approaches