PyTorch Recipes: A Problem-Solution Approach to Build, Train and Deploy Neural Network Models, 2nd Edition

Learn how to use PyTorch to build neural network models using code snippets updated for this second edition. This book includes new chapters covering topics such as distributed PyTorch modeling, deploying PyTorch models in production, and developments around PyTorch with updated code. You'll start by learning how to use tensors to develop and fine-tune neural network models and implement deep learning models such as LSTMs, and RNNs. Next, you'll explore probability distribution concepts using PyTorch, as well as supervised and unsupervised algorithms with PyTorch. This is followed by a deep dive on building models with convolutional neural networks, deep neural networks, and recurrent neural networks using PyTorch. This new edition covers also topics such as Scorch, a compatible module equivalent to the Scikit machine learning library, model quantization to reduce parameter size, and preparing a model for deployment within a production system. Distributed parallel processing for balancing PyTorch workloads, using PyTorch for image processing, audio analysis, and model interpretation are also covered in detail. Each chapter includes recipe code snippets to perform specific activities. By the end of this book, you will be able to confidently build neural network models using PyTorch. PyTorch is a recent entrant to the league of graph computation tools/programming languages. Addressing the limitations of previous frameworks, PyTorch promises a better user experience in the deployment of deep learning models and the creation of advanced models using a combination of convolutional neural networks, recurrent neural networks, LSTMs, and deep neural networks. What You Will Learn Utilize new code snippets and models to train machine learning models using PyTorch Train deep learning models with fewer and smarter implementations Explore the PyTorch framework for model explainability and to bring transparency to model interpretation Build, train, and deploy neural network models designed to scale with PyTorch Understand best practices for evaluating and fine-tuning models using PyTorch Use advanced torch features in training deep neural networks Explore various neural network models using PyTorch Discover functions compatible with sci-kit learn compatible models Perform distributed PyTorch training and execution Who This Book Is For Machine learning engineers, data scientists and Python programmers and software developers interested in learning the PyTorch framework. Table of Contents About the Author About the Technical Reviewer Acknowledgments Introduction Chapter 1: Introduction to PyTorch, Tensors, and Tensor Operations What Is PyTorch? PyTorch Installation Recipe 1-1. Using Tensors Problem Solution How It Works Conclusion Chapter 2: Probability Distributions Using PyTorch Recipe 2-1. Sampling Tensors Problem Solution How It Works Recipe 2-2. Variable Tensors Problem Solution How It Works Recipe 2-3. Basic Statistics Problem Solution How It Works Recipe 2-4. Gradient Computation Problem Solution How It Works Recipe 2-5. Tensor Operations Problem Solution How It Works Recipe 2-6. Tensor Operations Problem Solution How It Works Recipe 2-7. Distributions Problem Solution How It Works Conclusion Chapter 3: CNN and RNN Using PyTorch Recipe 3-1. Setting Up a Loss Function Problem Solution How It Works Recipe 3-2. Estimating the Derivative of the Loss Function Problem Solution How It Works Recipe 3-3. Fine-Tuning a Model Problem Solution How It Works Recipe 3-4. Selecting an Optimization Function Problem Solution How It Works Recipe 3-5. Further Optimizing the Function Problem Solution How It Works Recipe 3-6. Implementing a Convolutional Neural Network (CNN) Problem Solution How It Works Recipe 3-7. Reloading a Model Problem Solution How It Works Recipe 3-8. Implementing a Recurrent Neural Network Problem Solution How It Works Recipe 3-9. Implementing a RNN for Regression Problems Problem Solution How It Works Recipe 3-10. Using PyTorch’s Built-In Functions Problem Solution How It Works Recipe 3-11. Working with Autoencoders Problem Solution How It Works Recipe 3-12. Fine-Tuning Results Using Autoencoder Problem Solution How It Works Recipe 3-13. Restricting Model Overfitting Problem Solution How It Works Recipe 3-14. Visualizing the Model Overfit Problem Solution How It Works Recipe 3-15. Initializing Weights in the Dropout Rate Problem Solution How It Works Recipe 3-16. Adding Math Operations Problem Solution How It Works Recipe 3-17. Embedding Layers in RNN Problem Solution How It Works Conclusion Chapter 4: Introduction to Neural Networks Using PyTorch Recipe 4-1. Working with Activation Functions Problem Solution How It Works Linear Function Bilinear Function Sigmoid Function Hyperbolic Tangent Function Log Sigmoid Transfer Function ReLU Function Leaky ReLU Recipe 4-2. Visualizing the Shape of Activation Functions Problem Solution How It Works Recipe 4-3. Basic Neural Network Model Problem Solution How It Works Recipe 4-4. Tensor Differentiation Problem Solution How It Works Conclusion Chapter 5: Supervised Learning Using PyTorch Introduction to Linear Regression Recipe 5-1. Data Preparation for a Supervised Model Problem Solution How It Works Recipe 5-2. Forward and Backward PropagationNeural network Problem Solution How It Works Recipe 5-3. Optimization and Gradient Computation Problem Solution How It Works Recipe 5-4. Viewing Predictions Problem Solution How It Works Recipe 5-5. Supervised Model Logistic Regression Problem Solution How It Works Conclusion Chapter 6: Fine-Tuning Deep Learning Models Using PyTorch Recipe 6-1. Building Sequential Neural Networks Problem Solution How It Works Recipe 6-2. Deciding the Batch Size Problem Solution How It Works Recipe 6-3. Deciding the Learning Rate Problem Solution How It Works Recipe 6-4. Performing Parallel Training Problem Solution How It Works Conclusion Chapter 7: Natural Language Processing Using PyTorch Recipe 7-1. Word Embedding Problem Solution How It Works Recipe 7-2. CBOW Model in PyTorch Problem Solution How It Works Recipe 7-3. LSTM Model Problem Solution How It Works Summary Chapter 8: Distributed PyTorch Modelling, Model Optimization, and Deployment Recipe 8-1. Distributed Torch Architecture Problem Solution How It Works Recipe 8-2. Components of Torch Distributed Problem Solution How It Works Recipe 8-3. Setting Up Distributed PyTorch Problem Solution How It Works Recipe 8-4. Loading Data to Distributed PyTorch Problem Solution How It Works Recipe 8-5. Quantization of Models in PyTorch Problem Solution How It Works Recipe 8-6. Quantization Observer Application Problem Solution How It Works Recipe 8-7. Quantization Application Using the MNIST Dataset Problem Solution How It Works Summary Chapter 9: Data Augmentation, Feature Engineering, and Extractions for Image and Audio Recipe 9-1. Spectogram for Audio Processing Problem Solution How It Works Recipe 9-2. Installation of Torchaudio Problem Solution How It Works Recipe 9-3. Loading Audio Files into PyTorch Problem Solution How It Works Recipe 9-4. Installation of Librosa for Audio Problem Solution How It Works Recipe 9-5. Spectogram Transformation Problem Solution How It Works Recipe 9-6. Griffin-Lim Transformation Problem Solution How It Works Recipe 9-7. Mel Scale Transformation Using a Filter Bank Problem Solution How It Works Recipe 9-8. Librosa Mel Scale Conversion vs. the Torchaudio Version Problem Solution How It Works Recipe 9-9. MFCC and LFCC Using Librosa and Torchaudio Problem Solution How It Works Recipe 9-10. Data Augmentation for Images Problem Solution How It Works Conclusion Chapter 10: PyTorch Model Interpretability and Interface to Sklearn Recipe 10-1. Installation of Captum Problem Solution How It Works Recipe 10-2. Primary Attribution Feature Importance of a Deep Learning Model Problem Solution How It Works Recipe 10-3. Neuron Importance of a Deep Learning Model Problem Solution How It Works Recipe 10-4. Installation of Skorch Problem Solution How It Works Recipe 10-5. Skorch Components for a Neuralnet Classifier Problem Solution How It Works Recipe 10-6. Skorch Neuralnet Regressor Problem Solution How It Works Recipe 10-7. Skorch Model Save and Load Problem Solution How It Works Recipe 10-8. Skorch Model Pipeline Creation Problem Solution How It Works Recipe 10-9. Skorch Model Epoch Scoring Problem Solution How It Works Recipe 10-10. Grid Search for Best Hyper Parameter Problem Solution How It Works Conclusion Index