Hands-on Deep Learning: A Guide to Deep Learning with Projects and Applications

This book discusses deep learning, from its fundamental principles to its practical applications, with hands-on exercises and coding. It focuses on deep learning techniques and shows how to apply them across a wide range of practical scenarios. The book begins with an introduction to the core concepts of deep learning. It delves into topics such as transfer learning, multi-task learning, and end-to-end learning, providing insights into various deep learning models and their real-world applications. Next, it covers neural networks, progressing from single-layer perceptrons to multi-layer perceptrons, and solving the complexities of backpropagation and gradient descent. It explains optimizing model performance through effective techniques, addressing key considerations such as hyperparameters, bias, variance, and data division. It also covers convolutional neural networks (CNNs) through two comprehensive chapters, covering the architecture, components, and significance of kernels implementing well-known CNN models such as AlexNet and LeNet. It concludes with exploring autoencoders and generative models such as Hopfield Networks and Boltzmann Machines, applying these techniques to a diverse set of practical applications. These applications include image classification, object detection, sentiment analysis, COVID-19 detection, and ChatGPT. By the end of this book, you will have gained a thorough understanding of deep learning, from its fundamental principles to its innovative applications, enabling you to apply this knowledge to solve a wide range of real-world problems. What You Will Learn What are deep neural networks? What is transfer learning, multi-task learning, and end-to-end learning? What are hyperparameters, bias, variance, and data division? What are CNN and RNN? Who This Book Is For Machine learning engineers, data scientists, AI practitioners, software developers, and engineers interested in deep learning Table of Contents About the Author About the Technical Reviewers Acknowledgments Chapter 1: Revisiting Machine Learning Machine Learning: Brief History, Definition, and Applications Types of Machine Learning: Task (T) Performance (P) Conventional Machine Learning Pipeline Regression Feature Selection Filter Method Wrapper Method Filter vs. Wrapper Methods Feature Extraction Gray-Level Co-occurrence Matrix Local Binary Pattern Histogram of Oriented Gradients Principal Component Analysis Bias–Variance Trade-off Overfitting and Underfitting Bias and Variance Application: Classification of Handwritten Digits Using a Conventional Machine Learning Pipeline Conclusion Exercises Multiple-Choice Questions Applications References Chapter 2: Introduction to Deep Learning Neurons From Perceptron to the Winter of Artificial Intelligence Imagery and Convolutional Neural Networks What’s New Sequences The Definition Generate Data Using Deep Learning Conclusion Exercises Multiple-Choice Questions Activity References Chapter 3: Neural Networks Objectives Introduction Single-Layer Perceptron Implementation of a SLP XOR Problem Activation Functions 1. Sigmoid 2. Tanh 3. Rectified Linear Unit (ReLU) 4. Softmax Multi-layer Perceptron Solving the XOR Problem Using Multi-layer Perceptron Architecture of MLP and Forward Pass Gradient Descent Backpropagation Implementation Conclusion Exercises Multiple-Choice Questions Theory Numerical References Chapter 4: Training Deep Networks Introduction Train–Test Split Train–Validation–Test Split K-Fold Split Batch, Stochastic, and Mini-batch Gradient Descent Batch Gradient Descent Stochastic Gradient Descent Mini-batch Gradient Descent RMSprop Adam Optimizer Conclusion Exercises Multiple-Choice Questions Theory Experiments References Chapter 5: Hyperparameter Tuning Introduction Bias–Variance Revisited Hyperparameter Tuning Experiments: Hyperparameter Tuning Conclusion Exercises Multiple-Choice Questions Experiments References Chapter 6: Convolutional Neural Networks: I Convolutional Layer Implementing Convolution Padding Stride and Other Layers Stride Pooling Normalization Fully Connected Layer Importance of Kernels Architecture of LeNet Conclusion Exercises Multiple-Choice Questions Numerical Applications Chapter 7: Convolutional Neural Network: II Sequential Model Creating the Model Adding Layers in the Model Removing the Last Layer from the Model Initializing Weights Summary Keras Layers 1. Dense Layer 2. Conv2D Layer 3. Pooling 4. Activations 4.1 Softmax 4.2 ReLU 5. Initializing Weights 6. Miscellaneous MNIST Dataset Classification Using LeNet: Prerequisite LeNet Structure Implementation AlexNet Some More Architectures GoogLeNet ResNet DenseNet Conclusion Exercises Multiple-Choice Questions Implementations References Chapter 8: Transfer Learning Introduction Idea VGG 16 and VGG 19 for Binary Classification Types and Strategies Limitations and Applications of Transfer Learning Conclusion Exercises Multiple-Choice Questions Application References Chapter 9: Recurrent Neural Network Introduction Why Neural Networks Cannot Infer Sequences Idea Backpropagation Through Time Types of RNN Applications Sentiment Classification Parts of Speech Tagging Handwritten Text Recognition Speech to Text Conclusion Exercises Multiple-Choice Questions Theory Image Captioning References Chapter 10: Gated Recurrent Unit and Long Short-Term Memory Introduction GRU Long Short-Term Memory Named Entity Recognition Sentiment Classification Conclusion Exercises Multiple-Choice Questions Theory Application-Based Questions References Chapter 11: Autoencoders Introduction Concept and Types The Math Types of Autoencoders Under-complete Autoencoder Over-complete Autoencoder Autoencoder and Principal Component Analysis Training of an Autoencoder Latent Representation Using Autoencoders Experiment 1 Experiment 2 Finding Latent Representation Using Multiple Layers Variants of Autoencoders Sparse Autoencoder Denoising Autoencoder Variational Autoencoder Conclusion Exercises Multiple-Choice Questions Theory Applications Chapter 12: Introduction to Generative Models Introduction Hopfield Networks Boltzmann Machines A Gentle Introduction to Transformers An Introduction to Self-Attention The Transformer Conclusion Exercise Multiple-Choice Questions Theory References Appendix A: Classifying The Simpsons Characters Appendix B: Face Detection Introduction Appendix C: Sentiment Classification Revisited Introduction Appendix D: Predicting Next Word Appendix E: COVID Classification Class Activation Layer Appendix F: Alzheimer's Classification Appendix G: Music Genre Classification Using MFCC and Convolutional Neural Network Dataset Feature Extraction Convolutional Neural Network Architecture Index