Deep Learning : A Beginners' Guide

This book focuses on deep learning (DL), which is an important aspect of data science, that includes predictive modeling. DL applications are widely used in domains such as finance, transport, healthcare, automanufacturing, and advertising. The design of the DL models based on artificial neural networks is influenced by the structure and operation of the brain. This book presents a comprehensive resource for those who seek a solid grasp of the techniques in DL. Key features: Provides knowledge on theory and design of state-of-the-art deep learning models for real-world applications Explains the concepts and terminology in problem-solving with deep learning Explores the theoretical basis for major algorithms and approaches in deep learning Discusses the enhancement techniques of deep learning models Identifies the performance evaluation techniques for deep learning models Accordingly, the book covers the entire process flow of deep learning by providing awareness of each of the widely used models. This book can be used as a beginners’ guide where the user can understand the associated concepts and techniques. This book will be a useful resource for undergraduate and postgraduate students, engineers, and researchers, who are starting to learn the subject of deep learning. This book focuses on deep learning (DL), which is an important aspect of data science, that includes predictive modeling. DL applications are widely used in domains such as finance, transport, healthcare, automanufacturing, and advertising. The design of the DL models based on artificial neural networks is influenced by the structure and operation of the brain. This book presents a comprehensive resource for those who seek a solid grasp of the techniques in DL.Accordingly, the book covers the entire process flow of deep learning by providing awareness of each of the widely used models. This book can be used as a beginners' guide where the user can understand the associated concepts and techniques. This book will be a useful resource for undergraduate and postgraduate students, engineers, and researchers, who are starting to learn the subject of deep learning.Key features: Provides knowledge on theory and design of state-of-the-art deep learning models for real-world applications.Explains the concepts and terminology in problem-solving with deep learning.Explores the theoretical basis for major algorithms and approaches in deep learning.Discusses the enhancement techniques of deep learning models.Identifies the performance evaluation techniques for deep learning models. Cover 1 Half Title 2 Title Page 4 Copyright Page 5 Table of Contents 6 Preface 10 Acknowledgements 12 Abbreviations 14 1 Introduction 16 1.1 Data-Driven Decision-Making and Society 16 1.2 Overview of Deep Learning 17 1.3 Bias and Variance 21 1.3.1 Skewness of Data 22 1.3.2 Bias 22 1.3.3 Variance 23 1.3.4 Trade-Off Between Bias and Variance 23 1.4 Supervised and Unsupervised Learning 26 1.5 Supportive Tools and Libraries 27 1.5.1 TensorFlow 28 1.5.2 Keras 28 1.5.3 PyTorch 29 1.5.4 Jupyter Notebook 29 1.5.5 NumPy and Pandas 29 1.5.6 Tensor Hub 29 Review Questions 30 2 Concepts and Terminology 31 2.1 Understanding Neural Networks 31 2.2 Regression 33 2.2.1 Linear Regression 34 2.2.2 Logistic Regression 35 2.2.3 Other Regression Methods 35 2.3 Classification 36 2.4 Hyperparameters 37 2.4.1 Overview 37 2.4.2 Weight Initialization 37 2.4.3 Activation Function 39 2.4.4 Learning Rate 44 2.4.5 Loss Function 44 2.4.6 Other Hyperparameters 47 2.5 Model Training 48 2.5.1 Model Selection 48 2.5.2 Model Convergence 48 2.5.3 Overfitting and Underfitting 49 2.5.4 Regularization 52 2.5.5 Network Gradients 53 Review Questions 56 3 State-Of-The-Art Deep Learning Models: Part I 57 3.1 Overview of Neural Networks 57 3.2 Artificial Neural Networks 58 3.3 Recurrent Neural Network (RNN) 60 3.4 Convolutional Neural Networks 63 3.4.1 Overview of Convolutional Neural Network 63 3.4.2 Concepts of CNN 64 3.4.3 Convolutional Layer 67 3.4.4 Pooling Layer 69 3.4.5 Fully Connected Layer 70 3.5 Comparison of ANN, RNN, and CNN 71 Review Questions 73 4 State-Of-The-Art Deep Learning Models: Part II 74 4.1 Feed-Forward Neural Network 74 4.2 Multi-Layer Perceptrons 76 4.3 Generative Adversarial Network (GAN) 77 4.4 Variations of CNNs 79 4.4.1 Residual Networks (ResNet) 79 4.4.2 Inception Model 81 4.4.3 GoogLeNet 82 4.4.4 Xception Model 84 4.4.5 DenseNet Model 84 4.4.6 MobileNet Model 85 4.4.7 VGG Model 86 4.4.8 Comparison of CNN Architectures 86 4.5 Capsule Network 88 4.6 Autoencoders 91 4.7 Transformers 93 Review Questions 98 5 Advanced Learning Techniques 99 5.1 Transfer Learning 99 5.1.1 Overview of Transfer Learning 99 5.1.2 Transfer Learning Process 100 5.1.3 Transfer Learning Types, Categories, and Strategies 102 5.1.4 Transfer Learning Applications 104 5.1.5 Transfer Learning Challenges 105 5.2 Reinforcement Learning 106 5.2.1 Overview of Reinforcement Learning 106 5.2.2 Reinforcement Learning Process 106 5.2.3 Implementation and Scheduling Types 109 5.2.4 Applications of Reinforcement Learning 110 5.2.5 Challenges of Reinforcement Learning 110 5.3 Federated Learning 111 5.3.1 Overview of Federated Learning 111 5.3.2 Federated Learning Process 112 5.3.3 Types and Properties of Federated Learning 115 5.3.4 Applications of Federated Learning 116 5.3.5 Challenges of Federated Learning 117 5.4 Multi-Modeling With Ensemble Learning 118 5.4.1 Overview of Ensemble Learning 118 5.4.2 Ensemble Learning Process 118 5.4.3 Ensemble Learning Techniques 121 5.4.4 Applications of Ensemble Learning 125 Review Questions 125 6 Enhancement of Deep Learning Architectures 127 6.1 Model Performance Improvement 127 6.2 Regularization 130 6.3 Augmentation 134 6.4 Normalization 135 6.5 Hyperparameter Tuning 138 6.6 Model Optimization 140 6.6.1 Overview of Model Optimization 140 6.6.2 Gradient-Based Optimization Algorithms 142 6.6.3 Other Optimization Algorithms 145 6.7 Neural Architecture Search (NAS) 147 6.7.1 Overview of NAS 147 6.7.2 NAS Process 148 6.7.3 Search Space 149 6.7.4 Search Strategies of NAS 151 6.7.5 Strategies for Performance Measures 154 6.8 Adversarial Training 155 6.8.1 Overview of Adversarial Training 155 6.8.2 Types of Adversarial Attacks 156 6.8.3 Adversarial Attack Generation Techniques 157 6.8.4 Adversarial Attack Defensive Methods 159 6.8.5 Best Practices to Avoid Adversarial Attacks 160 Review Questions 160 7 Performance Evaluation Techniques 162 7.1 Overview of Performance Measures 162 7.2 Types of Performance Metrics 163 7.2.1 Confusion Matrix 163 7.2.2 Accuracy 163 7.2.3 Precision and Recall 165 7.2.4 F-Measure 166 7.2.5 Specificity and Sensitivity 167 7.2.6 Receiving Operating Characteristic Curve (ROC) 167 7.2.7 Area Under the ROC Curve (AUROC) and AUC 168 7.2.8 Cross-Validation 168 7.2.9 Kappa Score 172 7.2.10 Grad-CAM Heat Map 172 7.2.11 Metrics for Imbalanced Datasets 173 7.2.12 Metrics for Regression Problems 174 7.2.13 Summary of Performance Metrics 178 Review Questions 178 Appendix – Frequently Asked Questions 180 References 188 Index 196 transfer,learning;,ensemble,learning;,adversarial,training;,deep,learning;,hyperparameters;,reinforcement,learning;,federated,learning;,model,optimization;,performance,evaluation;,neural,networks transfer learning,ensemble learning,adversarial training,deep learning,hyperparameters,reinforcement learning,federated learning,model optimization,performance evaluation,neural networks