یادگیری عمیق و محیط محاسبات موازی برای سیستم‌های مهندسی زیستی

Deep Learning and Parallel Computing Environment for Bioengineering Systems delivers a significant forum for the technical advancement of deep learning in parallel computing environment across bio-engineering diversified domains and its applications. Pursuing an interdisciplinary approach, it focuses on methods used to identify and acquire valid, potentially useful knowledge sources. Managing the gathered knowledge and applying it to multiple domains including health care, social networks, mining, recommendation systems, image processing, pattern recognition and predictions using deep learning paradigms is the major strength of this book. This book integrates the core ideas of deep learning and its applications in bio engineering application domains, to be accessible to all scholars and academicians. The proposed techniques and concepts in this book can be extended in future to accommodate changing business organizations' needs as well as practitioners' innovative ideas. Presents novel, in-depth research contributions from a methodological/application perspective in understanding the fusion of deep machine learning paradigms and their capabilities in solving a diverse range of problems Illustrates the state-of-the-art and recent developments in the new theories and applications of deep learning approaches applied to parallel computing environment in bioengineering systems Provides concepts and technologies that are successfully used in the implementation of today's intelligent data-centric critical systems and multi-media Cloud-Big data Cover Deep Learning and Parallel Computing Environment for Bioengineering Systems Copyright List of Contributors Preface Organization of the Book Audience Foreword Acknowledgment Contents 1 Parallel Computing, Graphics Processing Unit (GPU) and New Hardware for Deep Learning in Computational Intelligence Research 1.1 Introduction 1.1.1 Machine and Deep Learning 1.1.2 Graphics Processing Unit (GPU) 1.1.3 Computational Intelligence 1.1.4 GPU, Deep Learning and Computational Intelligence 1.2 Deep Learning and Parallelization 1.2.1 Parallel Processing Concepts 1.2.2 Deep Learning Using Parallel Algorithms 1.2.2.1 Understanding the Needs and Benefits of Parallel Algorithms in Deep Learning 1.2.2.2 Challenges in Implementing Parallel Algorithms in Deep Learning 1.2.3 Parallelization Methods to Distribute Computation Across Multiple Machines 1.2.3.1 Local Training 1.2.3.2 Distributed Training 1.2.4 Methods to Train Deep Neural Networks Using Parallelization 1.2.4.1 Inter-Model Parallelism 1.2.4.2 Data Parallelism 1.2.4.3 Intra-Model Parallelism 1.2.5 Parallelization Over Data, Function, Parameter and Prediction Scale 1.2.5.1 Data Partitioning 1.2.5.2 Function Partitioning 1.2.5.3 Hyperparameter Learning 1.2.5.4 Prediction at Scale 1.2.6 Types of Speed-Up and Scaling 1.2.6.1 Problem Constrained (PC) Scaling 1.2.6.2 Time Constrained (TC) Scaling 1.3 Role of Graphics Processing Unit in Parallel Deep Learning 1.3.1 Hardware Architecture of CPU and GPU 1.3.1.1 Conventional CPU Architecture 1.3.1.2 Modern GPU Architecture 1.3.2 Suitability of GPU to Parallel Deep Learning 1.3.3 CPU vs. GPU 1.3.4 Advantages of Using GPU in Parallel Deep Learning 1.3.5 Disadvantages of Using GPU in Parallel Deep Learning 1.3.6 Famous GPUs on the Market 1.3.6.1 NVIDIA 1.3.6.2 AMD 1.4 GPU Based Parallel Deep Learning on Computational Intelligence Applications With Case Study 1.4.1 Dataflow of the Deep Parallelized Training and Testing of Computational Intelligence Applications 1.4.2 Numerical Example for a Generic Computational Intelligence Application 1.4.3 Dealing With Limited GPU Memory 1.4.4 Computational Intelligence Applications 1.5 Implementation Screenshots to Visualize the Training Process 1.6 Summary References 2 Big Data Analytics and Deep Learning in Bioinformatics With Hadoop 2.1 Introduction 2.2 From Big Data to Knowledge Discovery With Hadoop 2.2.1 Hadoop Big Data Framework 2.2.2 Big Data Collection and Ingestion 2.2.2.1 Apache Sqoop (SQL-to-Hadoop) 2.2.2.2 Apache Flume 2.2.2.3 Apache Kafka 2.2.3 Data Staging and Storage on Hadoop 2.2.4 Data Processing and Analysis Frameworks 2.2.4.1 Batch Processing Only Framework - MapReduce 2.2.4.2 Stream Processing Only Framework 2.2.4.3 Hybrid Processing Framework 2.2.5 Big Data Analysis and Visualization 2.3 Machine Learning for Big Data Analysis 2.3.1 Machine Learning Methods 2.3.1.1 Supervised Machine Learning 2.3.1.2 Unsupervised Machine Learning 2.3.1.3 Semi-Supervised Machine Learning 2.3.1.4 Reinforcement Learning 2.3.2 Deep Learning and Neural Networks 2.3.2.1 Artificial Neural Networks (ANNs) 2.3.3 Machine Learning and Hadoop 2.3.3.1 Spark, Mahout and MLlib 2.3.4 Distributed Deep Learning and Hadoop 2.4 Conclusions References 3 Image Fusion Through Deep Convolutional Neural Network 3.1 Introduction 3.2 Image Fusion 3.3 Registration 3.3.1 Image Registration Stages 3.3.1.1 Feature Detection 3.3.1.2 Feature Matching 3.3.1.3 Transform Model Estimation 3.3.1.4 Resampling 3.3.2 Need for Image Registration in Medical Imaging 3.3.2.1 SURF Based Registration 3.3.2.2 BRISK Based Registration 3.3.2.3 Implementation 3.4 Existing Image Fusion Methods - Overview 3.5 Deep Learning 3.6 Convolutional Neural Network (CNN) 3.7 CNN Based Image Fusion Algorithms 3.7.1 Deep Stacked CNN (DSCNN) 3.7.2 CNN Based Similarity Learning 3.7.3 CNN Based Fusion Using Pyramidal Decomposition 3.8 Evaluation Metrics 3.8.1 Entropy (S) 3.8.2 Standard Deviation (σ) 3.8.3 Spatial Frequency (SF) 3.8.4 Mutual Information (MI) 3.8.5 Image Quality Index (IQI) 3.8.6 QRS/F 3.8.7 QG 3.8.8 Structural Similarity Index Metric (SSIM) 3.8.9 FSIM 3.8.10 Contrast (C) 3.8.11 QE 3.8.12 Average Gradient (AG) 3.8.13 Human Perception-Based Metric (QCB) 3.8.14 Processing Time (T) 3.9 Results Interpretation and Discussion 3.10 Issues in Existing CNN Based Image Fusion Methods 3.11 Conclusions References 4 Medical Imaging With Intelligent Systems: A Review 4.1 Introduction 4.2 Tumor Types and Grading 4.2.1 Why Are Studies Concentrated Mostly on Gliomas? 4.2.2 Grading 4.2.3 Symptoms of a Brain Tumor 4.2.4 Diagnosis and Treatment of a Brain Tumor 4.3 Imaging Techniques 4.3.1 Reading an MR Image 4.3.2 Advanced Magnetic Resonance Imaging 4.3.2.1 MRS 4.3.2.2 Ultra-High-Field 7 T MRI 4.4 Machine Learning (ML) - Supervised and Unsupervised Methods 4.4.1 ML Software Packages/Toolboxes 4.5 Deep Learning (DL) 4.5.1 DL Tools/Libraries 4.6 Evaluation and Validation Metrics 4.7 Embedding Into Clinics 4.8 Current State-of-the-Art 4.8.1 Deep Learning Concepts for Brain Tumor Grading 4.9 Discussion 4.10 Conclusions Short Authors Biographies Acknowledgments Funding References 5 Medical Image Analysis With Deep Neural Networks 5.1 Introduction 5.2 Convolutional Neural Networks 5.3 Convolutional Neural Network Methods 5.4 Convolutional Layer 5.4.1 Tiled Convolution 5.4.2 Transposed Convolution 5.4.3 Dilated Convolution 5.4.4 Network-in-Network 5.4.5 Inception Module 5.5 Pooling Layer 5.5.1 Lp Pooling 5.5.2 Mixed Pooling 5.5.3 Stochastic Pooling 5.5.4 Spectral Pooling 5.5.5 Spatial Pyramid Pooling 5.5.6 Multi-Scale Orderless Pooling 5.6 Activation Function 5.6.1 Rectified Linear Unit (ReLU) 5.6.2 Leaky ReLU 5.6.3 Parametric ReLU 5.6.4 Randomized ReLU 5.6.5 Exponential Linear Unit (ELU) 5.6.6 Maxout 5.6.7 Probout 5.7 Applications of CNN in Medical Image Analysis 5.7.1 Image Classification 5.7.2 Object Classification 5.7.3 Region, Organ, and Landmark Localization 5.7.4 Object or Lesion Detection 5.7.5 Organ and Substructure Segmentation 5.7.6 Lesion Segmentation 5.8 Discussion 5.9 Conclusions References 6 Deep Convolutional Neural Network for Image Classification on CUDA Platform 6.1 Introduction 6.2 Image Classification 6.2.1 Image Classification Approach 6.2.2 Image Classification Techniques 6.2.3 Research Gaps 6.2.4 Research Challenge 6.2.5 Problem Definition 6.2.6 Objective 6.3 Deep Convolutional Neuron Network 6.4 Compute Unified Device Architecture (CUDA) 6.5 TensorFlow 6.6 Implementation 6.6.1 Deep Convolutional Neural Networks 6.6.2 Dataset 6.6.3 Implementing an Image Classifier 6.6.4 Installation and System Requirements 6.6.5 Algorithms 6.7 Result Analysis 6.7.1 Neural Networks in TensorFlow 6.7.2 Understanding the Original Image Dataset 6.7.3 Understanding the Original Labels 6.7.4 Implementing Pre-process Functions 6.7.5 Output of the Model 6.7.6 Training a Model Using Multiple GPU Cards/CUDA 6.8 Conclusions References 7 Efficient Deep Learning Approaches for Health Informatics 7.1 Introduction Machine Learning Vs. Deep Learning 7.2 Deep Learning Approaches Deep Autoencoders Recurrent Neural Networks (RNNs) Restricted Boltzmann Machine (RBM) Deep Belief Network Deep Boltzmann Machine (DBM) Convolutional Neural Network 7.3 Applications Translational Bioinformatics Clinical Imaging Electronic Health Records Genomics Mobile Devices 7.4 Challenges and Limitations 7.5 Conclusions References 8 Deep Learning and Semi-Supervised and Transfer Learning Algorithms for Medical Imaging 8.1 Introduction 8.2 Image Acquisition in the Medical Field 8.3 Deep Learning Over Machine Learning 8.4 Neural Network Architecture 8.5 Defining Deep Learning 8.6 Deep Learning Architecture 8.6.1 Convolution Neural Network (CNN) 8.6.2 Recurrent Neural Network 8.6.3 Deep Neural Network 8.7 Deep Learning in Medical Imaging 8.7 8.7.1 Diabetic Retinopathy 8.7.2 Cardiac Imaging 8.7.3 Tumor Classification in Homogeneous Breast Tissue 8.8 Developments in Deep Learning Methods 8.8.1 Black Box and Deep Learning 8.8.2 Semi-Supervised and Transfer Learning Algorithms 8.8.2.1 Semi-Supervised Learning 8.8.2.2 Supervised Learning 8.8.2.3 Unsupervised Learning 8.8.3 Applications of Semi-Supervised Learning in Medical Imaging Pelvic MR Image Segmentation Based on Multi-Task Residual Fully Convolutional Networks 8.8.4 Method 8.9 Transfer Learning 8.9.1 Transfer Learning in Image Data 8.9.1.1 Mechanisms of Deep Transfer Learning for Medical Imaging 8.9.1.2 Dataset and Training 8.9.1.3 Transferred Learned Features 8.9.1.4 Traditional Texture Feature Results 8.9.2 Transfer Learning Technique for the Detection of Breast Cancer 8.9.2.1 Dataset 8.9.2.2 Preprocessing 8.9.2.3 Transfer Learning Part 8.9.2.4 Results References 9 Survey on Evaluating the Performance of Machine Learning Algorithms: Past Contributions and Future Roadmap 9.1 Introduction 9.2 Methodology 9.3 Linear Regression 9.4 Nonlinear Regression 9.4.1 Support Vector Machine 9.4.2 K-Nearest Neighbors 9.4.3 Neural Network 9.5 Nonlinear Decision Tree Regression 9.5.1 Regression With Decision Trees 9.5.2 Random Forest 9.6 Linear Classification in R 9.7 Results and Discussion 9.8 Conclusions References 10 Miracle of Deep Learning Using IoT 10.1 Introduction 10.2 Inspiration 10.3 Decisions in an Area of Deep Learning and IoT 10.3.1 IoT Reference Model 10.3.2 Rudiments of Machine Learning 10.3.3 Algorithms for Efficient Training 10.3.4 Secure Deep Learning 10.3.5 Robust and Resolution-Invariant Image Classification 10.3.6 Planning With Flawless and Noisy Images 10.3.7 Smart and Fast Data Processing 10.4 Simulation Results and Performance Analysis of Handwritten Digits Recognition in IoT 10.5 An Intelligent Traffic Load Prediction 10.6 Performance of Deep Learning Based Channel Assignment 10.6.1 A Deep Learning System for the Individual Pursuit 10.6.2 Network Architecture 10.6.3 Dataset 10.6.4 Statistics 10.6.5 Effectiveness of Online Instance Matching 10.7 Discussion 10.8 Conclusion References 11 Challenges in Storing and Processing Big Data Using Hadoop and Spark 11.1 Introduction 11.2 Background and Main Focus 11.2.1 Challenges of Big Data Technologies 11.2.1.1 Memory Issues 11.2.1.2 Limitations of Apache Big Data Technologies 11.2.2 Real Time Applications of Big Data Frameworks 11.3 Hadoop Architecture 11.4 MapReduce Architecture 11.5 Joins in MapReduce 11.6 Apache Storm 11.7 Apache Spark Environment 11.7.1 Use Cases of Big Data Technologies 11.8 Graph Analysis With GraphX 11.9 Streaming Data Analytics 11.10 Futer Research Directions 11.11 Conclusion References 12 An Efficient Biography-Based Optimization Algorithm to Solve the Location Routing Problem With Intermediate Depots for Multiple Perishable Products 12.1 Introduction 12.2 Model Development 12.2.1 Mathematical Model 12.2.1.1 Linearization of the Nonlinear Equations 12.2.2 An Illustration 12.3 Solution Method 12.3.1 Introduction to Biography Based Optimization Algorithm 12.3.1.1 Migration Operator 12.3.1.2 Mutation Operator 12.3.2 Solution Representation 12.3.3 Initial Solution Generation 12.3.4 Immigration Phase 12.3.5 Mutation Phase 12.3.6 Optimal Parameter Design 12.4 Computational Results 12.4.1 Optimal Solutions for Instance Problems 12.4.2 Sensitivity Analysis 12.5 Discussion, Concluding Remarks and Future Research Directions References 13 Evolutionary Mapping Techniques for Systolic Computing System 13.1 Introduction 13.2 Systolic Arrays 13.3 Evolutionary Algorithms 13.4 Swarm Intelligence (SI) 13.5 Mapping Techniques 13.6 Systolic Implementation of Texture Analysis 13.7 Results and Discussion 13.7.1 Performance of EA for F8 Optimization 13.7.2 Texture Analysis 13.8 Conclusions List of Acronyms and Abbreviations References 14 Varied Expression Analysis of Children With ASD Using Multimodal Deep Learning Technique 14.1 Introduction 14.2 State-of-the-Art 14.3 Methodology 14.3.1 Detection of Human Faces 14.3.1.1 Selection of Haar Features in Selected Faces 14.3.1.2 Construction of an Integral Image 14.3.1.3 AdaBoost Technique to Build Strong Classifiers 14.3.1.4 Cascade Classifier 14.3.2 Extraction of Features 14.3.3 Expression Classifier 14.3.4 Expression Identification Through a Convolution Neural Network (CNN) 14.3.4.1 Convolution Layer 14.3.4.2 Max Pool Layer 14.3.4.3 Fully Connected Layer 14.4 Results and Analysis 14.4.1 Accuracy of the Face Expression Analysis 14.5 Conclusions 14.6 Future Work References 15 Parallel Machine Learning and Deep Learning Approaches for Bioinformatics 15.1 Introduction 15.1.1 Machine Learning and Deep Learning 15.1.2 Role of Parallelization in Deep Learning 15.1.3 Deep Learning Applications in Bioinformatics 15.2 Deep Learning and Parallel Processing 15.2.1 Parallel Processing 15.2.2 Scalability of Parallelization Methods 15.2.3 Deep Learning Using Parallel Algorithms 15.2.3.1 Advantages of Using Parallelism in Deep Learning 15.2.3.2 Parallelism Challenges in Deep Learning 15.3 Deep Learning and Bioinformatics 15.3.1 Bioinformatics Applications 15.3.1.1 Sequence Analysis 15.3.1.2 Genome Annotation 15.3.1.3 Analysis of Gene Expression 15.3.1.4 Analysis of Protein Expression 15.3.1.5 Analysis of Mutations in Cancer 15.3.1.6 Protein Structure Prediction 15.3.1.7 Modeling Biological Systems 15.3.1.8 High-Throughput Image Analysis 15.3.1.9 Microarrays 15.3.1.10 Systems Biology 15.3.1.11 Text Mining 15.3.2 Advantages of Using Parallel Deep Learning in Bioinformatics Applications 15.3.3 Challenges in Using Parallel Deep Learning for Bioinformatics Applications 15.4 Parallel Deep Learning in Bioinformatics Applications With Implementation and Real Time Numerical Example 15.5 Sample Implementation Screenshots to Visualize the Training Process 15.6 Summary References Index Back Cover __Deep Learning and Parallel Computing Environment for Bioengineering Systems__ delivers a significant forum for the technical advancement of deep learning in parallel computing environment across bio-engineering diversified domains and its applications. Pursuing an interdisciplinary approach, it focuses on methods used to identify and acquire valid, potentially useful knowledge sources. Managing the gathered knowledge and applying it to multiple domains including health care, social networks, mining, recommendation systems, image processing, pattern recognition and predictions using deep learning paradigms is the major strength of this book. This book integrates the core ideas of deep learning and its applications in bio engineering application domains, to be accessible to all scholars and academicians. The proposed techniques and concepts in this book can be extended in future to accommodate changing business organizations' needs as well as practitioners' innovative ideas.