Statistical Modeling in Machine Learning : Concepts and Applications

Statistical Modeling in Machine Learning: Concepts and Applications presents the basic concepts and roles of statistics, exploratory data analysis and machine learning. The various aspects of Machine Learning are discussed along with basics of statistics. Concepts are presented with simple examples and graphical representation for better understanding of techniques. This book takes a holistic approach - putting key concepts together with an in-depth treatise on multi-disciplinary applications of machine learning. New case studies and research problem statements are discussed, which will help researchers in their application areas based on the concepts of statistics and machine learning. Statistical Modeling in Machine Learning: Concepts and Applications will help statisticians, machine learning practitioners and programmers solving various tasks such as classification, regression, clustering, forecasting, recommending and more. Provides a comprehensive overview of the state-of-the-art in statistical concepts applied to Machine Learning with the help of real-life problems, applications and tutorials Presents a step-by-step approach from fundamentals to advanced techniques Includes Case Studies with both successful and unsuccessful applications of Machine Learning to understand challenges in its implementation, along with worked examples Front Cover Statistical Modeling in Machine Learning Statistical Modeling in Machine Learning Copyright Dedication Contents Contributors Editors' biographies Preface Acknowledgments 1 - Introduction to statistical modeling in machine learning: a case study 1.1 Introduction 1.1.1 Machine-learning research in early age 1.1.2 Ensemble machine-learning technique 1.2 Classification of algorithms in machine learning 1.3 Regression algorithms in machine learning 1.4 Case study: prison crowding prediction 1.4.1 Methods and material 1.4.2 Data collection 1.4.3 Data preprocessing 1.4.4 Proposed regression-based prison overcrowding prediction model (RBPOPM) 1.5 Result and discussion 1.6 Conclusion References 2 - A technique of data collection: web scraping with python 2.1 Introduction 2.2 Basics of web scraping 2.2.1 Definition 2.2.2 Why do we need to scrape data? 2.2.3 Choice of programming language 2.2.4 Ethics behind web scraping 2.3 Elements of web scraping 2.3.1 Architecture of web scraping 2.3.2 Components of web scraping 2.4 An implementation walkthrough 2.4.1 Libraries for web scraping 2.4.1.1 Scrapy 2.4.1.2 Selenium 2.4.1.3 Beautiful Soup 2.4.2 Importing required libraries 2.4.3 Accessing website data 2.4.3.1 Output 2.4.4 Web crawling 2.4.5 Data extraction 2.4.5.1 Output 2.4.6 Data framing 2.4.6.1 Output 2.4.7 Stages of data transformation 2.5 Web scraping in reality 2.6 Conclusion References 3 - Analysis of Covid-19 using machine learning techniques 3.1 Introduction 3.2 Literature survey 3.3 Study of algorithms 3.3.1 Multiple regression 3.3.1.1 Ordinary least squares (OLS) 3.3.1.2 Multiple regression analysis has two main uses 3.3.1.3 Collinearity 3.3.1.4 Hypothesis test 3.3.1.5 Feature selection 3.3.1.6 Coefficient of determination 3.3.2 Logistic regression 3.3.3 Support vector machine 3.3.4 Building a decision tree 3.3.5 Random forest regressor 3.4 Experimental analysis and results 3.4.1 Analysis for multiple linear regression [] 3.4.1.1 Select the input variable and output variable 3.4.1.2 Splitting the data set 3.4.1.3 Build and train the predictor 3.4.1.4 Evaluate performance 3.4.1.5 Results of the hypothesis test 3.4.1.6 Understanding the outputs of the model from the above OLS regression results: is this statistically significant? 3.4.1.6.1 What is the significance of p-value? 3.4.1.6.2 R squared and adjusted R squared? 3.4.1.6.3 Checking for errors 3.4.1.7 Forward selection 3.4.1.8 Visualization plots for each symptom against the target variable 3.4.1.8.1 Difficulty in breathing symptom versus infected 3.4.1.8.2 High fever symptom versus infected 3.4.1.8.3 Dry Cough versus infected 3.4.1.8.4 Sore throat versus infected 3.4.2 Evaluation of machine learning algorithms 3.5 Conclusion and future study References Further reading 4 - Discriminative dictionary learning based on statistical methods 4.1 Introduction 4.1.1 Regularization and dimension reduction 4.1.2 Sparse representation (SR) 4.2 Notation 4.3 Sparse coding methods 4.3.1 Importance of statistical concepts in sparse coding methods 4.4 Dictionary learning 4.4.1 Orthogonal dictionary learning 4.4.2 Overcomplete dictionary learning 4.4.3 Structured dictionary learning 4.4.4 Unsupervised DL algorithms 4.4.5 Supervised DL algorithms 4.5 Statistical concepts in dictionary learning 4.5.1 Histogram of oriented gradients (HoG) 4.5.2 Use of correlation analysis in dictionary learning 4.6 Parametric approaches to estimation of dictionary parameters 4.6.1 Hidden Markov model (HMM): discriminative dictionary learning 4.7 Nonparametric approaches to discriminative DL 4.7.1 UHTelPCC 4.7.2 MNIST 4.8 Conclusion References 5 - Artificial intelligence–based uncertainty quantification technique for external flow computational fluid dynamic (CFD) ... 5.1 Introduction 5.2 Formulation 5.2.1 Governing equations and model for compressible flow over missile 5.2.2 Evolutionary neural architecture Search strategy 5.2.3 Determination of sample size for training the ANN 5.3 Results and discussions 5.4 Conclusions Acknowledgments References 6 - Contrast between simple and complex classification algorithms 6.1 Introduction 6.2 Data preprocessing and feature extraction 6.2.1 Data preprocessing 6.2.2 Feature study 6.2.2.1 Chroma 6.2.2.2 Root-mean-square (RMS) 6.2.2.3 Centroid 6.2.2.4 Bandwidth 6.2.2.5 Zero-crossing rate 6.2.2.6 Roll-off 6.2.2.7 Tempo 6.2.2.8 Mel-frequency cepstral coefficients 6.2.2.9 Harmony 6.3 Data modeling 6.3.1 Fitting linear discriminant analysis 6.3.2 Fitting quadratic discriminant analysis 6.3.3 Fitting k-nearest neighbors 6.3.4 Feedforward neural networks 6.3.5 Fitting feedforward neural networks 6.3.5.1 Gradient descent 6.3.5.2 Activation functions 6.3.5.3 Regularization 6.3.5.4 Model building 6.4 Conclusion References 7 - Classification model of machine learning for medical data analysis 7.1 Introduction 7.2 Machine learning techniques for diseases detection 7.2.1 Logistic regression (LR) 7.2.2 Decision tree 7.2.2.1 Build our decision tree 7.2.2.2 Advantage 7.2.2.3 Disadvantage 7.2.3 Random forest 7.2.4 Support vector machine 7.2.5 Radial basis function neural network (RBFNN) 7.2.6 Deep learning 7.3 Disease detected by machine learning techniques 7.3.1 Glaucoma and diabetic retinopathy 7.3.2 Brain tumor 7.3.3 Breast cancer 7.3.4 Heart disease 7.3.5 Multimodal classification 7.4 Challenges in ML based classification for medical data 7.4.1 Data 7.4.2 Selection of algorithm 7.4.3 Overfitting 7.4.4 Underfitting 7.5 Conclusion References 8 - Regression tasks for machine learning 8.1 Introduction 8.2 Steps in statistical modeling 8.2.1 Regression models and classification models 8.2.2 Regression model 8.2.3 Classification model 8.3 General linear regression model 8.4 Simple linear regression (SLR) 8.4.1 Estimate the regression parameters 8.5 Authentication of the simple linear regression model 8.5.1 Squared R 8.5.2 Interpretation of squared R 8.5.3 Hypothesis tests to the regression coefficients and p values 8.5.4 Inclusion/exclusion of explanatory variable decision 8.6 Multiple linear regression 8.7 Polynomial regression 8.8 Implementation using R programming 8.9 Conclusion References 9 - Model selection and regularization 9.1 Introduction 9.2 Subset selection 9.2.1 Best subset selection 9.2.2 Stepwise selection 9.3 Regularization 9.4 Shrinkage methods 9.4.1 Ridge Regression 9.4.1.1 Bias, mean square error, and L2-Risk of RRE 9.4.1.2 Graphical representation of RRE 9.4.2 Lasso Regression 9.4.2.1 Computation of the Lasso solution 9.5 Dimensional reduction 9.5.1 How many principal components? 9.6 Implementation of Ridge and Lasso Regression 9.7 Conclusion References 10 - Data clustering using unsupervised machine learning 10.1 Introduction 10.2 Techniques in unsupervised learning 10.2.1 Issues with unsupervised learning 10.2.2 Why is unsupervised learning needed despite these issues? 10.2.3 Applications of unsupervised learning 10.3 Unsupervised clustering 10.3.1 Hierarchical clustering 10.3.2 Partitional clustering 10.3.3 Latent variable models for clustering 10.3.4 Dimensionality reduction 10.3.5 The search-based clustering approaches 10.3.6 Bayesian clustering 10.3.7 Spectral clustering 10.4 Taxonomy of neural network-based deep clustering 10.4.1 Autoencoder (AE) based deep clustering 10.4.2 CDNN based deep clustering 10.4.3 Variational AE-based deep clustering 10.4.4 GAN-based deep clustering 10.5 Cluster evolution criteria 10.5.1 Similarity measurements 10.5.1.1 Lp and L1 based distance measurements 10.5.2 Internal quality criteria 10.5.2.1 Sum of squared error (SSE) 10.5.2.2 Scatter criteria 10.5.2.3 Condorcet's criterion 10.5.3 External quality standards 10.5.4 Clustering loss 10.5.5 Nonclustering loss 10.6 Applications of clustering 10.7 Feature selection with ML for clustering 10.7.1 Unsupervised filter model 10.7.2 Unsupervised wrapper model 10.7.3 Challenges 10.8 Classification in ML: challenges and research issues 10.9 Key findings and open challenges 10.10 Conclusion References 11 - Emotion-based classification through fuzzy entropy-enhanced FCM clustering 11.1 Introduction 11.2 Related work 11.3 Emotion-based models 11.3.1 Ekman's emotions 11.3.2 The Plutchik's main emotions 11.3.3 The Hourglass of emotion 11.4 Theoretical background 11.4.1 Fuzzy entropy and fuzziness 11.4.2 FCM and weighted FCM algorithms 11.4.3 Entropy-based FCM algorithm 11.5 Logical design model 11.6 Experimental results 11.7 Conclusion References 12 - Fundamental optimization methods for machine learning 12.1 Introduction 12.2 First-order optimization methods 12.2.1 Gradient descent 12.2.2 Gradient descent variants 12.2.2.1 Batch gradient descent 12.2.2.2 Stochastic gradient descent 12.2.2.3 Mini-batch gradient descent 12.2.3 Gradient descent optimization algorithms 12.2.3.1 Momentum 12.2.3.2 Nesterov accelerated gradient 12.2.3.3 AdaGrad 12.2.3.4 AdaDelta 12.2.3.5 Adaptive moment estimation 12.2.3.6 AdaMax 12.2.3.7 Nesterov accelerated adaptive moment estimation 12.3 High-order optimization method 12.3.1 Hessian-free Newton method 12.3.2 Quasi-Newton method 12.3.3 Gauss-Newton method 12.3.4 Natural gradient method 12.4 Derivative-free optimization methods 12.4.1 Methods for convex objective 12.4.2 Methods for stochastic optimization 12.5 Optimization methods challenges and issues in machine learning 12.6 Conclusion References 13 - Stochastic optimization of industrial grinding operation through data-driven robust optimization 13.1 Introduction 13.2 Optimization under uncertainty 13.2.1 Brief overview of robust optimization 13.2.2 Uncertainty set: a paramount element during calculations of statistical terms in RO 13.2.3 Description of FCM: unsupervised ML approach 13.2.4 Issues in conventional FCM approach 13.3 DDRO: data-driven robust optimization for grinding model 13.3.1 Industrial grinding model: description 13.3.2 Formulation of optimization problem under uncertainty problem: grinding model 13.3.3 ANN assisted fuzzy C-means clustering technique 13.3.4 Generative modeling framework in the identified clusters 13.4 Results and discussions 13.5 Conclusion Acknowledgments References 14 - Dimensionality reduction using PCAs in feature partitioning framework 14.1 Introduction 14.2 Principal component analysis (PCA) 14.3 PCAs in feature partitioning framework 14.3.1 What is feature partitioning framework? 14.3.2 Subpattern principal component analysis (SpPCA) 14.3.3 Cross-correlation subpattern principal component analysis (SubXPCA) 14.3.4 Hybrid principal component analysis (HPCA) 14.3.4.1 Procedure to extract hybrid PCA features from 2D face images Local pattern feature extraction Global pattern feature extraction Hybrid pattern feature extraction using ESpPCA Hybrid pattern feature extraction using ESubXPCA 14.3.5 Pattern reconstruction 14.3.6 Theoretical analysis 14.3.6.1 Summarization of variance 14.3.6.2 Time complexity analysis 14.3.6.3 Space complexity analysis 14.4 Summary Acknowledgments References 15 - Impact of Midday Meal Scheme in primary schools in India using exploratory data analysis and data visualization 15.1 Introduction and background 15.2 Nutrition in primary schools in rural India 15.2.1 Malnutrition 15.3 Midday Meal Scheme 15.3.1Objectives 15.4 Exploratory data analysis and visualization methodology 15.4.1 Stacked-bar plot 15.4.1.1 Mean percent of anemia in non-MDM versus MDM states 15.4.1.2 Mean percent of Stunt-Growth in non-MDM versus MDM states 15.4.1.3 Mean percent of enrollment in non-MDM and MDM states 15.4.2 Box plot 15.4.2.1 Quartiles and outliers of anemia in non-MDM and MDM states 15.4.2.2 Quartiles and outliers of Stunt-Growth in non-MDM and MDM states 15.4.2.3 Quartiles and outliers of enrollment in non-MDM and MDM states 15.4.3 Violin plot 15.4.3.1 Density plot distribution of anemia in non-MDM and MDM states 15.4.3.2 Density plot distribution of Stunt-Growth in non-MDM and MDM states 15.4.3.3 Density plot distribution of enrollment in non-MDM and MDM states 15.4.4 Scatter plot 15.4.4.1 Prevalence of anemia for all MDM states in the year 2005–2006 (in India) 15.4.4.2 Prevalence of anemia for all MDM states in 2015–2016 (in India) 15.4.4.3 Percentage of Stunt-Growth for all MDM states in the year 2016–2018 (in India) 15.4.4.4 Percentage of enrollment for all MDM states in the year 2005–2006 (in India) 15.4.4.5 Percentage of enrollment for all MDM states in 2015–2016 (in India) 15.4.5 Heatmap 15.5 Data visualization insights on impact of MDM 15.6 Conclusion References 16 - Nonlinear system identification of environmental pollutants using recurrent neural networks and Global Sensitivity Ana ... 16.1 Introduction 16.2 Formulation 16.2.1 Environmental pollutants data 16.2.2 Algorithm for design of RNNs 16.2.3 Global Sensitivity Analysis 16.3 Results and discussions 16.4 Conclusions Acknowledgments References 17 - Comparative study of automated deep learning techniques for wind time-series forecasting 17.1 Introduction 17.2 Formulation 17.2.1 Data description and analysis 17.2.2 Techniques for modeling the data 17.2.2.1 ADAM optimizer 17.2.2.2 Recurrent Neural Networks 17.2.2.3 Long short-term memory networks 17.2.3 Design of optimal networks 17.3 Results 17.3.1 Time-series analysis and decomposition 17.3.2 Optimal design of RNNs and LSTMs 17.3.3 Significance of predictions in wind farm studies 17.4 Conclusions Acknowledgments References Index A B C D E F G H I J K L M N O P Q R S T U V W Back Cover