Pro Machine Learning Algorithms : A Hands-On Approach to Implementing Algorithms in Python and R

Chapter 1: Basics of Machine Learning -- Chapter 2: Linear regression -- Chapter 3: Logistic regression -- Chapter 4: Decision tree -- Chapter 5: Random forest -- Chapter 6: GBM -- Chapter 7: Neural network -- Chapter 8: word2vec -- Chapter 9: Convolutional neural network -- Chapter 10: Recurrent Neural Network -- Chapter 11: Clustering -- Chapter 12: PCA -- Chapter 13: Recommender systems -- Chapter 14: Implementing algorithms in the cloud.;Bridge the gap between a high-level understanding of how an algorithm works and knowing the nuts and bolts to tune your models better. This book will give you the confidence and skills when developing all the major machine learning models. In Pro Machine Learning Algorithms, you will first develop the algorithm in Excel so that you get a practical understanding of all the levers that can be tuned in a model, before implementing the models in Python/R. You will cover all the major algorithms: supervised and unsupervised learning, which include linear/logistic regression; k-means clustering; PCA; recommender system; decision tree; random forest; GBM; and neural networks. You will also be exposed to the latest in deep learning through CNNs, RNNs, and word2vec for text mining. You will be learning not only the algorithms, but also the concepts of feature engineering to maximize the performance of a model. You will see the theory along with case studies, such as sentiment classification, fraud detection, recommender systems, and image recognition, so that you get the best of both theory and practice for the vast majority of the machine learning algorithms used in industry. Along with learning the algorithms, you will also be exposed to running machine-learning models on all the major cloud service providers. You are expected to have minimal knowledge of statistics/software programming and by the end of this book you should be able to work on a machine learning project with confidence. You will: Get an in-depth understanding of all the major machine learning and deep learning algorithms Fully appreciate the pitfalls to avoid while building models Implement machine learning algorithms in the cloud Follow a hands-on approach through case studies for each algorithm Gain the tricks of ensemble learning to build more accurate models Discover the basics of programming in R/Python and the Keras framework for deep learning. Table of Contents About the Author About the Technical Reviewer Acknowledgments Introduction Chapter 1: Basics of Machine Learning Regression and Classification Training and Testing Data The Need for Validation Dataset Measures of Accuracy Absolute Error Root Mean Square Error Confusion Matrix AUC Value and ROC Curve Unsupervised Learning Typical Approach Towards Building a Model Where Is the Data Fetched From? Which Data Needs to Be Fetched? Pre-processing the Data Feature Interaction Feature Generation Building the Models Productionalizing the Models Build, Deploy, Test, and Iterate Summary Chapter 2: Linear Regression Introducing Linear Regression Variables: Dependent and Independent Correlation Causation Simple vs. Multivariate Linear Regression Formalizing Simple Linear Regression The Bias Term The Slope Solving a Simple Linear Regression More General Way of Solving a Simple Linear Regression Minimizing the Overall Sum of Squared Error Solving the Formula Working Details of Simple Linear Regression Complicating Simple Linear Regression a Little Arriving at Optimal Coefficient Values Introducing Root Mean Squared Error Running a Simple Linear Regression in R Residuals Coefficients SSE of Residuals (Residual Deviance) Null Deviance R Squared F-statistic Running a Simple Linear Regression in Python Common Pitfalls of Simple Linear Regression Multivariate Linear Regression Working details of Multivariate Linear Regression Multivariate Linear Regression in R Multivariate Linear Regression in Python Issue of Having a Non-significant Variable in the Model Issue of Multicollinearity Mathematical Intuition of Multicollinearity Further Points to Consider in Multivariate Linear Regression Assumptions of Linear Regression Summary Chapter 3: Logistic Regression Why Does Linear Regression Fail for Discrete Outcomes? A More General Solution: Sigmoid Curve Formalizing the Sigmoid Curve (Sigmoid Activation) From Sigmoid Curve to Logistic Regression Interpreting the Logistic Regression Working Details of Logistic Regression Estimating Error Scenario 1 Scenario 2 Least Squares Method and Assumption of Linearity Running a Logistic Regression in R Running a Logistic Regression in Python Identifying the Measure of Interest Common Pitfalls Time Between Prediction and the Event Happening Outliers in Independent variables Summary Chapter 4: Decision Tree Components of a Decision Tree Classification Decision Tree When There Are Multiple Discrete Independent Variables Information Gain Calculating Uncertainty: Entropy Calculating Information Gain Uncertainty in the Original Dataset Measuring the Improvement in Uncertainty Which Distinct Values Go to the Left and Right Nodes Gini Impurity Splitting Sub-nodes Further When Does the Splitting Process Stop? Classification Decision Tree for Continuous Independent Variables Classification Decision Tree When There Are Multiple Independent Variables Classification Decision Tree When There Are Continuous and Discrete Independent Variables What If the Response Variable Is Continuous? Continuous Dependent Variable and Multiple Continuous Independent Variables Continuous Dependent Variable and Discrete Independent Variable Continuous Dependent Variable and Discrete, Continuous Independent Variables Implementing a Decision Tree in R Implementing a Decision Tree in Python Common Techniques in Tree Building Visualizing a Tree Build Impact of Outliers on Decision Trees Summary Chapter 5: Random Forest A Random Forest Scenario Bagging Working Details of a Random Forest Implementing a Random Forest in R Parameters to Tune in a Random Forest Variation of AUC by Depth of Tree Implementing a Random Forest in Python Summary Chapter 6: Gradient Boosting Machine Gradient Boosting Machine Working details of GBM Shrinkage AdaBoost Theory of AdaBoost Working Details of AdaBoost Additional Functionality for GBM Implementing GBM in Python Implementing GBM in R Summary Chapter 7: Artificial Neural Network Structure of a Neural Network Working Details of Training a Neural Network Forward Propagation Applying the Activation Function Back Propagation Working Out Back Propagation Stochastic Gradient Descent Diving Deep into Gradient Descent Why Have a Learning Rate? Batch Training The Concept of Softmax Different Loss Optimization Functions Scaling a Dataset Scenario Without Scaling the Input Scenario with Input Scaling Implementing Neural Network in Python Avoiding Over-fitting using Regularization Assigning Weightage to Regularization term Implementing Neural Network in R Summary Chapter 8: Word2vec Hand-Building a Word Vector Methods of Building a Word Vector Issues to Watch For in a Word2vec Model Frequent Words Negative Sampling Implementing Word2vec in Python Summary Chapter 9: Convolutional Neural Network The Problem with Traditional NN Scenario 1 Scenario 2 Scenario 3 Scenario 4 Understanding the Convolutional in CNN From Convolution to Activation From Convolution Activation to Pooling How Do Convolution and Pooling Help? Creating CNNs with Code Working Details of CNN Deep Diving into Convolutions/Kernels From Convolution and Pooling to Flattening: Fully Connected Layer From One Fully Connected Layer to Another From Fully Connected Layer to Output Layer Connecting the Dots: Feed Forward Network Other Details of CNN Backward Propagation in CNN Putting It All Together Data Augmentation Implementing CNN in R Summary Chapter 10: Recurrent Neural Network Understanding the Architecture Interpreting an RNN Working Details of RNN Time Step 1 Time Step 2 Time Step 3 Implementing RNN: SimpleRNN Compiling a Model Verifying the Output of RNN Implementing RNN: Text Generation Embedding Layer in RNN Issues with Traditional RNN The Problem of Vanishing Gradient The Problem of Exploding Gradients LSTM Implementing Basic LSTM in keras Implementing LSTM for Sentiment Classification Implementing RNN in R Summary Chapter 11: Clustering Intuition of clustering Building Store Clusters for Performance Comparison Ideal Clustering Striking a Balance Between No Clustering and Too Much Clustering: K-means Clustering The Process of Clustering Working Details of K-means Clustering Algorithm Applying the K-means Algorithm on a Dataset Properties of the K-means Clustering Algorithm Totss (Total Sum of Squares) Cluster Centers Tot.withinss Betweenss Implementing K-means Clustering in R Implementing K-means Clustering in Python Significance of the Major Metrics Identifying the Optimal K Top-Down Vs. Bottom-Up Clustering Hierarchical Clustering Major Drawback of Hierarchical Clustering Industry Use-Case of K-means Clustering Summary Chapter 12: Principal Component Analysis Intuition of PCA Working Details of PCA Scaling Data in PCA Extending PCA to Multiple Variables Implementing PCA in R Implementing PCA in Python Applying PCA to MNIST Summary Chapter 13: Recommender Systems Understanding k-nearest Neighbors Working Details of User-Based Collaborative Filtering Euclidian Distance Normalizing for a User Issue with Considering a Single User Cosine Similarity Weighted Average Rating Calculation Choosing the Right Approach Calculating the Error Issues with UBCF Item-Based Collaborative Filtering Implementing Collaborative Filtering in R Implementing Collaborative Filtering in Python Working Details of Matrix Factorization Implementing Matrix Factorization in Python Implementing Matrix Factorization in R Summary Chapter 14: Implementing Algorithms in the Cloud Google Cloud Platform Microsoft Azure Cloud Platform Amazon Web Services Transferring Files to the Cloud Instance Running Instance Jupyter Notebooks from Your Local Machine Installing R on the Instance Summary Appendix: Basics of Excel, R, and Python Basics of Excel Basics of R Downloading R Installing and Configuring RStudio Getting Started with RStudio Basics of Python Downloading and installing Python Basic operations in Python Numpy Number generation using Numpy Slicing and indexing Pandas Indexing and slicing using Pandas Summarizing data Index Bridge the gap between a high-level understanding of how an algorithm works and knowing the nuts and bolts to tune your models better. This book will give you the confidence and skills when developing all the major machine learning models. In Pro Machine Learning Algorithms , you will first develop the algorithm in Excel so that you get a practical understanding of all the levers that can be tuned in a model, before implementing the models in Python/R. You will cover all the major algorithms: supervised and unsupervised learning, which include linear/logistic regression; k-means clustering; PCA; recommender system; decision tree; random forest; GBM; and neural networks. You will also be exposed to the latest in deep learning through CNNs, RNNs, and word2vec for text mining. You will be learning not only the algorithms, but also the concepts of feature engineering to maximize the performance of a model. You will see the theory along with case studies, such as sentiment classification, fraud detection, recommender systems, and image recognition, so that you get the best of both theory and practice for the vast majority of the machine learning algorithms used in industry. Along with learning the algorithms, you will also be exposed to running machine-learning models on all the major cloud service providers. You are expected to have minimal knowledge of statistics/software programming and by the end of this book you should be able to work on a machine learning project with confidence. What You Will Learn Get an in-depth understanding of all the major machine learning and deep learning algorithms Fully appreciate the pitfalls to avoid while building models Implement machine learning algorithms in the cloud Follow a hands-on approach through case studies for each algorithm Gain the tricks of ensemble learning to build more accurate models Discover the basics of programming in R/Python and the Keras framework for deep learning Who This Book Is For Business analysts/ IT professionals who want to transition into data science roles. Data scientists who want to solidify their knowledge in machine learning. Annotation Bridge the gap between a high-level understanding of how an algorithm works and knowing the nuts and bolts to tune your models better. This book will give you the confidence and skills when developing all the major machine learning models. In Pro Machine Learning Algorithms, you will first develop the algorithm in Excel so that you get a practical understanding of all the levers that can be tuned in a model, before implementing the models in Python/R. You will cover all the major algorithms: supervised and unsupervised learning, which include linear/logistic regression; k-means clustering; PCA; recommender system; decision tree; random forest; GBM; and neural networks. You will also be exposed to the latest in deep learning through CNNs, RNNs, and word2vec for text mining. You will be learning not only the algorithms, but also the concepts of feature engineering to maximize the performance of a model. You will see the theory along with case studies, such as sentiment classification, fraud detection, recommender systems, and image recognition, so that you get the best of both theory and practice for the vast majority of the machine learning algorithms used in industry. Along with learning the algorithms, you will also be exposed to running machine-learning models on all the major cloud service providers. You are expected to have minimal knowledge of statistics/software programming and by the end of this book you should be able to work on a machine learning project with confidence. What You Will LearnGet an in-depth understanding of all the major machine learning and deep learning algorithmsFully appreciate the pitfalls to avoid while building modelsImplement machine learning algorithms in the cloudFollow a hands-on approach through case studies for each algorithmGain the tricks of ensemble learning to build more accurate modelsDiscover the basics of programming in R/Python and the Keras framework for deep learningWho This Book Is ForBusiness analysts/ IT professionals who want to transition into data science roles. Data scientists who want to solidify their knowledge in machine learning