Python Machine Learning By Example_Fourth Edition

Author Yuxi (Hayden) Liu teaches machine learning from the fundamentals to building NLP transformers and multimodal models with best practice tips and real-world examples using PyTorch, TensorFlow, scikit-learn, and pandas Key Features Discover new and updated content on NLP transformers, PyTorch, and computer vision modeling Includes a dedicated chapter on best practices and additional best practice tips throughout the book to improve your ML solutions Implement ML models, such as neural networks and linear and logistic regression, from scratch Book Description The fourth edition of Python Machine Learning By Example is a comprehensive guide for beginners and experienced machine learning practitioners who want to learn more advanced techniques, such as multimodal modeling. Written by experienced machine learning author and ex-Google machine learning engineer Yuxi (Hayden) Liu, this edition emphasizes best practices, providing invaluable insights for machine learning engineers, data scientists, and analysts. Explore advanced techniques, including two new chapters on natural language processing transformers with BERT and GPT, and multimodal computer vision models with PyTorch and Hugging Face. You’ll learn key modeling techniques using practical examples, such as predicting stock prices and creating an image search engine. This hands-on machine learning book navigates through complex challenges, bridging the gap between theoretical understanding and practical application. Elevate your machine learning and deep learning expertise, tackle intricate problems, and unlock the potential of advanced techniques in machine learning with this authoritative guide. What you will learn Follow machine learning best practices throughout data preparation and model development Build and improve image classifiers using convolutional neural networks (CNNs) and transfer learning Develop and fine-tune neural networks using TensorFlow and PyTorch Analyze sequence data and make predictions using recurrent neural networks (RNNs), transformers, and CLIP Build classifiers using support vector machines (SVMs) and boost performance with PCA Avoid overfitting using regularization, feature selection, and more Who this book is for This expanded fourth edition is ideal for data scientists, ML engineers, analysts, and students with Python programming knowledge. The real-world examples, best practices, and code prepare anyone undertaking their first serious ML project. Preface Who this book is for What this book covers To get the most out of this book Get in touch Getting Started with Machine Learning and Python An introduction to machine learning Understanding why we need machine learning Differentiating between machine learning and automation Machine learning applications Knowing the prerequisites Getting started with three types of machine learning A brief history of the development of machine learning algorithms Digging into the core of machine learning Generalizing with data Overfitting, underfitting, and the bias-variance trade-off Overfitting Underfitting The bias-variance trade-off Avoiding overfitting with cross-validation Avoiding overfitting with regularization Avoiding overfitting with feature selection and dimensionality reduction Data preprocessing and feature engineering Preprocessing and exploration Dealing with missing values Label encoding One-hot encoding Dense embedding Scaling Feature engineering Polynomial transformation Binning Combining models Voting and averaging Bagging Boosting Stacking Installing software and setting up Setting up Python and environments Installing the main Python packages NumPy SciPy pandas scikit-learn TensorFlow PyTorch Summary Exercises Join our book’s Discord space Building a Movie Recommendation Engine with Naïve Bayes Getting started with classification Binary classification Multiclass classification Multi-label classification Exploring Naïve Bayes Bayes’ theorem by example The mechanics of Naïve Bayes Implementing Naïve Bayes Implementing Naïve Bayes from scratch Implementing Naïve Bayes with scikit-learn Building a movie recommender with Naïve Bayes Preparing the data Training a Naïve Bayes model Evaluating classification performance Tuning models with cross-validation Summary Exercises References Join our book’s Discord space Predicting Online Ad Click-Through with Tree-Based Algorithms A brief overview of ad click-through prediction Getting started with two types of data – numerical and categorical Exploring a decision tree from the root to the leaves Constructing a decision tree The metrics for measuring a split Gini Impurity Information Gain Implementing a decision tree from scratch Implementing a decision tree with scikit-learn Predicting ad click-through with a decision tree Ensembling decision trees – random forests Ensembling decision trees – gradient-boosted trees Summary Exercises Join our book’s Discord space Predicting Online Ad Click-Through with Logistic Regression Converting categorical features to numerical – one-hot encoding and ordinal encoding Classifying data with logistic regression Getting started with the logistic function Jumping from the logistic function to logistic regression Training a logistic regression model Training a logistic regression model using gradient descent Predicting ad click-through with logistic regression using gradient descent Training a logistic regression model using stochastic gradient descent (SGD) Training a logistic regression model with regularization Feature selection using L1 regularization Feature selection using random forest Training on large datasets with online learning Handling multiclass classification Implementing logistic regression using TensorFlow Summary Exercises Join our book’s Discord space Predicting Stock Prices with Regression Algorithms What is regression? Mining stock price data A brief overview of the stock market and stock prices Getting started with feature engineering Acquiring data and generating features Estimating with linear regression How does linear regression work? Implementing linear regression from scratch Implementing linear regression with scikit-learn Implementing linear regression with TensorFlow Estimating with decision tree regression Transitioning from classification trees to regression trees Implementing decision tree regression Implementing a regression forest Evaluating regression performance Predicting stock prices with the three regression algorithms Summary Exercises Join our book’s Discord space Predicting Stock Prices with Artificial Neural Networks Demystifying neural networks Starting with a single-layer neural network Layers in neural networks Activation functions Backpropagation Adding more layers to a neural network: DL Building neural networks Implementing neural networks from scratch Implementing neural networks with scikit-learn Implementing neural networks with TensorFlow Implementing neural networks with PyTorch Picking the right activation functions Preventing overfitting in neural networks Dropout Early stopping Predicting stock prices with neural networks Training a simple neural network Fine-tuning the neural network Summary Exercises Join our book’s Discord space Mining the 20 Newsgroups Dataset with Text Analysis Techniques How computers understand language – NLP What is NLP? The history of NLP NLP applications Touring popular NLP libraries and picking up NLP basics Installing famous NLP libraries Corpora Tokenization PoS tagging NER Stemming and lemmatization Semantics and topic modeling Getting the newsgroups data Exploring the newsgroups data Thinking about features for text data Counting the occurrence of each word token Text preprocessing Dropping stop words Reducing inflectional and derivational forms of words Visualizing the newsgroups data with t-SNE What is dimensionality reduction? t-SNE for dimensionality reduction Representing words with dense vectors – word embedding Building embedding models using shallow neural networks Utilizing pre-trained embedding models Summary Exercises Join our book’s Discord space Discovering Underlying Topics in the Newsgroups Dataset with Clustering and Topic Modeling Learning without guidance – unsupervised learning Getting started with k-means clustering How does k-means clustering work? Implementing k-means from scratch Implementing k-means with scikit-learn Choosing the value of k Clustering newsgroups dataset Clustering newsgroups data using k-means Describing the clusters using GPT Discovering underlying topics in newsgroups Topic modeling using NMF Topic modeling using LDA Summary Exercises Join our book’s Discord space Recognizing Faces with Support Vector Machine Finding the separating boundary with SVM Scenario 1 – identifying a separating hyperplane Scenario 2 – determining the optimal hyperplane Scenario 3 – handling outliers Implementing SVM Scenario 4 – dealing with more than two classes One-vs-rest One-vs-one Multiclass cases in scikit-learn Scenario 5 – solving linearly non-separable problems with kernels Choosing between linear and RBF kernels Classifying face images with SVM Exploring the face image dataset Building an SVM-based image classifier Boosting image classification performance with PCA Estimating with support vector regression Implementing SVR Summary Exercises Join our book’s Discord space Machine Learning Best Practices Machine learning solution workflow Best practices in the data preparation stage Best practice 1 – Completely understanding the project goal Best practice 2 – Collecting all fields that are relevant Best practice 3 – Maintaining the consistency and normalization of field values Best practice 4 – Dealing with missing data Best practice 5 – Storing large-scale data Best practices in the training set generation stage Best practice 6 – Identifying categorical features with numerical values Best practice 7 – Deciding whether to encode categorical features Best practice 8 – Deciding whether to select features and, if so, how to do so Best practice 9 – Deciding whether to reduce dimensionality and, if so, how to do so Best practice 10 – Deciding whether to rescale features Best practice 11 – Performing feature engineering with domain expertise Best practice 12 – Performing feature engineering without domain expertise Binarization and discretization Interaction Polynomial transformation Best practice 13 – Documenting how each feature is generated Best practice 14 – Extracting features from text data tf and tf-idf Word embedding Word2Vec embedding Best practices in the model training, evaluation, and selection stage Best practice 15 – Choosing the right algorithm(s) to start with Naïve Bayes Logistic regression SVM Random forest (or decision tree) Neural networks Best practice 16 – Reducing overfitting Best practice 17 – Diagnosing overfitting and underfitting Best practice 18 – Modeling on large-scale datasets Best practices in the deployment and monitoring stage Best practice 19 – Saving, loading, and reusing models Saving and restoring models using pickle Saving and restoring models in TensorFlow Saving and restoring models in PyTorch Best practice 20 – Monitoring model performance Best practice 21 – Updating models regularly Summary Exercises Join our book’s Discord space Categorizing Images of Clothing with Convolutional Neural Networks Getting started with CNN building blocks The convolutional layer The non-linear layer The pooling layer Architecting a CNN for classification Exploring the clothing image dataset Classifying clothing images with CNNs Architecting the CNN model Fitting the CNN model Visualizing the convolutional filters Boosting the CNN classifier with data augmentation Flipping for data augmentation Rotation for data augmentation Cropping for data augmentation Improving the clothing image classifier with data augmentation Advancing the CNN classifier with transfer learning Development of CNN architectures and pretrained models Improving the clothing image classifier by fine-tuning ResNets Summary Exercises Join our book’s Discord space Making Predictions with Sequences Using Recurrent Neural Networks Introducing sequential learning Learning the RNN architecture by example Recurrent mechanism Many-to-one RNNs One-to-many RNNs Many-to-many (synced) RNNs Many-to-many (unsynced) RNNs Training an RNN model Overcoming long-term dependencies with LSTM Analyzing movie review sentiment with RNNs Analyzing and preprocessing the data Building a simple LSTM network Stacking multiple LSTM layers Revisiting stock price forecasting with LSTM Writing your own War and Peace with RNNs Acquiring and analyzing the training data Constructing the training set for the RNN text generator Building and training an RNN text generator Summary Exercises Join our book’s Discord space Advancing Language Understanding and Generation with the Transformer Models Understanding self-attention Key, value, and query representations Attention score calculation and embedding vector generation Multi-head attention Exploring the Transformer’s architecture The encoder-decoder structure Positional encoding Layer normalization Improving sentiment analysis with BERT and Transformers Pre-training BERT MLM NSP Fine-tuning of BERT Fine-tuning a pre-trained BERT model for sentiment analysis Using the Trainer API to train Transformer models Generating text using GPT Pre-training of GPT and autoregressive generation Writing your own version of War and Peace with GPT Summary Exercises Join our book’s Discord space Building an Image Search Engine Using CLIP: a Multimodal Approach Introducing the CLIP model Understanding the mechanism of the CLIP model Vision encoder Text encoder Contrastive learning Exploring applications of the CLIP model Zero-shot image classification Zero-shot text classification Image and text retrieval Image and text generation Transfer learning Getting started with the dataset Obtaining the Flickr8k dataset Loading the Flickr8k dataset Architecting the CLIP model Vision encoder Text encoder Projection head for contrastive learning CLIP model Finding images with words Training a CLIP model Obtaining embeddings for images and text to identify matches Image search using the pre-trained CLIP model Zero-shot classification Summary Exercises References Join our book’s Discord space Making Decisions in Complex Environments with Reinforcement Learning Setting up the working environment Introducing OpenAI Gym and Gymnasium Installing Gymnasium Introducing reinforcement learning with examples Elements of reinforcement learning Cumulative rewards Approaches to reinforcement learning Policy-based approach Value-based approach Solving the FrozenLake environment with dynamic programming Simulating the FrozenLake environment Solving FrozenLake with the value iteration algorithm Solving FrozenLake with the policy iteration algorithm Performing Monte Carlo learning Simulating the Blackjack environment Performing Monte Carlo policy evaluation Performing on-policy Monte Carlo control Solving the Blackjack problem with the Q-learning algorithm Introducing the Q-learning algorithm Developing the Q-learning algorithm Summary Exercises Join our book’s Discord space Index Learn machine learning (ML) with this hands-on guide from best-selling author and ex-Google ML engineer Yuxi (Hayden) Liu. He teaches the basics of ML algorithms to NLP transformers and multimodal models with best practice tips and real-world examples Key FeaturesNew and updated content on NLP transformers, PyTorch, and computer vision modelingBest practices have expanded beyond one chapter with tips to improve your ML solutions showcased throughout the bookImplement ML algorithms, such as neural networks and decision trees from scratchBook DescriptionThe fourth edition of Python Machine Learning by Example is a comprehensive guide for beginners and experienced ML practitioners who want to learn more advanced techniques like multimodal modeling. Written by best-selling author and ex-Google ML engineer Yuxi (Hayden) Liu, this edition emphasizes best practices, providing invaluable insights for ML engineers, data scientists, and analysts. Explore advanced techniques, including two new chapters on NLP transformers with BERT and GPT-4 and multimodal computer vision models with PyTorch and Hugging Face. Youll learn advanced modeling techniques using practical examples, such as predicting stock prices and creating an image search engine. This book navigates through complex challenges, bridging the gap between theoretical understanding and practical application. Elevate your ML expertise, tackle intricate problems, and unlock the potential of advanced techniques in machine learning with this authoritative guide. What you will learnFollow machine learning best practices for data preparation, model training, and evaluationBuild and improve image classifiers using CNNs, transfer learning, and data augmentationBuild and fine-tune neural networks using TensorFlow and PyTorch for stock price prediction and image searchAnalyze sequence data and make predictions using RNNs and transformersBuild classifiers using SVMs and boost performance with principal component analysisLearn to avoid overfitting using cross-validation, regularization, feature selection, and dimensionality reductionWho this book is forThis expanded fourth edition is ideal for data scientists, ML engineers, analysts, and students with Python knowledge. The real-world lessons and code prepare anyone undertaking their first serious ML project. Table of ContentsGetting Started with Machine Learning and PythonBuilding a Movie Recommendation EnginePredicting Online Ad Click-Through with Tree-Based AlgorithmsPredicting Online Ad Click-Through with Logistic RegressionPredicting Stock Prices with Regression AlgorithmsPredicting Stock Prices with Artificial Neural NetworksMining the 20 Newsgroups Dataset with Text Analysis TechniquesDiscovering Underlying Topics in the Newsgroups Dataset with Clustering and Topic ModelingRecognizing Faces with Support Vector MachineMachine Learning Best PracticesCategorizing Images of Clothing with Convolutional Neural NetworksMaking Predictions with Sequences Using Recurrent Neural NetworksAdvancing Language Understanding and Generation with Transformer ModelsBuilding An Image Search Engine Using Multimodal ModelsMaking Decisions in Complex Environments with Reinforcement Learning Author Yuxi (Hayden) Liu teaches machine learning from the fundamentals to building NLP transformers and multimodal models with best practice tips and real-world examples using PyTorch, TensorFlow, scikit-learn, and pandasKey FeaturesDiscover new and updated content on NLP transformers, PyTorch, and computer vision modelingIncludes a dedicated chapter on best practices and additional best practice tips throughout the book to improve your ML solutionsImplement ML models, such as neural networks and linear and logistic regression, from scratchPurchase of the print or Kindle book includes a free PDF copyBook DescriptionThe fourth edition of Python Machine Learning By Example is a comprehensive guide for beginners and experienced machine learning practitioners who want to learn more advanced techniques, such as multimodal modeling. Written by experienced machine learning author and ex-Google machine learning engineer Yuxi (Hayden) Liu, this edition emphasizes best practices, providing invaluable insights for machine learning engineers, data scientists, and analysts. Explore advanced techniques, including two new chapters on natural language processing transformers with BERT and GPT, and multimodal computer vision models with PyTorch and Hugging Face. You'll learn key modeling techniques using practical examples, such as predicting stock prices and creating an image search engine. This hands-on machine learning book navigates through complex challenges, bridging the gap between theoretical understanding and practical application. Elevate your machine learning and deep learning expertise, tackle intricate problems, and unlock the potential of advanced techniques in machine learning with this authoritative guide.What you will learnFollow machine learning best practices throughout data preparation and model developmentBuild and improve image classifiers using convolutional neural networks (CNNs) and transfer learningDevelop and fine-tune neural networks using TensorFlow and PyTorchAnalyze sequence data and make predictions using recurrent neural networks (RNNs), transformers, and CLIPBuild classifiers using support vector machines (SVMs) and boost performance with PCAAvoid overfitting using regularization, feature selection, and moreWho this book is forThis expanded fourth edition is ideal for data scientists, ML engineers, analysts, and students with Python programming knowledge. The real-world examples, best practices, and code prepare anyone undertaking their first serious ML project.