Where There's A Will

Through a recent series of breakthroughs, deep learning has boosted the entire field of machine learning. Now, even programmers who know close to nothing about this technology can use simple, efficient tools to implement programs capable of learning from data. This best-selling book uses concrete examples, minimal theory, and production-ready Python frameworks--scikit-learn, Keras, and TensorFlow--to help you gain an intuitive understanding of the concepts and tools for building intelligent systems. With this updated third edition, author Aurelien Geron explores a range of techniques, starting with simple linear regression and progressing to deep neural networks. Numerous code examples and exercises throughout the book help you apply what you've learned. Programming experience is all you need to get started. Use scikit-learn to track an example machine learning project end to end Explore several models, including support vector machines, decision trees, random forests, and ensemble methods Exploit unsupervised learning techniques such as dimensionality reduction, clustering, and anomaly detection Dive into neural net architectures, including convolutional nets, recurrent nets, generative adversarial networks, and transformers Use TensorFlow and Keras to build and train neural nets for computer vision, natural language processing, generative models, and deep reinforcement learning Train neural nets using multiple GPUs and deploy them at scale using Google's Vertex AI Copyright Table of Contents Preface The Machine Learning Tsunami Machine Learning in Your Projects Objective and Approach Code Examples Prerequisites Roadmap Changes Between the First and the Second Edition Changes Between the Second and the Third Edition Other Resources Conventions Used in This Book O’Reilly Online Learning How to Contact Us Acknowledgments Part I. The Fundamentals of Machine Learning Chapter 1. The Machine Learning Landscape What Is Machine Learning? Why Use Machine Learning? Examples of Applications Types of Machine Learning Systems Training Supervision Batch Versus Online Learning Instance-Based Versus Model-Based Learning Main Challenges of Machine Learning Insufficient Quantity of Training Data Nonrepresentative Training Data Poor-Quality Data Irrelevant Features Overfitting the Training Data Underfitting the Training Data Stepping Back Testing and Validating Hyperparameter Tuning and Model Selection Data Mismatch Exercises Chapter 2. End-to-End Machine Learning Project Working with Real Data Look at the Big Picture Frame the Problem Select a Performance Measure Check the Assumptions Get the Data Running the Code Examples Using Google Colab Saving Your Code Changes and Your Data The Power and Danger of Interactivity Book Code Versus Notebook Code Download the Data Take a Quick Look at the Data Structure Create a Test Set Explore and Visualize the Data to Gain Insights Visualizing Geographical Data Look for Correlations Experiment with Attribute Combinations Prepare the Data for Machine Learning Algorithms Clean the Data Handling Text and Categorical Attributes Feature Scaling and Transformation Custom Transformers Transformation Pipelines Select and Train a Model Train and Evaluate on the Training Set Better Evaluation Using Cross-Validation Fine-Tune Your Model Grid Search Randomized Search Ensemble Methods Analyzing the Best Models and Their Errors Evaluate Your System on the Test Set Launch, Monitor, and Maintain Your System Try It Out! Exercises Chapter 3. Classification MNIST Training a Binary Classifier Performance Measures Measuring Accuracy Using Cross-Validation Confusion Matrices Precision and Recall The Precision/Recall Trade-off The ROC Curve Multiclass Classification Error Analysis Multilabel Classification Multioutput Classification Exercises Chapter 4. Training Models Linear Regression The Normal Equation Computational Complexity Gradient Descent Batch Gradient Descent Stochastic Gradient Descent Mini-Batch Gradient Descent Polynomial Regression Learning Curves Regularized Linear Models Ridge Regression Lasso Regression Elastic Net Regression Early Stopping Logistic Regression Estimating Probabilities Training and Cost Function Decision Boundaries Softmax Regression Exercises Chapter 5. Support Vector Machines Linear SVM Classification Soft Margin Classification Nonlinear SVM Classification Polynomial Kernel Similarity Features Gaussian RBF Kernel SVM Classes and Computational Complexity SVM Regression Under the Hood of Linear SVM Classifiers The Dual Problem Kernelized SVMs Exercises Chapter 6. Decision Trees Training and Visualizing a Decision Tree Making Predictions Estimating Class Probabilities The CART Training Algorithm Computational Complexity Gini Impurity or Entropy? Regularization Hyperparameters Regression Sensitivity to Axis Orientation Decision Trees Have a High Variance Exercises Chapter 7. Ensemble Learning and Random Forests Voting Classifiers Bagging and Pasting Bagging and Pasting in Scikit-Learn Out-of-Bag Evaluation Random Patches and Random Subspaces Random Forests Extra-Trees Feature Importance Boosting AdaBoost Gradient Boosting Histogram-Based Gradient Boosting Stacking Exercises Chapter 8. Dimensionality Reduction The Curse of Dimensionality Main Approaches for Dimensionality Reduction Projection Manifold Learning PCA Preserving the Variance Principal Components Projecting Down to d Dimensions Using Scikit-Learn Explained Variance Ratio Choosing the Right Number of Dimensions PCA for Compression Randomized PCA Incremental PCA Random Projection LLE Other Dimensionality Reduction Techniques Exercises Chapter 9. Unsupervised Learning Techniques Clustering Algorithms: k-means and DBSCAN k-means Limits of k-means Using Clustering for Image Segmentation Using Clustering for Semi-Supervised Learning DBSCAN Other Clustering Algorithms Gaussian Mixtures Using Gaussian Mixtures for Anomaly Detection Selecting the Number of Clusters Bayesian Gaussian Mixture Models Other Algorithms for Anomaly and Novelty Detection Exercises Part II. Neural Networks and Deep Learning Chapter 10. Introduction to Artificial Neural Networks with Keras From Biological to Artificial Neurons Biological Neurons Logical Computations with Neurons The Perceptron The Multilayer Perceptron and Backpropagation Regression MLPs Classification MLPs Implementing MLPs with Keras Building an Image Classifier Using the Sequential API Building a Regression MLP Using the Sequential API Building Complex Models Using the Functional API Using the Subclassing API to Build Dynamic Models Saving and Restoring a Model Using Callbacks Using TensorBoard for Visualization Fine-Tuning Neural Network Hyperparameters Number of Hidden Layers Number of Neurons per Hidden Layer Learning Rate, Batch Size, and Other Hyperparameters Exercises Chapter 11. Training Deep Neural Networks The Vanishing/Exploding Gradients Problems Glorot and He Initialization Better Activation Functions Batch Normalization Gradient Clipping Reusing Pretrained Layers Transfer Learning with Keras Unsupervised Pretraining Pretraining on an Auxiliary Task Faster Optimizers Momentum Nesterov Accelerated Gradient AdaGrad RMSProp Adam AdaMax Nadam AdamW Learning Rate Scheduling Avoiding Overfitting Through Regularization l1 and l2 Regularization Dropout Monte Carlo (MC) Dropout Max-Norm Regularization Summary and Practical Guidelines Exercises Chapter 12. Custom Models and Training with TensorFlow A Quick Tour of TensorFlow Using TensorFlow like NumPy Tensors and Operations Tensors and NumPy Type Conversions Variables Other Data Structures Customizing Models and Training Algorithms Custom Loss Functions Saving and Loading Models That Contain Custom Components Custom Activation Functions, Initializers, Regularizers, and Constraints Custom Metrics Custom Layers Custom Models Losses and Metrics Based on Model Internals Computing Gradients Using Autodiff Custom Training Loops TensorFlow Functions and Graphs AutoGraph and Tracing TF Function Rules Exercises Chapter 13. Loading and Preprocessing Data with TensorFlow The tf.data API Chaining Transformations Shuffling the Data Interleaving Lines from Multiple Files Preprocessing the Data Putting Everything Together Prefetching Using the Dataset with Keras The TFRecord Format Compressed TFRecord Files A Brief Introduction to Protocol Buffers TensorFlow Protobufs Loading and Parsing Examples Handling Lists of Lists Using the SequenceExample Protobuf Keras Preprocessing Layers The Normalization Layer The Discretization Layer The CategoryEncoding Layer The StringLookup Layer The Hashing Layer Encoding Categorical Features Using Embeddings Text Preprocessing Using Pretrained Language Model Components Image Preprocessing Layers The TensorFlow Datasets Project Exercises Chapter 14. Deep Computer Vision Using Convolutional Neural Networks The Architecture of the Visual Cortex Convolutional Layers Filters Stacking Multiple Feature Maps Implementing Convolutional Layers with Keras Memory Requirements Pooling Layers Implementing Pooling Layers with Keras CNN Architectures LeNet-5 AlexNet GoogLeNet VGGNet ResNet Xception SENet Other Noteworthy Architectures Choosing the Right CNN Architecture Implementing a ResNet-34 CNN Using Keras Using Pretrained Models from Keras Pretrained Models for Transfer Learning Classification and Localization Object Detection Fully Convolutional Networks You Only Look Once Object Tracking Semantic Segmentation Exercises Chapter 15. Processing Sequences Using RNNs and CNNs Recurrent Neurons and Layers Memory Cells Input and Output Sequences Training RNNs Forecasting a Time Series The ARMA Model Family Preparing the Data for Machine Learning Models Forecasting Using a Linear Model Forecasting Using a Simple RNN Forecasting Using a Deep RNN Forecasting Multivariate Time Series Forecasting Several Time Steps Ahead Forecasting Using a Sequence-to-Sequence Model Handling Long Sequences Fighting the Unstable Gradients Problem Tackling the Short-Term Memory Problem Exercises Chapter 16. Natural Language Processing with RNNs and Attention Generating Shakespearean Text Using a Character RNN Creating the Training Dataset Building and Training the Char-RNN Model Generating Fake Shakespearean Text Stateful RNN Sentiment Analysis Masking Reusing Pretrained Embeddings and Language Models An Encoder–Decoder Network for Neural Machine Translation Bidirectional RNNs Beam Search Attention Mechanisms Attention Is All You Need: The Original Transformer Architecture An Avalanche of Transformer Models Vision Transformers Hugging Face’s Transformers Library Exercises Chapter 17. Autoencoders, GANs, and Diffusion Models Efficient Data Representations Performing PCA with an Undercomplete Linear Autoencoder Stacked Autoencoders Implementing a Stacked Autoencoder Using Keras Visualizing the Reconstructions Visualizing the Fashion MNIST Dataset Unsupervised Pretraining Using Stacked Autoencoders Tying Weights Training One Autoencoder at a Time Convolutional Autoencoders Denoising Autoencoders Sparse Autoencoders Variational Autoencoders Generating Fashion MNIST Images Generative Adversarial Networks The Difficulties of Training GANs Deep Convolutional GANs Progressive Growing of GANs StyleGANs Diffusion Models Exercises Chapter 18. Reinforcement Learning Learning to Optimize Rewards Policy Search Introduction to OpenAI Gym Neural Network Policies Evaluating Actions: The Credit Assignment Problem Policy Gradients Markov Decision Processes Temporal Difference Learning Q-Learning Exploration Policies Approximate Q-Learning and Deep Q-Learning Implementing Deep Q-Learning Deep Q-Learning Variants Fixed Q-value Targets Double DQN Prioritized Experience Replay Dueling DQN Overview of Some Popular RL Algorithms Exercises Chapter 19. Training and Deploying TensorFlow Models at Scale Serving a TensorFlow Model Using TensorFlow Serving Creating a Prediction Service on Vertex AI Running Batch Prediction Jobs on Vertex AI Deploying a Model to a Mobile or Embedded Device Running a Model in a Web Page Using GPUs to Speed Up Computations Getting Your Own GPU Managing the GPU RAM Placing Operations and Variables on Devices Parallel Execution Across Multiple Devices Training Models Across Multiple Devices Model Parallelism Data Parallelism Training at Scale Using the Distribution Strategies API Training a Model on a TensorFlow Cluster Running Large Training Jobs on Vertex AI Hyperparameter Tuning on Vertex AI Exercises Thank You! Appendix A. Machine Learning Project Checklist Frame the Problem and Look at the Big Picture Get the Data Explore the Data Prepare the Data Shortlist Promising Models Fine-Tune the System Present Your Solution Launch! Appendix B. Autodiff Manual Differentiation Finite Difference Approximation Forward-Mode Autodiff Reverse-Mode Autodiff Appendix C. Special Data Structures Strings Ragged Tensors Sparse Tensors Tensor Arrays Sets Queues Appendix D. TensorFlow Graphs TF Functions and Concrete Functions Exploring Function Definitions and Graphs A Closer Look at Tracing Using AutoGraph to Capture Control Flow Handling Variables and Other Resources in TF Functions Using TF Functions with Keras (or Not) Index About the Author Colophon Through a series of recent breakthroughs, deep learning has boosted the entire field of machine learning. Now, even programmers who know close to nothing about this technology can use simple, efficient tools to implement programs capable of learning from data. The updated edition of this best-selling book uses concrete examples, minimal theory, and two production-ready Python frameworks-Scikit-Learn and TensorFlow 2-to help you gain an intuitive understanding of the concepts and tools for building intelligent systems. Practitioners will learn a range of techniques that they can quickly put to use on the job. Part 1 employs Scikit-Learn to introduce fundamental machine learning tasks, such as simple linear regression. Part 2, which has been significantly updated, employs Keras and TensorFlow 2 to guide the reader through more advanced machine learning methods using deep neural networks. With exercises in each chapter to help you apply what you've learned, all you need is programming experience to get started. NEW FOR THE SECOND EDITION:Updated all code to TensorFlow 2 ; Introduced the high-level Keras API ; New and expanded coverage including TensorFlow's Data API, Eager Execution, Estimators API, deploying on Google Cloud ML, handling time series, embeddings and more With Early Release ebooks, you get books in their earliest form-the author's raw and unedited content as he or she writes-so you can take advantage of these technologies long before the official release of these titles. You'll also receive updates when significant changes are made, new chapters are available, and the final ebook bundle is released Through a recent series of breakthroughs, deep learning has boosted the entire field of machine learning. Now, even programmers who know close to nothing about this technology can use simple, efficient tools to implement programs capable of learning from data. This bestselling book uses concrete examples, minimal theory, and production-ready Python frameworks (Scikit-Learn, Keras, and TensorFlow) to help you gain an intuitive understanding of the concepts and tools for building intelligent systems.With this updated third edition, author Aurélien Géron explores a range of techniques, starting with simple linear regression and progressing to deep neural networks. Numerous code examples and exercises throughout the book help you apply what you've learned. Programming experience is all you need to get started.Use Scikit-learn to track an example ML project end to endExplore several models, including support vector machines, decision trees, random forests, and ensemble methodsExploit unsupervised learning techniques such as dimensionality reduction, clustering, and anomaly detectionDive into neural net architectures, including convolutional nets, recurrent nets, generative adversarial networks, autoencoders, diffusion models, and transformersUse TensorFlow and Keras to build and train neural nets for computer vision, natural language processing, generative models, and deep reinforcement learning Descripción del editor: "Through a series of recent breakthroughs, deep learning has boosted the entire field of machine learning. Now, even programmers who know close to nothing about this technology can use simple, efficient tools to implement programs capable of learning from data. This practical book shows you how.By using concrete examples, minimal theory, and two production-ready Python frameworks--Scikit-Learn and TensorFlow--author Aurélien Géron helps you gain an intuitive understanding of the concepts and tools for building intelligent systems. You'll learn a range of techniques, starting with simple linear regression and progressing to deep neural networks. With exercises in each chapter to help you apply what you've learned, all you need is programming experience to get started.Explore the machine learning landscape, particularly neural netsUse Scikit-Learn to track an example machine-learning project end-to-endExplore several training models, including support vector machines, decision trees, random forests, and ensemble methodsUse the TensorFlow library to build and train neural netsDive into neural net architectures, including convolutional nets, recurrent nets, and deep reinforcement learningLearn techniques for training and scaling deep neural nets" (O' Reilly) Through a series of recent breakthroughs, deep learning has boosted the entire field of machine learning. Now, even programmers who know close to nothing about this technology can use simple, efficient tools to implement programs capable of learning from data. The updated edition of this best-selling book uses concrete examples, minimal theory, and two production-ready Python frameworks--Scikit-Learn and TensorFlow 2--to help you gain an intuitive understanding of the concepts and tools for building intelligent systems. Practitioners will learn a range of techniques that they can quickly put to use on the job. Part 1 employs Scikit-Learn to introduce fundamental machine learning tasks, such as simple linear regression. Part 2, which has been significantly updated, employs Keras and TensorFlow 2 to guide the reader through more advanced machine learning methods using deep neural networks. With exercises in each chapter to help you apply what you've learned, all you need is programming experience to get started. NEW FOR THE SECOND EDITION: Updated all code to TensorFlow 2Introduced the high-level Keras APINew and expanded coverage including TensorFlow's Data API, Eager Execution, Estimators API, deploying on Google Cloud ML, handling time series, embeddings and more.