Allen JEE Mains 2021 Physics Chapterwise

Companies are spending billions on machine learning projects, but it's money wasted if the models can't be deployed effectively. In this practical guide, Hannes Hapke and Catherine Nelson walk you through the steps of automating a machine learning pipeline using the TensorFlow ecosystem. You'll learn the techniques and tools that will cut deployment time from days to minutes, so that you can focus on developing new models rather than maintaining legacy systems. Data scientists, machine learning engineers, and DevOps engineers will discover how to go beyond model development to successfully productize their data science projects, while managers will better understand the role they play in helping to accelerate these projects. The book also explores new approaches for integrating data privacy into machine learning pipelines. Understand the machine learning management lifecycle Implement data pipelines with Apache Airflow and Kubeflow Pipelines Work with data using TensorFlow tools like ML Metadata, TensorFlow Data Validation, and TensorFlow Transform Analyze models with TensorFlow Model Analysis and ship them with the TFX Model Pusher Component after the ModelValidator TFX Component confirmed that the analysis results are an improvement Deploy models in a variety of environments with TensorFlow Serving, TensorFlow Lite, and TensorFlow.js Learn methods for adding privacy, including differential privacy with TensorFlow Privacy and federated learning with TensorFlow Federated Design model feedback loops to increase your data sets and learn when to update your machine learning models Copyright Table of Contents Foreword Preface What Are Machine Learning Pipelines? Who Is This Book For? Why TensorFlow and TensorFlow Extended? Overview of the Chapters Conventions Used in This Book Using Code Examples O’Reilly Online Learning How to Contact Us Acknowledgments Chapter 1. Introduction Why Machine Learning Pipelines? When to Think About Machine Learning Pipelines Overview of the Steps in a Machine Learning Pipeline Data Ingestion and Data Versioning Data Validation Data Preprocessing Model Training and Tuning Model Analysis Model Versioning Model Deployment Feedback Loops Data Privacy Pipeline Orchestration Why Pipeline Orchestration? Directed Acyclic Graphs Our Example Project Project Structure Our Machine Learning Model Goal of the Example Project Summary Chapter 2. Introduction to TensorFlow Extended What Is TFX? Installing TFX Overview of TFX Components What Is ML Metadata? Interactive Pipelines Alternatives to TFX Introduction to Apache Beam Setup Basic Data Pipeline Executing Your Basic Pipeline Summary Chapter 3. Data Ingestion Concepts for Data Ingestion Ingesting Local Data Files Ingesting Remote Data Files Ingesting Data Directly from Databases Data Preparation Splitting Datasets Spanning Datasets Versioning Datasets Ingestion Strategies Structured Data Text Data for Natural Language Problems Image Data for Computer Vision Problems Summary Chapter 4. Data Validation Why Data Validation? TFDV Installation Generating Statistics from Your Data Generating Schema from Your Data Recognizing Problems in Your Data Comparing Datasets Updating the Schema Data Skew and Drift Biased Datasets Slicing Data in TFDV Processing Large Datasets with GCP Integrating TFDV into Your Machine Learning Pipeline Summary Chapter 5. Data Preprocessing Why Data Preprocessing? Preprocessing the Data in the Context of the Entire Dataset Scaling the Preprocessing Steps Avoiding a Training-Serving Skew Deploying Preprocessing Steps and the ML Model as One Artifact Checking Your Preprocessing Results in Your Pipeline Data Preprocessing with TFT Installation Preprocessing Strategies Best Practices TFT Functions Standalone Execution of TFT Integrate TFT into Your Machine Learning Pipeline Summary Chapter 6. Model Training Defining the Model for Our Example Project The TFX Trainer Component run_fn() Function Running the Trainer Component Other Trainer Component Considerations Using TensorBoard in an Interactive Pipeline Distribution Strategies Model Tuning Strategies for Hyperparameter Tuning Hyperparameter Tuning in TFX Pipelines Summary Chapter 7. Model Analysis and Validation How to Analyze Your Model Classification Metrics Regression Metrics TensorFlow Model Analysis Analyzing a Single Model in TFMA Analyzing Multiple Models in TFMA Model Analysis for Fairness Slicing Model Predictions in TFMA Checking Decision Thresholds with Fairness Indicators Going Deeper with the What-If Tool Model Explainability Generating Explanations with the WIT Other Explainability Techniques Analysis and Validation in TFX ResolverNode Evaluator Component Validation in the Evaluator Component TFX Pusher Component Summary Chapter 8. Model Deployment with TensorFlow Serving A Simple Model Server The Downside of Model Deployments with Python-Based APIs Lack of Code Separation Lack of Model Version Control Inefficient Model Inference TensorFlow Serving TensorFlow Architecture Overview Exporting Models for TensorFlow Serving Model Signatures Signature Methods Inspecting Exported Models Inspecting the Model Testing the Model Setting Up TensorFlow Serving Docker Installation Native Ubuntu Installation Building TensorFlow Serving from Source Configuring a TensorFlow Server Single Model Configuration Multiple Model Configuration REST Versus gRPC REST gRPC Making Predictions from the Model Server Getting Model Predictions via REST Using TensorFlow Serving via gRPC Model A/B Testing with TensorFlow Serving Requesting Model Metadata from the Model Server REST Requests for Model Metadata gRPC Requests for Model Metadata Batching Inference Requests Configuring Batch Predictions Other TensorFlow Serving Optimizations TensorFlow Serving Alternatives BentoML Seldon GraphPipe Simple TensorFlow Serving MLflow Ray Serve Deploying with Cloud Providers Use Cases Example Deployment with GCP Model Deployment with TFX Pipelines Summary Chapter 9. Advanced Model Deployments with TensorFlow Serving Decoupling Deployment Cycles Workflow Overview Optimization of Remote Model Loading Model Optimizations for Deployments Quantization Pruning Distillation Using TensorRT with TensorFlow Serving TFLite Steps to Optimize Your Model with TFLite Serving TFLite Models with TensorFlow Serving Monitoring Your TensorFlow Serving Instances Prometheus Setup TensorFlow Serving Configuration Simple Scaling with TensorFlow Serving and Kubernetes Summary Chapter 10. Advanced TensorFlow Extended Advanced Pipeline Concepts Training Multiple Models Simultaneously Exporting TFLite Models Warm Starting Model Training Human in the Loop Slack Component Setup How to Use the Slack Component Custom TFX Components Use Cases of Custom Components Writing a Custom Component from Scratch Reusing Existing Components Summary Chapter 11. Pipelines Part 1: Apache Beam and Apache Airflow Which Orchestration Tool to Choose? Apache Beam Apache Airflow Kubeflow Pipelines Kubeflow Pipelines on AI Platform Converting Your Interactive TFX Pipeline to a Production Pipeline Simple Interactive Pipeline Conversion for Beam and Airflow Introduction to Apache Beam Orchestrating TFX Pipelines with Apache Beam Introduction to Apache Airflow Installation and Initial Setup Basic Airflow Example Orchestrating TFX Pipelines with Apache Airflow Pipeline Setup Pipeline Execution Summary Chapter 12. Pipelines Part 2: Kubeflow Pipelines Introduction to Kubeflow Pipelines Installation and Initial Setup Accessing Your Kubeflow Pipelines Installation Orchestrating TFX Pipelines with Kubeflow Pipelines Pipeline Setup Executing the Pipeline Useful Features of Kubeflow Pipelines Pipelines Based on Google Cloud AI Platform Pipeline Setup TFX Pipeline Setup Pipeline Execution Summary Chapter 13. Feedback Loops Explicit and Implicit Feedback The Data Flywheel Feedback Loops in the Real World Design Patterns for Collecting Feedback Users Take Some Action as a Result of the Prediction Users Rate the Quality of the Prediction Users Correct the Prediction Crowdsourcing the Annotations Expert Annotations Producing Feedback Automatically How to Track Feedback Loops Tracking Explicit Feedback Tracking Implicit Feedback Summary Chapter 14. Data Privacy for Machine Learning Data Privacy Issues Why Do We Care About Data Privacy? The Simplest Way to Increase Privacy What Data Needs to Be Kept Private? Differential Privacy Local and Global Differential Privacy Epsilon, Delta, and the Privacy Budget Differential Privacy for Machine Learning Introduction to TensorFlow Privacy Training with a Differentially Private Optimizer Calculating Epsilon Federated Learning Federated Learning in TensorFlow Encrypted Machine Learning Encrypted Model Training Converting a Trained Model to Serve Encrypted Predictions Other Methods for Data Privacy Summary Chapter 15. The Future of Pipelines and Next Steps Model Experiment Tracking Thoughts on Model Release Management Future Pipeline Capabilities TFX with Other Machine Learning Frameworks Testing Machine Learning Models CI/CD Systems for Machine Learning Machine Learning Engineering Community Summary Appendix A. Introduction to Infrastructure for Machine Learning What Is a Container? Introduction to Docker Introduction to Docker Images Building Your First Docker Image Diving into the Docker CLI Introduction to Kubernetes Some Kubernetes Definitions Getting Started with Minikube and kubectl Interacting with the Kubernetes CLI Defining a Kubernetes Resource Deploying Applications to Kubernetes Appendix B. Setting Up a Kubernetes Cluster on Google Cloud Before You Get Started Kubernetes on Google Cloud Selecting a Google Cloud Project Setting Up Your Google Cloud Project Creating a Kubernetes Cluster Accessing Your Kubernetes Cluster with kubectl Using Your Kubernetes Cluster with kubectl Persistent Volume Setups for Kubeflow Pipelines Appendix C. Tips for Operating Kubeflow Pipelines Custom TFX Images Exchange Data Through Persistent Volumes TFX Command-Line Interface TFX and Its Dependencies TFX Templates Publishing Your Pipeline with TFX CLI Index About the Authors Colophon Companies are spending billions on machine learning projects, but it's money wasted if the models can't be deployed effectively. In this practical guide, Hannes Hapke and Catherine Nelson walk you through the steps of automating a machine learning pipeline using the TensorFlow ecosystem. You'll learn the techniques and tools that will cut deployment time from days to minutes, so that you can focus on developing new models rather than maintaining legacy systems. Data scientists, machine learning engineers, and DevOps engineers will discover how to go beyond model development to successfully productize their data science projects, while managers will better understand the role they play in helping to accelerate these projects. The book also explores new approaches for integrating data privacy into machine learning pipelines. Understand the machine learning management lifecycle Implement data pipelines ; Build your pipeline using components from TensorFlow extended ; Orchestrate your machine learning pipeline with Apache Beam, Apache Airflow, and Kubeflow Data Validation and TensorFlow Transform ; Analyze a model in detail using TensorFlow model analysis ; Examine fairness and bias in your model performance ; Deploy models with TensorFlow serving or TensorFlow Lite for mobile devices ; Learn privacy-preserving machine learning techniques Companies are spending billions on machine learning projects, but it's money wasted if the models can't be deployed effectively. In this practical guide, Hannes Hapke and Catherine Nelson walk you through the steps of automating a machine learning pipeline using the TensorFlow ecosystem. You'll learn the techniques and tools that will cut deployment time from days to minutes, so that you can focus on developing new models rather than maintaining legacy systems. Data scientists, machine learning engineers, and DevOps engineers will discover how to go beyond model development to successfully productize their data science projects, while managers will better understand the role they play in helping to accelerate these projects. • Understand the steps that make up a machine learning pipeline • Build your pipeline using components from TensorFlow Extended • Orchestrate your machine learning pipeline with Apache Beam, Apache Airflow and Kubeflow Pipelines • Work with data using TensorFlow Data Validation and TensorFlow Transform • Analyze a model in detail using TensorFlow Model Analysis • Examine fairness and bias in your model performance • Deploy models with TensorFlow Serving or convert them to TensorFlow Lite for mobile devices • Understand privacy-preserving machine learning techniques Companies are spending billions on machine learning projects, but it's money wasted if the models can't be deployed effectively. In this practical guide, Hannes Hapke and Catherine Nelson walk you through the steps of automating a machine learning pipeline using the TensorFlow ecosystem. You'll learn the techniques and tools that will cut deployment time from days to minutes, so that you can focus on developing new models rather than maintaining legacy systems.Data scientists, machine learning engineers, and DevOps engineers will discover how to go beyond model development to successfully productize their data science projects, while managers will better understand the role they play in helping to accelerate these projects.Understand the steps to build a machine learning pipelineBuild your pipeline using components from TensorFlow ExtendedOrchestrate your machine learning pipeline with Apache Beam, Apache Airflow, and Kubeflow PipelinesWork with data using TensorFlow Data Validation and TensorFlow TransformAnalyze a model in detail using TensorFlow Model AnalysisExamine fairness and bias in your model performanceDeploy models with TensorFlow Serving or TensorFlow Lite for mobile devicesLearn privacy-preserving machine learning techniques